TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

Abstract: Recently, multivariate time series forecasting tasks have garnered increasing attention due to their significant practical applications, leading to the emergence of various deep forecasting models. However, real-world time series exhibit pronounced non-stationary distribution characteristics. These characteristics are not solely limited to time-varying statistical properties highlighted by non-stationary Transformer but also encompass three key aspects: nested periodicity, absence of periodic distributions, and hysteresis among time variables. In this paper, we begin by validating this theory through wavelet analysis and propose the Transformer-based TwinS model, which consists of three modules to address the non-stationary periodic distributions: Wavelet Convolution, Period-Aware Attention, and Channel-Temporal Mixed MLP. Specifically, The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet transform. The Period-Aware Attention guides attention computation by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed MLP captures the overall relationships between time series through channel-time mixing learning. TwinS achieves SOTA performance compared to mainstream TS models, with a maximum improvement in MSE of 25.8\% over PatchTST.

• The paper introduces the TwinS model that leverages wavelet convolution, period-aware attention, and a mixed MLP to capture non-stationary periodic patterns in time series data.
• Experimental results reveal up to a 25.8% reduction in MSE on benchmark datasets, outperforming eight mainstream forecasting models.
• The approach offers robust forecasting for complex temporal dynamics in domains like weather, traffic, and finance, paving the way for future advancements.

TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

Introduction

The paper "TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting" (2406.03710) presents a novel approach addressing the challenges of non-stationary distributions in real-world multivariate time series (MTS) data. This approach is centered on the TwinS model, integrating complex modules designed to capture and model non-stationary periodic distributions inherent in these data. The proposed solution is crucial for applications such as weather prediction, traffic forecasting, and financial analysis, where the dynamic, nested, and overlapping nature of time series presents significant forecasting challenges.

Key Contributions

The TwinS model introduces three specialized modules: Wavelet Convolution, Period-Aware Attention, and Channel-Temporal Mixed MLP. These components work collectively to address the non-stationary characteristics of MTS:

1. Wavelet Convolution Module: Inspired by wavelet transforms, this module efficiently extracts embedded periods by varying convolution kernel sizes to capture different frequency components in the time series.
2. Period-Aware Attention: Utilizing a convolutional sub-network, this attention mechanism decouples nested periods and manages missing states in periodic distributions, thereby enhancing the model's capacity to recognize intricate periodic patterns.
3. Channel-Temporal Mixed MLP: This component effectively learns the overall relationships between temporal variables by integrating channel and time domain dependencies, ameliorating the hierarchical interdependencies typical in MTS.

Methodology

TwinS is designed to improve upon limitations seen in earlier models that fail to effectively decouple temporal and frequency domain information in nested period structures and cannot adequately capture the hysteresis effects among time variables. Utilizing wavelet convolution offers a precise extraction of non-stationary frequency domain features, while the novel Period-Aware Attention mechanism enables the selective focus on relevant time patches, adjusting attention scores adaptively. This enhances the robustness and responsiveness of the TwinS model to non-stationary characteristics prevalent in MTS data.

Experimental Results

The TwinS model demonstrates SOTA (state-of-the-art) performance across standard benchmark datasets, surpassing eight mainstream time series forecasting models with significant margins—in some cases achieving up to a 25.8% reduction in Mean Squared Error (MSE) compared to previous models like PatchTST. Notably, the model shows exceptional robustness in datasets characterized by pronounced non-stationary periodic distributions.

Implications for Future Research

The TwinS framework establishes a critical step toward understanding and modeling complex non-stationary features in time series data. Its architecture opens new pathways for designing forecasting models that are not only effective in capturing dynamic patterns but are also resource-efficient in computation. Future research may focus on further optimizing period detection and adaptability in Model architectures, potentially expanding the applicability of these methods beyond time series to other domains dealing with dynamic data distributions.

Conclusion

The innovative TwinS model successfully addresses key issues related to non-stationarity in multivariate time series forecasting. By implementing wavelet-based convolution and period-aware attention, the model captures complex dynamic patterns with notable efficiency and accuracy. This work not only advances the understanding of non-stationary periodic distributions but also sets a new benchmark for future developments in time series analysis and forecasting.