Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping (2305.10721v1)

Abstract: Long-term time series forecasting has gained significant attention in recent years. While there are various specialized designs for capturing temporal dependency, previous studies have demonstrated that a single linear layer can achieve competitive forecasting performance compared to other complex architectures. In this paper, we thoroughly investigate the intrinsic effectiveness of recent approaches and make three key observations: 1) linear mapping is critical to prior long-term time series forecasting efforts; 2) RevIN (reversible normalization) and CI (Channel Independent) play a vital role in improving overall forecasting performance; and 3) linear mapping can effectively capture periodic features in time series and has robustness for different periods across channels when increasing the input horizon. We provide theoretical and experimental explanations to support our findings and also discuss the limitations and future works. Our framework's code is available at \url{https://github.com/plumprc/RTSF}.

• The paper demonstrates that linear mapping is a critical factor driving forecasting accuracy, even rivaling sophisticated architectures.
• The paper shows that normalization techniques like RevIN and CI significantly enhance the model’s ability to capture periodic features.
• The paper validates that extending the input horizon boosts performance on multivariate datasets with diverse seasonal patterns.

Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping

The recent work, "Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mapping," provides a comprehensive analysis of the efficacy of linear models in the domain of long-term time series forecasting. The paper explores the potential of simple linear layers compared to more sophisticated architectures such as those based on RNN, CNN, and Transformer models. This investigation is centered on the capability of linear models to capture periodic features inherent in time series data and examines the role of normalization techniques in enhancing forecasting performance.

The authors present three pivotal observations:

1. Critical Role of Linear Mapping: The paper finds that linear mapping significantly contributes to the forecasting capabilities of current state-of-the-art models, sometimes surpassing the complex architectures they employ.
2. Enhancement through RevIN and CI: Reversible normalization (RevIN) and Channel Independent (CI) approaches are identified as key elements that enhance forecasting accuracy, particularly by facilitating the learning of periodic features.
3. Robustness to Different Periodic Features: Linear mapping demonstrates an inherent robustness to varying periods across channels, particularly when the input horizon is extended. This suggests that linear models can handle multivariate time series data with distinct periodic channels effectively.

Theoretical analysis supports these findings, indicating that even a single linear layer can inherently decompose time series into seasonal and trend components. It is shown that linear models can accurately predict periodic signals when the input sequence length accommodates the period of the data. Furthermore, a theoretical upper bound on the performance of linear models for mixed seasonal and trend components is provided, highlighting the limitations inherent in linear models when addressing non-periodic components.

Empirical evaluations substantiate the theoretical results on both simulated and six public real-world datasets: ETTh1, ETTh2, ETTm1, ETTm2, Weather, and ECL. The results reveal that the simple RLinear model, when enhanced with RevIN, often achieves comparable or superior performance to more complex baselines like PatchTST, TimesNet, and DLinear. Notably, much of the success attributed to these advanced models is, in fact, a function of their linear mapping and normalization techniques rather than their sophisticated architecture.

The paper further examines how the linear models cope with the challenges posed by multichannel datasets with different periodic channels. It suggests that the performance can vary due to the mismatch in channel periods, and highlights that using CI modeling or integrating simple nonlinear units can mitigate these issues.

Moreover, the research emphasizes that increasing the input horizon can improve forecasting performance by extending coverage over the various periods present in the data. However, the extent of this improvement appears to be limited, depending significantly on the periodic characteristics of the datasets.

The investigation concludes with a note on the future directions for research, pointing out the applicability of these findings to short-term forecasting tasks and potential adjustments for datasets where seasonal patterns vary more dramatically. Overall, this work challenges the conventional reliance on complex architectures and underscores the latent potential of simpler, statistically-sound solutions in time series forecasting.

This paper contributes to a nuanced understanding of time series forecasting, emphasizing the often-overlooked effectiveness of linear layers, especially when supplemented by well-designed normalization techniques. It poses significant implications for the development and benchmark evaluation of forecasting models, encouraging a reevaluation of the significance of simplicity and statistical soundness in model design.