Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions (1909.00590v5)

Abstract: Recurrent Neural Networks (RNN) have become competitive forecasting methods, as most notably shown in the winning method of the recent M4 competition. However, established statistical models such as ETS and ARIMA gain their popularity not only from their high accuracy, but they are also suitable for non-expert users as they are robust, efficient, and automatic. In these areas, RNNs have still a long way to go. We present an extensive empirical study and an open-source software framework of existing RNN architectures for forecasting, that allow us to develop guidelines and best practices for their use. For example, we conclude that RNNs are capable of modelling seasonality directly if the series in the dataset possess homogeneous seasonal patterns, otherwise we recommend a deseasonalization step. Comparisons against ETS and ARIMA demonstrate that the implemented (semi-)automatic RNN models are no silver bullets, but they are competitive alternatives in many situations.

• The paper demonstrates that RNN models with robust architectures and preprocessing can outperform traditional methods in specific forecasting tasks.
• The study systematically compares RNN variants like LSTM and GRU, highlighting how techniques such as deseasonalization and optimizer selection impact performance.
• The research underlines the potential of global modeling to leverage cross-series information while addressing computational challenges and motivating future hybrid approaches.

Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions

The paper authored by Hansika Hewamalage, Christoph Bergmeir, and Kasun Bandara critically examines the role of Recurrent Neural Networks (RNNs) in time series forecasting, evaluating their competitive position against established statistical methods like ETS and ARIMA. The research offers a systematic evaluation of RNN architectures, complemented by robust empirical studies, to delineate guidelines and best practices for applying RNNs in forecasting tasks.

Key Aspects of the Study

1. Motivation and Context: The research acknowledges the challenge that RNNs face in practical forecasting scenarios, especially when compared to robust, efficient univariate statistical models that are user-friendly for non-experts. Despite RNNs' success in contests like the M4 competition, there remains a gap in understanding their full potential in automated settings without expert intervention.
2. Methodological Framework:
   • The paper employs a comprehensive range of RNN architectures, including Stacked, Sequence to Sequence (S2S) with Dense Layer, and others, integrating three types of recurrent units: Elman, LSTM with peephole connections, and GRU.
   • Robust data preprocessing techniques such as STL decomposition for seasonality and local trend normalization are examined. The experimentation assesses deseasonalization's impact, revealing that RNNs can model seasonality independently if the series share homogeneous seasonal characteristics.
   • Multiple optimizers, including the COCOB optimizer, are evaluated, with COCOB showing promise by removing the dependency on initial learning rate settings.
3. Performance Insights:
   • Through empirical comparisons across diverse datasets, the paper surfaces critical insights: RNNs, particularly in a semi-automatic setup, can outperform traditional models in certain datasets like NN5 and Wikipedia Web Traffic datasets.
   • Analysis of computational costs indicates that while RNNs are resource-intensive, their accuracy gains justify their deployment in modern computational infrastructures.
4. Statistical Significance Testing: Utilizing non-parametric tests like the Friedman rank-sum and Hochberg's post hoc procedure, the paper rigorously assesses the statistical significance of the findings. Although RNNs demonstrate impressive performance in specific contexts, they do not universally outperform ETS and ARIMA across all datasets tested.
5. Applications and Global Models: Emphasizing global modeling approaches, the research illustrates RNNs' potential in leveraging cross-series information—a critical asset when handling time series with shared features across datasets.

Implications and Future Research Directions

The research provides a substantial foundation for advancing RNN applications in time series forecasting. It underlines the potential improvements in accuracy by distinguishing when and how to harness RNN architectures effectively. However, challenges remain in enhancing model interpretability and reducing dependency on extensive computational resources.

Looking forward, the paper suggests several pathways:

• Exploration of probabilistic forecasting to handle prediction uncertainties.
• Integration of CNN architectures like Temporal Convolution Networks (TCNs) that promise further improvements in computational efficiency and capacity to model complex dependencies.
• Development of hybrid models combining local and global parameters to accommodate time series with varied characteristics.

In conclusion, while the research marks significant progress in deploying RNNs for time series forecasting, it also highlights the nuanced understanding required for practitioners to transition from statistical models to more sophisticated RNN-based approaches, ensuring practical applicability and scalability. The paper contributes not only by providing competitive benchmarks but also by setting a framework for systematic exploration within this domain.