An Experimental Review on Deep Learning Architectures for Time Series Forecasting (2103.12057v2)

Abstract: In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting; and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50000 time series divided into 12 different forecasting problems. By training more than 38000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.

• The paper demonstrates that LSTM networks achieve superior accuracy with fewer layers compared to other models.
• The study shows that CNN models offer a balance between computational efficiency and forecasting accuracy, making them ideal for real-time applications.
• Extensive experiments across 12 datasets emphasize the need for optimal hyperparameter tuning in deep learning architectures for time series forecasting.

An Experimental Review on Deep Learning Architectures for Time Series Forecasting

The paper "An Experimental Review on Deep Learning Architectures for Time Series Forecasting" presents a comparative analysis of several prominent deep learning models applied to time series forecasting (TSF), namely, multilayer perceptron (MLP), Elman recurrent neural network (ERNN), long short-term memory (LSTM), gated recurrent unit (GRU), echo state network (ESN), convolutional neural network (CNN), and temporal convolutional network (TCN). The authors, Pedro Lara-Benítez, Manuel Carranza-García, and José C. Riquelme, focus on evaluating these models in terms of accuracy and computational efficiency across 12 different datasets, comprising over 50,000 time series instances and involving extensive experimental setups with more than 38,000 model configurations.

Among the critical points in the findings, LSTM networks demonstrate superior accuracy compared to other models in this paper, not only offering the best performance for several datasets but also maintaining a strong consistency across various configurations. GRU models follow closely, showing a comparable predictive capability, though with slightly less pronounced results. CNN architectures are highlighted as providing a favorable balance between computational efficiency and forecasting accuracy. This dual advantage positions CNNs as attractive candidates for real-time applications, where responsive performance is crucial.

The paper also investigates the optimal architectural parameters of each model. Significant results indicate that LSTM models tend to perform better with fewer layers, capturing the necessary temporal dependencies without the added complexity of deeper networks. On the other hand, CNNs require multiple layers to effectively process the input data but benefit significantly from configurations without pooling operations, which seems counterintuitive given their traditional use in image processing tasks.

The paper further uncovers that TCN architectures, although designed explicitly with time-dependent data in mind, do not outperform conventional CNNs significantly in terms of accuracy but exhibit higher computational demands. This outcome suggests potential areas for further refinement in TCN design for TSF.

The practical implications of these findings are noteworthy for researchers and practitioners in machine learning and related fields. The paper provides insights into model selection based on application-specific requirements, emphasizing the importance of considering both accuracy and computational overhead in real-world settings. For instance, while LSTMs are suitable for tasks with less stringent latency constraints, CNNs might be preferable for applications demanding faster inference times without significant trade-offs in accuracy.

From a theoretical standpoint, the exploration into hyperparameter configurations reveals the importance of an adequate model design tailored to the characteristics of the dataset. The statistically insignificant differences amongst several parameters call for devising better-optimized, perhaps automated, methods for hyperparameter tuning that align with the dynamic nature of streaming time series data.

The extensive empirical evaluations and comparisons offered in this paper contribute to the broader understanding of how various deep learning architectures perform under diverse TSF conditions. Moreover, these insights pave the way for future research exploring hybrid models, transfer learning, and scalable solutions capable of handling multivariate and more complex data scenarios. The authors anticipate that future work might expand on this experimental foundation, investigating novel architectures and ensemble approaches that continue to push the boundaries of predictive proficiency and operational efficiency in time series forecasting tasks.