Sequential Reservoir Computing for Efficient High-Dimensional Spatiotemporal Forecasting (2601.00172v1)

Abstract: Forecasting high-dimensional spatiotemporal systems remains computationally challenging for recurrent neural networks (RNNs) and long short-term memory (LSTM) models due to gradient-based training and memory bottlenecks. Reservoir Computing (RC) mitigates these challenges by replacing backpropagation with fixed recurrent layers and a convex readout optimization, yet conventional RC architectures still scale poorly with input dimensionality. We introduce a Sequential Reservoir Computing (Sequential RC) architecture that decomposes a large reservoir into a series of smaller, interconnected reservoirs. This design reduces memory and computational costs while preserving long-term temporal dependencies. Using both low-dimensional chaotic systems (Lorenz63) and high-dimensional physical simulations (2D vorticity and shallow-water equations), Sequential RC achieves 15-25% longer valid forecast horizons, 20-30% lower error metrics (SSIM, RMSE), and up to three orders of magnitude lower training cost compared to LSTM and standard RNN baselines. The results demonstrate that Sequential RC maintains the simplicity and efficiency of conventional RC while achieving superior scalability for high-dimensional dynamical systems. This approach provides a practical path toward real-time, energy-efficient forecasting in scientific and engineering applications.

• The paper presents Sequential Reservoir Computing as a scalable framework that segments a large reservoir into interconnected units for efficient forecasting.
• It demonstrates enhanced accuracy and prolonged valid prediction times over both low- and high-dimensional datasets while significantly reducing computational costs.
• The methodology leverages sequential reservoir layers to optimize memory usage and computational efficiency in modeling complex spatiotemporal dynamics.

Sequential Reservoir Computing for Efficient High-Dimensional Spatiotemporal Forecasting

Introduction

Recent advancements in machine learning have transformed scientific research, enabling the modeling and prediction of complex spatiotemporal systems. Traditional RNNs and LSTMs face significant challenges in this regard due to gradient-based training complexities and large memory requirements. Reservoir Computing (RC) offers an alternative by leveraging fixed recurrent layers paired with efficient readout optimization methods, but conventional RC architectures do not scale well to high input dimensions.

The paper introduces a Sequential Reservoir Computing (Sequential RC) framework designed to improve upon these limitations by segmenting a large reservoir into interconnected smaller units. This approach aims to optimize both memory and computational efficiency while maintaining the capability to capture long-term temporal dependencies. This framework demonstrates enhanced scalability and decreased computational demands for high-dimensional datasets, preserving the inherent simplicity of RC while substantively extending its applicability.

Methodology

Recurrent Neural Networks

Elman-type RNNs serve as the baseline by utilizing recurrent connections within hidden layers to model sequential data. Despite their architectural capacity to learn time-dependent patterns through recurrence and context units, these models struggle with long-term dependencies primarily due to gradient degradation.

Long Short-Term Memory Networks

LSTMs introduce gating mechanisms that regulate information flow, mitigating the vanishing and exploding gradient issues associated with traditional RNNs. By incorporating input, forget, and output gates, along with cell states functioning as long-term memory, LSTMs have improved training robustness across longer sequences.

Reservoir Computing

RC simplifies the network by fixing the parameters of the input and recurrent layers and training only the readout layer through ridge regression. This architecture allows RC models to efficiently capture temporal dependencies without extensive training computations, although scalability remains a challenge due to reservoir size and its memory requirements.

Figure 1: The schematic configuration of RC model architecture. The left-side blue nodes and right-side red nodes represent input and target variables, respectively.

Sequential Reservoir Computing

The Sequential RC framework extends typical RC by introducing multiple, smaller reservoirs connected in sequence, each enhancing the previous layer's representations. Each reservoir processes temporal features and feeds them into a readout layer for final predictions. This sequential structure balances complexity and resource efficiency, optimizing training for high-dimensional datasets.

Figure 2: The schematic configuration of the Sequential RC model architecture. Outputs of each reservoir layer along with input data to the readout layer.

Results and Discussion

Low-Dimensional Forecasting

Experimental results on the low-dimensional Lorenz63 system reveal that Sequential RC outperforms RNNs and LSTMs in terms of forecasting accuracy and valid prediction time (VPT). The release from extensive backpropagation computations allows Sequential RC to maintain superior forecasting horizons with notable reductions in parameter count and memory usage.

Figure 3: Forecasts based on 2,000 training samples, illustrating model performance for low-dimensional data.

High-Dimensional Forecasting

Sequential RC demonstrated substantial improvements in high-dimensional systems, including vorticity and shallow-water simulations. Compared to baseline models, the Sequential RC provided extended forecast horizons and improved error metrics (SSIM, PSNR), underscoring its capability to handle complex spatiotemporal interactions efficiently.

Figure 4: Comparison of Lorenz attractors for different models, illustrating trajectory learning capabilities.

Computational Efficiency

Sequential RC exhibits notable computational advantages. It significantly reduces FLOPS and required memory, supporting its scalability for real-time applications in scientific computing. The framework's efficiency also facilitates deployment on resource-constrained hardware environments.

Conclusion

The introduction of Sequential RC addresses key limitations of conventional RC architectures by offering a practical and scalable approach to high-dimensional spatiotemporal forecasting. Through this framework, users can achieve lower computational costs and improved accuracy in real-time applications, spanning areas such as meteorology, oceanography, and environmental monitoring. Future work may explore adaptive connections and potential implementations on neuromorphic or FPGA hardware to further enhance the framework's versatility and efficiency.