Forecasting causal dynamics with universal reservoirs

Abstract: An iterated multistep forecasting scheme based on recurrent neural networks (RNN) is proposed for the time series generated by causal chains with infinite memory. This forecasting strategy contains, as a particular case, the iterative prediction strategies for dynamical systems that are customary in reservoir computing. Explicit error bounds are obtained as a function of the forecasting horizon, functional and dynamical features of the specific RNN used, and the approximation error committed by it. In particular, the growth rate of the error is shown to be exponential and controlled by the top Lyapunov exponent of the proxy system. The framework in the paper circumvents difficult-to-verify embedding hypotheses that appear in previous references in the literature and applies to new situations like the finite-dimensional observations of functional differential equations or the deterministic parts of stochastic processes to which standard embedding techniques do not necessarily apply.