DA-CAE-ESN: Nonlinear Reduced-Order Modeling

• DA-CAE-ESN is a reduced-order modeling framework that combines a convolutional autoencoder for spatial compression with an echo state network for temporal dynamics.
• It effectively captures forecasting accuracy and invariant stability properties in chaotic PDEs, as demonstrated on Kolmogorov and Kuramoto-Sivashinsky flows.
• The framework offers orders-of-magnitude speedup over direct simulations while preserving key dynamical insights for both short-term prediction and long-term statistical consistency.

Searching arXiv for “DA-CAE-ESN” and closely related terms to ground the article in relevant papers. DA-CAE-ESN—Editor’s term for the divide-and-conquer use of the convolutional autoencoder echo state network (CAE-ESN)—denotes a latent-space reduced-order modelling strategy in which a convolutional autoencoder (CAE) provides the spatial decomposition of a high-dimensional state into a low-dimensional manifold and an echo state network (ESN) provides the temporal decomposition by advancing the latent variables autonomously. In the cited literature, this framework is introduced for two-dimensional Kolmogorov flow and then extended from forecasting to the inference of Lyapunov exponents, covariant Lyapunov vectors, and attractor geometry in chaotic PDEs and turbulent flows (Racca et al., 2022).

1. Definition, scope, and nomenclature

The CAE-ESN is a two-part reduced-order modelling framework designed for spatiotemporal chaotic dynamics. Its central decomposition is explicit: the turbulent or chaotic state is divided into a spatial problem and a temporal problem. The CAE computes the latent space, described as the manifold onto which the dynamics live, and the ESN predicts the time evolution in that latent space. Assembled together, the two components form an autonomous dynamical system that acts as a reduced-order model (Racca et al., 2022).

Later work places this construction within a broader “divide-and-conquer strategy” for data-driven solutions of partial differential equations. In that formulation, high-dimensional observations are first compressed to a latent space by an autoencoder and then evolved in time by a recurrent architecture, specifically the ESN in the CAE-ESN variant (Özalp et al., 2024).

Paper	System	Reported contribution
(Racca et al., 2022)	2D Kolmogorov flow	Forecasting and reduced-order modelling
(Özalp et al., 2024)	Kuramoto-Sivashinsky and Kolmogorov flow	Stability analysis in latent space
(Özalp et al., 2024)	Kuramoto-Sivashinsky equation	Inference of invariant stability properties

This terminology matters because the framework is not merely a compressor followed by a predictor. The cited papers treat the latent space as a low-dimensional manifold that preserves essential dynamical structure, including, in later work, tangent-space geometry and invariant stability quantities. A plausible implication is that the method occupies an intermediate position between classical reduced-order modelling and nonlinear latent-dynamics learning.

2. Architectural composition and mathematical formulation

The CAE stage maps the physical state $\mathbf{q} \in \mathbb{R}^{N_{\text{phys}}}$ to a latent vector $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$, with $N_{\text{lat}} \ll N_{\text{phys}}$, through nonlinear convolutional filtering. The decoder reconstructs the physical field from the latent code. In the Kolmogorov-flow implementation, the encoder uses multiple parallel paths with kernel sizes $3 \times 3$, $5 \times 5$, and $7 \times 7$ to extract multiscale spatial features, and the autoencoder is trained by minimizing a mean-squared reconstruction loss normalized per degree of freedom (Racca et al., 2022).

The encoder-decoder relation is written as

$$\hat{\mathbf{q}} = \mathbf{f}(\mathbf{z}), \qquad \mathbf{z} = \mathbf{g}(\mathbf{q}),$$

with training loss

$$\mathcal{L} = \sum_{i=1}^{N_t} \frac{1}{N_t N_{\text{phys}}}\left\Vert \hat{\mathbf{q}}(t_i)-\mathbf{q}(t_i)\right\Vert^2.$$

The ESN then advances the latent state in time by passing $\mathbf{z}(t_i)$ through a high-dimensional fixed random reservoir. Only the output weights are trained, typically by ridge regression, which avoids backpropagation through time. In the formulations reported for latent-space chaotic modelling, the reservoir update and readout are

$$\mathbf{r}(t_{i+1}) = \tanh\left(\mathbf{W}_{\text{in}}[\tilde{\mathbf{z}}(t_i); b_{\text{in}}] + \mathbf{W}\mathbf{r}(t_i)\right),$$

$\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$0

with $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$1 obtained from ridge regression with Tikhonov regularization parameter $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$2 (Racca et al., 2022). Later descriptions distinguish open-loop operation during training from closed-loop autonomous rollout during prediction (Özalp et al., 2024).

Hyperparameters such as input scaling, spectral radius, and noise are selected via Bayesian or grid search, and reported reservoir sizes range from $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$3 to $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$4 neurons (Racca et al., 2022). This division of labor—CAE for spatial structure, ESN for temporal recurrence—is the defining architectural principle of DA-CAE-ESN.

3. Latent-space reduction and nonlinear reduced-order representation

In the original turbulent-flow study, the physical state has $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$5 degrees of freedom, while tested latent spaces range from $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$6 to $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$7 variables, with the paper emphasizing that the learned representation uses less than $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$8 of the degrees of freedom of the physical space (Racca et al., 2022). The same work reports that the CAE achieves a reduction of roughly $\mathbf{z} \in \mathbb{R}^{N_{\text{lat}}}$9–$N_{\text{lat}} \ll N_{\text{phys}}$0 of the full-order model size while preserving accuracy.

For $N_{\text{lat}} \ll N_{\text{phys}}$1 latent variables, the reported captured energy is $N_{\text{lat}} \ll N_{\text{phys}}$2 in the quasiperiodic regime and $N_{\text{lat}} \ll N_{\text{phys}}$3 in the turbulent regime. The paper also reports attractor dimensions, estimated by the Kaplan–Yorke method, of approximately $N_{\text{lat}} \ll N_{\text{phys}}$4 in the quasiperiodic case and approximately $N_{\text{lat}} \ll N_{\text{phys}}$5 in the turbulent case, and states that latent spaces sized just above the attractor dimension are required for accuracy and robustness (Racca et al., 2022).

The CAE is also reported to outperform linear proper orthogonal decomposition: at the same number of latent variables, the reconstruction error is lower by two orders of magnitude. In later latent-space studies, CAE likewise outperforms PCA and dense autoencoders in reconstruction quality for chaotic PDE data (Özalp et al., 2024).

These results establish the reduced-order representation as explicitly nonlinear. This suggests that the latent variables are not merely compressed coordinates in a linear subspace, but a numerical approximation to a nonlinear attractor-compatible manifold. That interpretation is consistent with the later use of the same latent space for Lyapunov and covariant-vector analysis.

4. Forecasting performance in quasiperiodic and turbulent regimes

On two-dimensional Kolmogorov flow, the CAE-ESN is reported to time-accurately and statistically predict both quasiperiodic and turbulent regimes, to be robust across Reynolds numbers, and to require less than $N_{\text{lat}} \ll N_{\text{phys}}$6 of the computational time of solving the governing equations (Racca et al., 2022). The paper also summarizes the speedup as $N_{\text{lat}} \ll N_{\text{phys}}$7–$N_{\text{lat}} \ll N_{\text{phys}}$8 orders of magnitude faster than direct numerical simulation.

For the quasiperiodic regime at $N_{\text{lat}} \ll N_{\text{phys}}$9, the model predicts the future evolution for many characteristic periods, with time-averaged normalized RMSE often below $3 \times 3$0. For the turbulent regime at $3 \times 3$1, the model predicts up to $3 \times 3$2 Lyapunov times ahead. Long closed-loop forecasts reproduce ground-truth statistics, including PDFs of dissipation, almost identically; the Kantorovich metric used to compare the PDFs is reported to be extremely low (Racca et al., 2022).

The same paper emphasizes robustness with respect to regime changes and latent-space dimension. Increasing the latent dimension above the minimum needed for embedding the attractor improves consistency and robustness. Later stability-oriented work reproduces the same qualitative limitation familiar from chaotic forecasting: pointwise trajectories diverge after roughly a few Lyapunov times, but dominant statistical and dynamical properties remain accurate (Özalp et al., 2024).

A recurring misconception is that divergence of long closed-loop trajectories implies model failure. The cited work supports a different interpretation. In chaotic systems, finite-horizon trajectory accuracy and long-horizon invariant fidelity are distinct objectives; the CAE-ESN is reported to satisfy both short-term forecast accuracy and long-term statistical consistency, even when exact trajectories decorrelate.

5. Stability inference in latent space

The most substantial development after the original forecasting paper is the use of the CAE-ESN latent dynamics for stability analysis. The later papers argue that the latent-space ESN defines a reduced dynamical system whose tangent-space dynamics can be interrogated directly, allowing the computation of Lyapunov exponents (LEs), Kaplan–Yorke dimension, and covariant Lyapunov vectors (CLVs) (Özalp et al., 2024).

For this purpose, the ESN Jacobian is analytically available. One reported form is

$3 \times 3$3

while a more explicit form is given as

$3 \times 3$4

which is then used in the tangent equation for perturbation growth and QR-based Lyapunov analysis (Özalp et al., 2024). The Kaplan–Yorke dimension is computed as

$3 \times 3$5

On the Kuramoto-Sivashinsky equation with $3 \times 3$6 and domain length $3 \times 3$7, a CAE latent space of dimension $3 \times 3$8 is reported to preserve the leading stability properties. The reference system has $3 \times 3$9 and maximal LE $5 \times 5$0, whereas the inferred latent-space value is $5 \times 5$1. The distribution of angles between CLVs in latent space nearly identically matches the full-system distribution, with Wasserstein distance approximately $5 \times 5$2 (Özalp et al., 2024).

A broader study on various Kuramoto-Sivashinsky regimes reports that the mean absolute error of the first $5 \times 5$3 exponents is $5 \times 5$4 relative to the reference full-Jacobian calculation, that the inferred $5 \times 5$5 matches the reference within less than $5 \times 5$6, and that angle distributions between unstable, neutral, and stable subspaces match closely across attractor dimensions from $5 \times 5$7 to $5 \times 5$8 (Özalp et al., 2024). On Kolmogorov flow, the quasiperiodic regime yields leading exponents approximately zero within $5 \times 5$9, and the turbulent regime reproduces the dominant positive exponent and positive spectrum accurately, with larger discrepancies only for highly negative exponents (Özalp et al., 2024).

These results support a strong claim made in the later papers: the latent space can preserve not only observables and short-term dynamics, but also invariant stability properties and tangent-space geometry. A plausible implication is that nonlinear latent spaces learned by CAEs can serve as computationally tractable surrogates for stability analysis when the governing Jacobian is unavailable.

6. Relation to adjacent reservoir-computing frameworks, strengths, and limitations

Within reservoir computing, the CAE-ESN belongs to a wider class of hybrid architectures that separate representation learning from temporal processing. Deep-ESN, for example, alternates reservoir and encoding layers to capture multiscale dynamics, mitigate collinearity, preserve the echo-state property under suitable conditions, and maintain $7 \times 7$0 time complexity (Ma et al., 2017). The CAE-ESN differs in that its encoder is a convolutional autoencoder acting directly on the physical field, rather than a sequence of projection-encoding stages inside the reservoir hierarchy.

Separate work on control-aware ESNs shows that ESN-based predictors can be incorporated into PID and model predictive control for the suppression of extreme events, achieving up to two orders of magnitude reduction in event occurrence in a chaotic-turbulent flow relative to standard PID and the uncontrolled case (Racca et al., 2023). This suggests a possible future integration of latent-space modelling and control, although such an integration is not established in the cited CAE-ESN papers.

The explicitly reported strengths of the latent-space CAE-ESN approach are its data-driven and equation-free character, efficient training because only output weights are trained in the ESN, strong nonlinear compression, and preservation of leading dynamical invariants (Özalp et al., 2024). Its limitations are also stated clearly. The latent dimension must be at least as large as the attractor’s true dimension; decoder approximation can remove critical dynamical information if the latent space is too small or overly constrained; closed-loop trajectories diverge after a few Lyapunov times in chaotic regimes; and for extremely high-dimensional turbulence or systems without a well-captured low-dimensional attractor, larger latent spaces may be required and rare or extreme transient dynamics may be missed (Özalp et al., 2024).

Taken together, these papers position DA-CAE-ESN as a latent-space reduced-order methodology with two distinct roles. In its original formulation, it is a nonlinear forecasting and acceleration tool for turbulent flow. In its later formulation, it is also a vehicle for dynamical-systems inference, including LE, CLV, and attractor-geometry estimation, performed entirely from data without direct access to the governing equations or their Jacobian (Özalp et al., 2024).