Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation

Abstract: Koopman theory promises linear structure in nonlinear dynamics, but numerical Koopman spectra are easy to compute and hard to trust. A finite EDMD matrix always has eigenvalues; the problem is that many of them may have nothing to do with the infinite-dimensional operator. In this paper we make spectral reliability the objective of dictionary learning. We train neural-network dictionaries not merely to predict the next snapshot, but to minimize Residual Dynamic Mode Decomposition residuals: operator-level a posteriori errors that test whether computed eigenvalues and modes are genuine Koopman spectral objects. To keep the learned observables from collapsing into an unstable coordinate system, the loss also penalizes the condition number of the lifted data matrix. Thus the method couples two requirements that should not be separated: small Koopman residuals and a well-conditioned representation. The result is a learned dictionary that is expressive, numerically stable, and spectrally disciplined. Across conservative and dissipative benchmark systems, the method sharply reduces spectral pollution, improves residual pseudospectral inclusion, and lowers forecast error relative to standard fixed dictionaries. On sea-surface temperature data, it gives cleaner Koopman diagnostics and substantially better one-step forecasts from noisy observations with no governing equations. The message is simple: neural Koopman learning should be judged not by prediction alone, but by whether its spectral claims can be certified. Residuals provide the certificate; conditioning makes it computable.

• The paper introduces a residual-guided neural dictionary learning framework that integrates ResDMD residuals with conditioning regularization for reliable Koopman spectral approximation.
• The methodology is validated on systems like the pendulum, harmonic oscillator, and Duffing oscillator, demonstrating significant improvements in forecast error and numerical conditioning over traditional approaches.
• The approach is also applied to sea-surface temperature dynamics, where it enhances residual loss metrics and enables robust extraction of Koopman modes critical for climate forecasting.

Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation

Introduction and Motivation

Koopman operator theory enables the study of nonlinear dynamical systems via an infinite-dimensional linear operator acting on observables. Recent thrusts in data-driven dynamical modeling, most notably EDMD and its extensions, have elevated spectral analysis for systems where no analytic model exists. However, spectra computed from finite-dimensional approximations (commonly via fixed dictionaries such as polynomials, Fourier, Hermite, or kernels) are notoriously unreliable: observed eigenvalues and eigenfunctions often suffer from spectral pollution, lacking correspondence to the true Koopman spectrum. Persistent questions remain on the reliability and trustworthiness of learned Koopman spectral objects—particularly the crucial links between expressive dictionary function classes, their numerical conditioning, and their spectral fidelity.

This work tackles these challenges by making spectral reliability the core objective of neural dictionary learning. Instead of training dictionaries purely for predictive fidelity, the authors build their objective around Residual Dynamic Mode Decomposition (ResDMD) residuals—operator-level, a posteriori errors which certify whether computed eigenpairs genuinely correspond to Koopman spectral components. To counteract numerical instability endemic to over-expressive or redundant dictionaries, the loss is augmented with explicit regularization on the condition number of the lifted data matrix. Thus, the proposed approach enforces both low-residual spectral certificates and robust conditioning, yielding neural dictionaries whose numerically computed spectra are both expressive and trustworthy.

Theoretical and Algorithmic Framework

The foundation of this approach is a rigorous integration of Koopman operator theory with modern neural dictionary learning and certified spectral diagnostics:

• Koopman Operator Setup: The evolution operator $\mathcal{K}_F$ acts on observables via composition with the system dynamics $F$. For numerics, it is compressed via projection onto a finite dictionary $\mathcal{D}_N$.
• ResDMD Residuals: Unlike standard EDMD, ResDMD computes not just eigenvalues of the projected operator, but also the residual for each eigenpair, converging (as data grows) to the true operator-level residual. Thus, low-residual eigenpairs are credible spectral candidates, while high-residual pairs are identified as pollution.
• Neural Dictionary Parameterization: Each dictionary element is a separately parameterized shallow ReLU network, initialized independently. Neural architectures are chosen for flexibility but require careful regularization to avoid redundancy and ill-conditioning.
• Joint Loss Function:

$$L(\mathbf{X}; \Theta) = \frac{1}{N}\sum_{j=1}^N r_j(\Theta)^2 + \alpha \operatorname{ReLU}\left(\log\frac{\kappa(\Theta)}{\kappa_0}\right)$$

Here $r_j(\Theta)$ denotes the ResDMD residual for the $j$th eigenpair with current parameters $\Theta$, and $\kappa(\Theta)$ is the condition number of the (weighted) lifted data matrix. The loss directly enforces the simultaneous minimization of spectral residuals and control of numerical instability.

• Optimization Scheme: Mini-batch AdamW is employed, with early stopping and held-out validation to prevent overfitting and select ridge/Tikhonov regularization.

Empirical Evaluation on Benchmarks

The methodology is evaluated across three canonical dynamical systems and a high-dimensional, noisy real-world climate dataset. For each, training dynamics and post-training diagnostics are quantitatively assessed using training/test loss, condition number, and spectral forecast error.

Pendulum System

On the nonlinear undamped pendulum, the learned neural dictionary demonstrates a sharp reduction in both spectral residuals and forecast error relative to traditional Fourier–Hermite dictionaries, even while sustaining a much lower (and controlled) condition number. The spectral pseudospectra post-training show significantly reduced pollution and tight inclusion on the unit circle, indicating robust recovery of the expected unitary Koopman structure.

Figure 1: Pendulum training diagnostics, demonstrating convergence of loss, improved conditioning, and substantial reduction in forecast error for the neural dictionary relative to Fourier–Hermite benchmarks.

Undamped Harmonic Oscillator

The approach yields a well-conditioned dictionary (mean $\kappa \approx 258$ versus $>10^6$ for Hermite), nearly order-of-magnitude improvements in forecast error, and strong match between computed and expected spectral structure on the unit circle. The ResDMD pseudospectra reveal that the neural dictionary, post-training, resolves additional genuine modes and eliminates spurious spectral pollution.

Figure 2: Undamped harmonic oscillator diagnostics: strong loss reduction, stable conditioning, and order-of-magnitude better forecast error with the trained neural dictionary.

Duffing Oscillator

For the dissipative Duffing oscillator, Chebyshev benchmark dictionaries suffer from large condition numbers (median $F$0) and significant spectral pollution; post-training, the neural dictionary achieves a mean $F$1 of $F$2 and a forecast error improved by nearly an order of magnitude. The residual pseudospectra tighten around genuine dynamical features and suppress spurious eigenvalues in the disk.

Figure 3: Duffing oscillator diagnostics: superior spectral residual minimization and forecast error lowering via the residual-guided neural dictionary.

Real-World Application: Sea-Surface Temperature (SST) Dynamics

To assess practical utility in high-dimensional, noisy, partially observed systems, the authors apply their method to monthly tropical Pacific sea-surface temperature data, preprocessed with PCA for dimensionality reduction. For both 5- and 10-dimensional state truncations, the neural dictionaries consistently outperform quadratic polynomial benchmarks across residual loss, conditioning, and one-step forecast error—progressing from mean errors of $F$3 (polynomial) to $F$4 (neural, $F$5) and $F$6 to $F$7 ($F$8). The approach also produces Koopman modes with substantial amplitude in the Niño 3.4 region (an area critical for ENSO prediction) and with substantially smaller associated residuals.

Figure 4: Loss curves for SST forecasting with $F$9 and $\mathcal{D}_N$0, showing consistent reduction with the trained neural dictionaries against polynomial reference.

Figure 5: Condition number evolution during SST neural dictionary training for $\mathcal{D}_N$1 and $\mathcal{D}_N$2.

Figure 6: One-step forecast error curves for SST data, with significantly improved performance from trained dictionaries.

Figure 7: Koopman mode with maximal amplitude in Niño 3.4 region, post-training, exhibiting strong residual reduction compared to pretraining and benchmark polynomial dictionaries.

Implications, Limitations, and Future Directions

The principal implication of this work is a rigorous shift in neural Koopman operator learning: spectral trustworthiness, assessed through operator-level residuals and stabilized through conditioning regularization, can and should be built directly into the learning process. This approach enables the computation of Koopman eigenvalues and modes that are robust, numerically stable, and can be meaningfully associated with the infinite-dimensional spectrum, even when starting from highly flexible neural dictionaries with little prior structural knowledge.

The method’s negligible sensitivity to hyperparameters and moderate computational cost substantiate its practicality for dynamical modeling in scientific and engineering contexts. Remaining limitations include reliance on sufficiently rich snapshot data and sensitivity to dictionary size relative to data—further work could explore adaptive dictionary sizing and extensions to continuous spectra and nonstationary or stochastic dynamics.

Theoretical advances could include extensions to continuous-time Koopman generator approximation, assessment in infinite dictionary limits, and integration with model discovery frameworks that simultaneously learn both operators and governing equations.

Conclusion

This study establishes residual-guided neural dictionary learning with conditioning penalties as a robust and principled paradigm for data-driven Koopman approximation, shifting focus from mere prediction to spectral fidelity. The approach yields spectrally disciplined neural dictionaries that minimize spectral pollution, ensure numerical stability, and produce Koopman objects with certified operator-level trust. This aligns data-driven dynamical modeling with rigorous operator-theoretic standards, paving the way for more reliable discovery and analysis of complex systems.

Citation: "Residual-Guided Dictionary Learning for Spectrally Accurate Koopman Approximation" (2606.29083)