Koopman-Based Learning Methods

• Koopman-based learning methods are data-driven techniques that recast nonlinear dynamics as linear evolutions in a high-dimensional function space using observable mappings.
• They leverage advanced tools from convex optimization, meta-learning, and neural architectures to automatically identify optimal embeddings and manage model complexity.
• Applications in control, reinforcement learning, and stability certification highlight their practical utility for real-time prediction, simulation, and nonlinear MPC.

Koopman-based learning methods are a class of data-driven techniques for modeling, analyzing, and controlling nonlinear dynamical systems by leveraging the properties of the Koopman operator—a linear (but infinite-dimensional) operator that acts on observables of the state space. These methods seek to reformulate nonlinear dynamics as linear evolutions in a suitably chosen function space, thus enabling the use of powerful linear systems theory, optimization, and control methods on problems that are fundamentally nonlinear. Recent advances cover convex optimization, deep learning architectures, meta-learning for spectral analysis, reinforcement learning integration, and certified stability analysis, with particular attention to computational scaling and applications in control-affine systems.

1. Koopman Operator Theory and Linear Embedding

Koopman operator theory concerns the evolution of observables of a dynamical system. For a discrete-time dynamical system $x_{k+1} = T(x_k)$ or a continuous-time system $\dot{x} = f(x)$, the Koopman operator $\mathcal{K}$ acts on scalar observables $g$ as $\mathcal{K}g(x) = g(T(x))$ or, in flow notation, $\mathcal{K}_t g = g \circ \varphi(t, \cdot)$. This operator is linear but infinite-dimensional. The significance is that if a finite-dimensional invariant subspace can be found—spanned by a collection of observables (Koopman eigenfunctions)—the nonlinear system reduces to a finite-dimensional linear (or bilinear) system in the lifted coordinates.

A typical goal is to find a mapping (embedding or "lifting") $\psi(x)$ such that the evolution in the lifted space is linear, i.e.,

$\psi(x_{k+1}) = K\psi(x_k)$

for some matrix $K$. In control settings (control-affine dynamics), this often takes the form of a bilinear system in the lifted space:

$z_{k+1} = Fz_k + \sum_{i} G_i z_k u_k^{(i)}$

where $z = \psi(x)$ and $\{F, G_i\}$ are learned matrices.

Crucially, finding both the embedding $\psi$ and the operator $K$ is nontrivial, since the optimal observables are unknown and may be infinite-dimensional. Approaches thus approximate this lifting using dictionaries of candidate functions, neural networks, or optimization.

2. Convex Optimization and SDP-based Koopman Operator Learning

Traditional methods such as Extended Dynamic Mode Decomposition (EDMD) require an a priori choice of a function dictionary, which can result in high model order and suboptimal representations. The convex optimization approach (Sznaier, 2021) recasts the task of learning Koopman operators as a rank-constrained semidefinite program (SDP):

• Objective: Construct lifted data matrices (e.g., Hankel matrices $H_\psi^{r+1}$ from time-series) and impose the constraint $\text{rank}(H_\psi^{r+1}) \le r$, ensuring low-order (parsimonious) models.
• Constraints:
  • Positive semidefinite Gram matrices to encode inner products in observable space.
  • Affine constraints on distances to preserve local geometry, e.g.,

  $$(1/M_u^2(x_s)) \|x_s - x_t\|_2^2 \leq K_{s,s} - 2K_{s,t} + K_{t,t} \leq M_\ell^2(x_s) \|x_s - x_t\|_2^2$$

  where $K$ is the Gram matrix of lifted observables. - Anchor point normalization constraints.

Because direct rank minimization is NP-hard, a weighted nuclear norm is used as a convex surrogate, giving rise to a sequence of convex SDPs:

$$\min \|W G\|_* + \lambda_1 \sum_j \|V^{(j)} L_{x_j}\|_* + \text{penalty terms}$$

where $W$, $V^{(j)}$ are weight matrices, $L_{x_j}$ are Loewner matrices, and $G$ is the Gram matrix.

Chordal sparsity is exploited to decompose large SDPs into smaller subproblems, leveraging graph-theoretic results (Grone’s theorem, Dancis’ theorem) so that the computational complexity scales linearly with the number of data points. This enables large-scale applications that would be intractable for standard SDP approaches.

Implications: The approach avoids pre-specifying function dictionaries and dimension; it automatically identifies the embedding, the Koopman operator, and the memory order $r$ from data, yielding models with lower order (e.g., for the Lorenz attractor, 7th vs 14th order compared to existing methods) and certified approximation quality.

3. Batch-Online, Meta-Learning, and Data-Driven Library Selection

Several Koopman-based learning methods aim at online adaptability, data efficiency, and automation of observable selection:

• Finite-time Batch-Online Identifier (Mazouchi et al., 2021):

  • Uses a lower-layer incremental update law which jointly minimizes instantaneous and mini-batch identification errors for the Koopman operator parameters.
  • Guarantees finite-time convergence (settling time depends on data richness) provided a rank condition on the stored data is satisfied:

  $$\sum_j \bar h_\theta(t_j) \bar h_\theta^\top(t_j) \ge d_\theta I$$ - Experience-replay via a "history stack" improves parameter convergence and ensures coverage of the state space.

• Meta Learner via Bayesian Optimization (Mazouchi et al., 2021):

  • Automatically selects the best set (library) of observable functions to minimize a meta-cost (identification error plus sparsity penalty).
  • Uses discrete Bayesian optimization with acquisition function:

  $$\alpha(\Xi|\mathcal{D}) = \mathbb{E}[\max\{0, J^{R*} - J^R(\Xi)\}]$$

  where $J^R$ quantifies fit over a held-out dataset and $|\Xi|$ is the number of observables.

• Meta-Learning for Short Time-series (Iwata et al., 2021):

  • Uses bidirectional LSTM to generate a representation $r$ for each short trajectory. Neural network embedding functions $\phi(y_t, r)$ are adapted in an episodic training framework that minimizes expected test prediction error.
  • Enables reliable Koopman spectral analysis and prediction with only short time-series, outperforming finetuning and popular meta-learning algorithms (e.g., MAML) on synthetic and benchmark data.

Context: These approaches address scalability, adaptability, and automation challenges. They enable Koopman operator learning in regimes with scarce or heterogeneous data, permit principled selection of observables, and exploit episodic, online, or meta-learning frameworks for improved generalization and sample efficiency.

4. Neural and Deep Learning Architectures for Koopman Embedding

Deep learning approaches introduce parametric, data-adaptive embeddings to address the complexities of unknown or high-dimensional nonlinear dynamics:

• Neural Koopman embeddings: Use feedforward neural networks to parameterize the observable map $g(x) = [x; \tau_g(x)]$ (Uchida et al., 2022), where $\tau_g$ is a learned nonlinear feature vector. Training optimizes both the networks and Koopman system matrices via loss functions that balance linearity in lifted space and state reconstruction fidelity.
• Autoencoder-based approaches (Dey et al., 2022): Employ encoder/decoder pairs to map data to a latent linear space where Koopman evolution is applied, supporting reconstruction and multi-step prediction tasks. DLKoopman software (Dey et al., 2022) provides modular code, hyperparameter search, and new performance metrics such as Average Normalized Absolute Error (ANAE).
• Joint Learning of Dictionary and Bilinear Model (Folkestad et al., 2021):
  • Neural networks jointly parameterize the function dictionary and the lifted bilinear model for control-affine systems, with losses on forward prediction, reconstruction, and consistency with bilinear Koopman dynamics.
  • Enables real-time nonlinear MPC for agile trajectory tracking in the presence of nonlinear effects (e.g., aerodynamic ground effect), outperforming characteristic-based or nominal controllers in physical quadrotor experiments.
• Control-aware Model Refinement (Uchida et al., 2022):
  • After standard supervised training, the system matrices are fine-tuned on closed-loop (controller-driven) data to reduce residual modeling error that can accumulate in feedback, thereby improving stability and closed-loop performance.
  • Deterministic, continuous input trajectories (e.g., sinusoids) during training further enhance model generalization under feedback control.

Context and Significance: These neural architectures are essential for modeling high-dimensional and complex systems where explicit physics-based dictionaries are infeasible or inadequate. They provide flexibility, scalability, and, with suitable loss formulations and sampling strategies, improved control and stability in real-world settings.

5. Advanced Optimization and Hybrid Methods

Recent developments target computational efficiency, model order selection, and integration of programmatic and learning-based techniques:

• Hybrid SDP–Autoencoder Framework (Estornell et al., 25 Apr 2025):
  • Stage 1: Uses semidefinite programming (SDP) to extract observables and the Koopman operator from data, minimizing a matrix rank within convex constraints (e.g., nuclear norm penalty). This sets both the dimension and spectral structure of the latent space.
  • Stage 2: An autoencoder is initialized with architecture informed by the SDP, then trained on data to reconstruct states and refine the embedding, using auxiliary tasks (eigenvalue prediction, multi-step linearity) to align the learned latent space with the spectral properties of the Koopman operator.
• Bi-level Optimization for Koopman Control (Huang et al., 2023):
  • Separates learning of the encoder/decoder (mapping to and from Koopman space) from learning the linear system matrices, imposing long-horizon dynamical constraints via an integral formulation. The inner problem fits dynamics under trajectory consistency, while the outer problem updates mappings for state reconstruction.
  • This structure yields improved accuracy, long-term prediction stability, and reduced training complexity compared to multi-level penalty-based methods.
• Logarithm-free Koopman Generator Estimation (Meng et al., 23 Mar 2024):
  • Introduces a Yosida-inspired, logarithm-free estimator for the Koopman infinitesimal generator. For large $\lambda$,

  $$L = \lambda^2 \int_0^\infty e^{-\lambda t}K_t \, dt - \lambda I$$ - A finite-horizon approximation enables regression to estimate the generator matrix directly from trajectory data without needing $\log K_t$ (which is ill-conditioned or ambiguous for general spectra).

Implications: These innovations reduce the need for a priori specification of hyperparameters (like observable dimension), accelerate convergence, and certify model order and spectral configuration directly from data. By combining optimization with learning, the methods scale to higher dimensions and yield interpretable, robust models.

6. Applications in Control, Reinforcement Learning, and Stability Certification

Koopman-based models are now integrated into a spectrum of decision-making and control workflows:

• Nonlinear Model Predictive Control (NMPC):

  • Utilizes a lifted bilinear Koopman model for efficient prediction, trajectory optimization, and computation in control-affine robotic systems (Folkestad et al., 2021, Folkestad et al., 2021).
  • The bilinear framework allows for fast Jacobian computation and warm-starting in SQP solvers, enabling real-time control.
• Reinforcement Learning:
  • Koopman operator is used for data augmentation (to infer symmetries and expand the training data) in offline RL (Weissenbacher et al., 2021), resulting in improved generalization and performance gains in standard RL benchmarks (D4RL, MetaWorld, RoboSuite).
  • Koopman-assisted RL methods (Rozwood et al., 4 Mar 2024) recast the BeLLMan or HJB equations into linear (or nearly linear) iterations in lifted space, facilitating efficient value iteration and actor-critic algorithms; Koopman tensor parameterization allows efficient representation of state–action dynamics.
• Stability Certification:
  • Neural Koopman Lyapunov control (Zinage et al., 2022) and Lyapunov certificate learning (Zhou et al., 3 Dec 2024) frameworks integrate generator learning with automated Lyapunov candidate construction, leveraging physics-informed neural networks and automated verification (e.g., SMT solvers) to provide formal guarantees on region of attraction estimates.
  • Logarithm-free generator estimation enables direct compatibility with system-theoretic tools that require access to the Lie derivative of candidate Lyapunov or barrier functions.
• Simulation and Prediction in Physical Systems:
  • Koopman-based deep learning models generalize well in simulating complex nonlinear systems such as stiff strings (Diaz et al., 29 Aug 2024), demonstrating superior long-sequence accuracy and robustness to initial conditions compared to state space models and neural operator approaches.

Significance: The wide applicability of these methods, together with their data efficiency and scalability, renders Koopman-based learning effective for both modeling and real-time control of high-dimensional, uncertain, or nonlinear physical systems.

7. Challenges, Open Problems, and Outlook

Despite significant progress, several challenges and research directions remain:

• Construction and Selection of Lifting Dictionaries: Automated, systematic procedures for choosing the ideal observable set are not fully established. The use of meta-learning, Bayesian optimization, and neural parameterizations is still an active area of development.
• Constraint Handling and Robustness: Effective methods for mapping state and input constraints from the original to lifted space are required, particularly in safety-critical domains.
• Accuracy Guarantees and Model Certification: Quantitative, basis-independent measures of approximation accuracy remain lacking, especially when using neural approaches.
• Stochastic and Hybrid System Extension: Current theory and algorithms for systems with uncertainty, jumps, or hybrid dynamics are less mature, with fundamental questions regarding the behavior and interpretation of uncertainty in the lifted space.
• Scalability and Data Requirements: Efficient methods for high-dimensional systems—with guarantees on both accuracy and computational complexity—are a continuing concern. Strategies for active learning and real-time model refinement are being investigated.
• Interpretability and Manipulability: Interpreting learned Koopman eigenfunctions and using them for explicit control synthesis or manipulation of behavior is an open problem.
• Extrapolation and Generalization: Robust extrapolation beyond the training horizon, especially in highly nonlinear or chaotic regimes, remains a significant practical challenge.

The sustained integration of convex optimization, deep learning, system-theoretic analysis, and formal verification will continue to expand the scope and reliability of Koopman-based learning methods in scientific and engineering applications.