---
title: Deep Koopman-Layered Models
url: https://www.emergentmind.com/topics/deep-koopman-layered-models
type: topic
---

# Deep Koopman-Layered Models

Deep Koopman-layered models are a class of data-driven surrogate models that leverage neural-network-parameterized nonlinear lifts to map complex dynamical systems into a latent space where (often high-dimensional) dynamics are enforced to be finite-dimensional linear, approximately linear, or, in certain variants, convex or otherwise structurally constrained. The central architectural motif is a composition of a nonlinear encoder, a (typically) linear latent propagator corresponding to the Koopman operator, and a nonlinear or linear decoder reconstructing states or outputs from the latent. These models are motivated by Koopman operator theory, which states that any (suitably regular) nonlinear dynamical system admits an infinite-dimensional linear propagation of observables; deep Koopman-layered frameworks aim to learn finite-dimensional, expressive parametrizations that capture long-term and global dynamics while enabling fast prediction and control.

## 1. Architectural Principles and Network Structures

Koopman-layered models universally consist of three “layers”:

1. **Encoder (Lifting map):** A neural network φ that maps the physical state x (or possibly input-output history) into a latent coordinate z ∈ ℝ^r, intended to represent a set of observables that, ideally, span a Koopman-invariant subspace. Common choices include fully connected MLPs, ResNets, CNNs for structured data [1708.06850][2409.06522], and, in advanced variants, Kolmogorov–Arnold Networks (KANs) [2406.02875].

2. **Latent-space propagator (Koopman layer):** The lifted variable is propagated linearly: z_{k+1} = A z_k + B u_k, where (A, B) are trainable (or structured) matrices. However, model variants include Wiener-type (encoder–linear–decoder), input-affine, bilinear, convex-ICNN, and extended/parametric forms (e.g., with invertible control transforms, time-varying Toeplitz layers, or innovation noise) [2201.12669][2010.06845][2410.02199][2507.09646].

3. **Decoder:** Maps z back to the physical or observation space. It may be a linear map, a fully connected network, or a structured operator projecting only the original state components [2503.23396][2511.04437].

These models can be summarized diagrammatically (with φ as encoder, ψ as decoder, A as Koopman matrix):

  xₖ —[φ]→ zₖ —[A,B dynamics]→ zₖ₊₁ —[ψ]→ x̂ₖ₊₁

The key to expressivity and generalization is the selection of the encoder architecture and the precise mathematical constraints imposed on A and/or B during optimization.

## 2. Training Objectives and Loss Formulations

The prevailing goal in deep Koopman-layered models is to enforce accurate linear latent propagation and (where needed) reconstructability. Typical loss functions include:

- **Latent linearity loss:** Measures the agreement between the predicted next latent and the encoder’s output on the actual next state:

  $$
  L_{\mathrm{lin}} = \sum_{k} \|\phi(x_{k+1}) - A \phi(x_k) - B u_k\|^2
  $$

- **Reconstruction loss:** Enforces that the encoder–decoder pair approximates an autoencoder:

  $$
  L_{\mathrm{rec}} = \sum_{k} \|x_k - \psi(\phi(x_k))\|^2
  $$

- **Multi-step prediction loss:** Penalizes deviation between simulated (unrolled) model predictions and ground truth over a specified horizon:

  $$
  L_\mathrm{multi} = \sum_{k,\,h=1}^T \|x_{k+h} - \psi(A^h \phi(x_k) + \cdots)\|^2
  $$

- **Application-specific terms:** Physics-informed acceleration losses, Lyapunov regularization for latent-stability, innovation-form noise modeling, or information bottleneck objectives for controlling simplicity/expressiveness tradeoff [2503.23396][2405.00627][2510.13025].

These losses are typically combined, possibly with weight regularization (||W||_1 or ||W||_2^2), in a weighted sum tailored to the task and dataset.

## 3. Structural and Theoretical Extensions

Deep Koopman frameworks are extended in several key directions:

- **Wiener- and Hammerstein-Block Structures:** Linear latent propagation sandwiched between nonlinear encoder/decoder networks; enables extremely low-dimensional (r=1–3) surrogates with high accuracy, especially beneficial for model reduction and control [2201.12669][2110.02583].
  
- **Convex and Extended Koopman Models:** Convex dynamics in latent space (implemented via input convex neural networks, ICNNs), and invertible reparameterizations of the control input, lead to improved long-horizon predictivity and robustness to control uncertainty [2010.06845].

- **Toeplitz-Matrix and Krylov Subspace Construction:** For systems with periodic or nonautonomous structure, deep Koopman models with Toeplitz-structured latent maps and matrix exponentials (computed via Arnoldi iterations) ensure universality and scalability (universality theorems and Rademacher-complexity bounds are provided) [2410.02199].

- **Probabilistic and Variational Koopman Models:** Incorporate uncertainty via variational inference over the latent, yielding ensembles of confidence-aware, linearized surrogates. The Deep Variational Koopman and Deep Probabilistic Koopman variants employ stochastic autoencoders and parameterize time-varying predictive distributions [1902.09742][2106.06033].

- **Information-regularized Koopman Networks:** Explicitly balance simplicity (mutual information minimization with the input) and expressiveness (maximizing von Neumann entropy to prevent mode collapse) with Lagrangian objectives, empirically improving stability and representational coverage [2510.13025].

## 4. Modeling with Inputs, Control, and Innovation Noise

Modern deep Koopman-layered surrogates handle controlled and noisy systems by incorporating:

- **Input-affine and Bilinear terms:** Latent update laws such as
  z_{k+1} = A z_k + ∑_i B^{(i)} z_k u_{k,i}, or more generally B(z_k, u_k) u_k,
generalizing the finite-dimensional (control-affine) Koopman theory to practical settings [2110.02583][2201.12669][2507.09646].

- **Innovation form noise models:** Innovation noise is incorporated in the latent update, i.e.,
  z_{k+1} = A z_k + B(z_k, u_k) u_k + K(z_k, u_k, e_k) e_k,
and the initial latent is successfully reconstructed from initial I/O history via a deep encoder [2507.09646].

- **Adaptive and online updates:** Real-time adaptation of A, B in the lifted space, e.g., via sliding window least squares, robustifies models to parameter drift or unmodeled disturbances without the need to retrain the encoder [2503.23396].

- **Integration with control algorithms:** Deep Koopman surrogates are now routinely deployed within MPC architectures (both economic and tracking/control), often outperforming classical subspace methods (such as N4SID) by a significant margin in both prediction error and closed-loop economic objectives [2511.04437][2503.03002].

## 5. Empirical Performance and Benchmarks

Deep Koopman-layered models have been quantified on diverse tasks:

| Model Type                   | Benchmark Domain                | Horizon/Metric              | Key Results                              |
|------------------------------|----------------------------------|-----------------------------|-------------------------------------------|
| Standard Deep Koopman AE     | Glycolytic oscillator, Power grid| up to 400 steps             | ≤1% one-step error, accurate long-horizon |
| Wiener-type Koopman-AE       | Chemical reactor, distillation   | NMSE, trajectory tracking   | r=1–2, lowest NMSE, strongest reduction   |
| Convex/Extended Koopman      | Double-well, quadruped           | Trajectory/rollout error    | ~4–5× lower error vs. linear benchmarks   |
| Physics-informed DK          | Autonomous vehicle, CarSim       | RMSE (wheel/vel/yaw)        | Up to 95% lower error after adaptation    |
| Probabilistic Koopman        | Electricity, Chem, NeuroScience  | Negative log-likelihood     | Outperforms all 177 domain-specific models|
| Toeplitz-layered Koopman     | Van der Pol, time-varying vortex | Eigenvalue estimation, MSE  | Unit-circle spectra where appropriate     |
| Information-regularized      | Lorenz-63, Kármán vortex         | NRMSE/SSIM, mode diversity  | Best NRMSE and eigenvalue spread          |

Almost everywhere, deep Koopman-layered models outperform shallow, fixed-dictionary, and purely linear control techniques, especially for nonlinear, multi-time-scale, and high-dimensional phenomena [1708.06850][2201.12669][2507.09646][2511.04437][2510.13025].

## 6. Implementation Practices and Challenges

Best practices in model development include:

- **Encoder/decoder design:** Prefer moderate-width MLPs (two or three layers, 20–100 units) for generic problems; adopt CNNs or ResNets for spatial or image data; hybridize with Kolmogorov–Arnold Networks (KANs) for parameter efficiency and fast convergence [2406.02875][2409.06522].

- **Latent dimension selection:** Empirical grid search is usually required to select the smallest nr that preserves accuracy; oversizing risks overfitting, undersizing can miss critical modes [2110.02583].

- **Multi-step losses and multiple-shooting:** Batch-parallel, truncated unrolls (“multiple shooting”) balance memory efficiency and capture long-time structure, especially when combined with early stopping [2110.02583][2507.09646].

- **Spectral, Lyapunov, and stability penalties:** Enforcing spectral radius, Lyapunov, and orthogonality constraints improves robustness and generalization, especially in long rollout or control settings [2508.02616].

- **Generalization and universality:** Theoretical guarantees on expressivity are available for specific structured networks (e.g., Toeplitz-based models with exponentials of banded matrices), but tuning remains empirical and can scale poorly in deep/broad settings [2410.02199].

- **Pitfalls:** Models can fail when B(z, u) is extrapolated far outside training data, or if the LR/spectral radius is not controlled, leading to latent drift or instability. Regularization, excitation design, and careful validation are essential.

## 7. Research Directions and Open Questions

Active research frontiers include:

- **Mode diversity and representation compression:** Information-theoretic regularization (mutual information, von Neumann entropy) improves latent code coverage and avoids mode collapse, delivering more interpretable and robust Koopman subspaces [2510.13025].
  
- **Adaptive, online, and nonautonomous learning:** Designs that allow for windowed or sequential updating of the Koopman operator address parameter drift and system nonstationarity.

- **Probabilistic and uncertainty-aware extensions:** Explicitly integrating variational Bayesian inference and distributional prediction into Koopman-layered models, as in DVK/DPK, allows for quantification of epistemic/model uncertainty [1902.09742][2106.06033].

- **Integration of physical constraints:** Time-reversibility, stochasticity, conservation laws (e.g., detailed balance) are now incorporated through architectural and optimization constraints, providing domain-specific guarantees and helping in sample-inefficient regimes [1912.07392].

- **Architectures for high-dimensional and structured domains:** CNN-, graph-, and Transformer-based encoders are now embedded in Koopman-layered pipelines for environmental, financial, or visual-control contexts, driving performance on synthetic and real-world benchmarks [2508.02616][2409.06522].

- **Scalability and numerical efficiency:** Toeplitz-matrix factorization, Krylov subspace methods, batched or parallel implementation, and compressed spectral layers are opening up applications in high-dimensional, large-scale dynamics while preserving mathematical properties [2410.02199].

---
**References**

Key citations above: [1708.06850][2201.12669][2010.06845][2110.02583][2409.06522][2410.02199][2405.00627][2503.23396][2511.04437][2507.09646][2510.13025][2508.02616][1902.09742][2106.06033][1912.07392].

Source: https://www.emergentmind.com/topics/deep-koopman-layered-models