Latent ODE Surrogates for Dynamical Systems

• Latent ODE surrogates are neural network frameworks that map high-dimensional states to tractable low-dimensional latent flows governed by ODEs.
• They integrate an encoder, a latent ODE solver, and a decoder, with training objectives combining reconstruction, dynamics consistency, and regularization losses.
• Extensive applications demonstrate that these surrogates enhance computational speed and accuracy in simulating PDEs, stiff ODEs, and complex dynamical systems.

A latent ODE surrogate is a neural network-based framework for approximating continuous-time dynamical systems by learning a low-dimensional latent representation whose evolution is governed by an ordinary differential equation (ODE) in latent space. This methodology provides a unifying, data-driven approach for model reduction, surrogate prediction, generative modeling, and control of complex systems governed by ODEs or PDEs. Surrogates of this type leverage autoencoders or structured encoders/decoders to map high-dimensional system states to latent codes, whose trajectories are then evolved using learned vector fields—typically parameterized by multilayer perceptrons (MLPs)—and mapped back to physical space for downstream tasks.

1. Formulation of Latent ODE Surrogates

The archetype of a latent ODE surrogate comprises three principal components: (1) an encoder $E_\theta: X \to Z$ that maps system states $x$ to latent states $z$; (2) a latent (neural) ODE $dz/dt = F_\theta(z, t)$ or, in special cases, analytic/structured surrogates; and (3) a decoder $D_\theta: Z \to X$ that returns predictions in the original state space. The surrogate thus acts as a reduced-order flow map: $$x(t_2) \approx D_\theta({\rm ODESolve}(F_\theta, E_\theta(x(t_1)), [t_1, t_2]))$$ (Chung et al., 24 Sep 2025).

Variants of this paradigm adapt the architecture and the governing latent dynamical system to match application requirements. Examples include:

• Smooth hypernetwork-driven latent ODEs with consistency regularization for parameterizing compact representation decoders (e.g., “Nonlinear Fourier Ansatz”) in advection-dominated PDEs (Wan et al., 2023).
• Elementwise latent ODE surrogates assembled via directional couplings and feature libraries for spatially scalable PDE solvers (Chung et al., 5 Jan 2026).
• Constant-velocity latent ODEs with nonlinear time warping for stiff ODEs, entirely bypassing numerical integration during inference (Nockolds et al., 14 Jan 2025).
• Symbolic surrogate ODEs learned via sparse regression (SINDy-FM) or compact neural ODEs for diffusion/Schrödinger bridge generative modeling in latent spaces (Khilchuk et al., 14 Dec 2025).
• Direct parametric curve surrogates replacing neural ODE integration for irregularly sampled time series forecasting (FLD) (Klötergens et al., 2024).

The goal is always to identify a low-dimensional representation in which the system's time evolution is tractable, smooth, and efficiently approximated.

2. Core Architectural Components and Loss Functions

A generic latent ODE surrogate implementation consists of the following elements:

Component	Mathematical Description	Architectural Example
Encoder	$z(0) = E_\theta(x(0))$	ResNet, CNN, or MLP
Latent ODE	$dz/dt = F_\theta(z, t)$	MLP, block-structured Θ, analytic forms
Decoder	$\hat x(t) = D_\theta(z(t))$	MLP, Nonlinear Fourier ansatz, VAE

The learning objective typically combines:

• Reconstruction loss: $$L_{\text{rec}} = \mathbb{E}[\|D_\theta(E_\theta(x)) - x\|^2]$$
• Dynamics consistency loss: $$L_{\text{dyn}} = \mathbb{E}[\|\Psi_\theta^{(t_2, t_1)}(E_\theta(x(t_1))) - E_\theta(x(t_2))\|^2]$$
• Regularization terms: Consistency-inverse (encode–decode–re-encode), stability via penalizing latent growth (Wan et al., 2023, Chung et al., 5 Jan 2026), quantile regression for uncertainty (Chapfuwa et al., 2022).

Advanced surrogates may impose additional structure, such as:

• Feature libraries in the ODE (polynomial, upwind, nonlinear interaction terms) (Chung et al., 5 Jan 2026, Khilchuk et al., 14 Dec 2025).
• Consistency-inducing regularization enforcing near-bijection in latent–ambient mapping (Wan et al., 2023).
• Stochasticity (VAE-like) or structured latent factors capturing inputs or process noise (Chapfuwa et al., 2022, Maurel et al., 3 Feb 2026).

3. Representative Methodologies and Algorithms

Several paradigms instantiate the latent ODE surrogate idea across domains:

1. Latent ODE Autoencoder for PDEs
   • Encode each solution snapshot; evolve latent code under a smooth ODE; decode via neural basis (e.g., Nonlinear Fourier, small MLP) (Wan et al., 2023).
   • Enforce consistency via $$L_{\text{consistency}}(\theta) = \|\theta - \Lambda(\Psi(\theta))\|^2$$.
   • Two-phase training: (1) learn (encoder, decoder); (2) learn latent ODE, staged by one-step pretraining and multi-step fine-tuning.
2. Modular Latent Surrogates (LSEM)
   • Train local latent ODE surrogates for subdomains, couple elements with learned blocks capturing upwind-like directional interactions, and blend predictions for scalable global surrogacy (Chung et al., 5 Jan 2026).
3. Time-Scale-Aware Neural ODE Surrogates
   • Quantify elimination of fast modes and optimal retention of slow modes via eigenvalue analysis of the latent ODE Jacobian (Nair et al., 2024).
   • Latent timescales governed mainly by rollout length during training, not latent dimension or architecture width.
4. Constant-Velocity + Time-Warp Latent ODEs (LiLan)
   • Model stiff ODEs by learning analytic latent trajectories parameterized by a constant latent velocity and adaptive time-warp; integration is reduced to a single closed-form computation (Nockolds et al., 14 Jan 2025).
   • Universality: latent dimension independent of required approximation ε.
5. Fast Symbolic Surrogates for Diffusion Bridge Models
   • SINDy-FM uses sparse regression of exact time derivatives in a symbolic basis, yielding sparse interpretable ODEs for latent bridging (Khilchuk et al., 14 Dec 2025).
   • DSBM-NeuralODE uses a compact neural ODE drift with supervised flow-matching losses, trained on latent diffusion trajectories.
6. Structured Latent ODEs for Input-Actionable Dynamics
   • Partition static input-induced and noise/stochastic factors in the latent code, enabling controlled generation of what-if trajectories and actionable uncertainty quantification (Chapfuwa et al., 2022).
7. Functional Latent Dynamics as ODE Surrogates
   • Replace neural-ODE integration with parametric closed-form curves (linear, quadratic, sinusoidal) in latent space, dramatically reducing inference cost for time series with irregular sampling (Klötergens et al., 2024).

4. Theoretical Properties and Approximation Guarantees

Latent ODE surrogates benefit from both universal approximation guarantees and explicit theoretical error controls under mild conditions:

• If the latent autoencoder and ODE vector field are sufficiently expressive, the surrogate can uniformly approximate the true flow map over compact domains (Chung et al., 24 Sep 2025, Nockolds et al., 14 Jan 2025).
• Approximation error at any time is controlled by the sum of the autoencoding and vector-field errors; there is no accumulation of local truncation errors as in recursive RNNs or classical time-marching (Chung et al., 24 Sep 2025).
• For stiff systems, constant-velocity/time-warp surrogates provide approximation with no increase in latent dimension as accuracy is improved (Nockolds et al., 14 Jan 2025).
• In modular assembly, surrogacy generalizes to longer or larger domains via tiling without retraining, due to locality and learned inter-element interactions (Chung et al., 5 Jan 2026).

5. Quantitative Performance and Application Domains

Latent ODE surrogates have demonstrated efficacy in a variety of high-impact domains.

Domain / Problem	Representative Surrogate	Quantitative Result (Test Error/Speedup)	Reference
Advection-dominated PDEs	Hypernetwork latent ODE	relRMSE: VB: 1.5%, KS: 8%, KdV: 5%; inference speedup ×2–×20	(Wan et al., 2023)
Large-domain PDEs	LSEM latent ODE assembly	Burgers scaled error: 0.41%, speedup ×32–×5.5×10³	(Chung et al., 5 Jan 2026)
Stiff ODEs	LiLan (constant-velocity, tw)	<0.3% error; speedup ×10³ over stiff solver; superior to DeepONet, NODE	(Nockolds et al., 14 Jan 2025)
Diffusion bridges (generative)	Symbolic SINDy-FM/Neural ODE	SINDy-FM: μs/sample, O(10²) params; matches neural ODE in accuracy	(Khilchuk et al., 14 Dec 2025)
Clinical PK/PD prediction	VAE latent ODE + ODE-RNN	RMSPE: 7.99% (internal), 10.82% (external), beats NLME it2B	(Maurel et al., 3 Feb 2026)
Partially observed/chaotic systems	Bilinear latent ODE, joint y opt.	RMSE <1e-5 for linear/chaotic, Lyap. exponent ≈ true (Lorenz)	(Ouala et al., 2019)
Dynamic 3D Scene Extrapolation	Transformer + latent ODE	PSNR: +3–9 dB over baselines; inference in 10–20 ms	(Wang et al., 5 Jun 2025)
Biomedical actionable modeling	Structured latent ODE w/ quantiles	Zero-shot input recovery, accurate quantiles, best L1 error	(Chapfuwa et al., 2022)
Irregular time series	FLD curves (linear, quad., sinus)	Test MSE: matches/bests ODEs, ×10² memory; ×10–×100× inference speed	(Klötergens et al., 2024)

These surrogates provide both predictive accuracy and computational acceleration, frequently outperforming baseline ODE solvers and deep sequence models.

6. Practical Guidelines, Strengths, and Limitations

Effective application of latent ODE surrogates is facilitated by several practical considerations:

• Latent dimension selection: Should be large enough to retain slow, coherent system modes; optimal N_w can often be guided by the number of energy-dominant POD modes (Nair et al., 2024).
• Rollout horizon in loss: For acceleration and smoother latent trajectories, rollout length in the training loss is the principal control lever (Nair et al., 2024).
• Non-intrusive surrogacy: In modular assembly or operator-learning scenarios, no access to PDE residuals or explicit operator forms is required; only state snapshots suffice (Chung et al., 5 Jan 2026, Chung et al., 24 Sep 2025).
• Uncertainty quantification: Integration of variational or quantile-regression likelihoods yields meaningful predictive intervals and enables interpretable downstream use (Chapfuwa et al., 2022, Maurel et al., 3 Feb 2026).
• Inference throughput: Approaches such as analytic latent times, symbolic flows, or parametric curves can deliver μs–ms-per-sample speeds (Nockolds et al., 14 Jan 2025, Klötergens et al., 2024, Khilchuk et al., 14 Dec 2025).

Limitations include potential reduction in mechanistic interpretability relative to physics-informed surrogates, reliance on the quality of the training distribution for generalization, and in some frameworks, modest data-specific hyperparameter tuning for stability or expressivity.

7. Extensions and Research Directions

Active areas of research and methodological extension include:

• Foundation-model surrogates: Training reusable latent ODE elements for arbitrary assembly in unseen domains, toward foundation surrogates for scientific computing (Chung et al., 5 Jan 2026).
• Learning on unstructured/irregular domains: Generalizing window functions, graph-based encoders, and dynamical assemblies to non-grid and high-dimensional settings.
• Control and data assimilation: Leveraging differentiable single-shot latent surrogates for optimal control, inverse design, and integration into 4D-Var or ensemble filtering pipelines (Chung et al., 24 Sep 2025).
• Nonlinear stochastic extensions: Incorporating SDEs in latent space, coupling to bridges or diffusion models in generative workflows (Khilchuk et al., 14 Dec 2025).
• Interpretability and symbolic regression: Continued development of interpretable surrogates via feature libraries and sparse identification for system discovery (Khilchuk et al., 14 Dec 2025).
• Handling input conditioning and actionable queries: Structured input-latent factorization to enable actionable, zero-shot, and controlled time-series synthesis (Chapfuwa et al., 2022).

Latent ODE surrogates thus form a rigorous, unifying, and rapidly advancing framework for efficient simulation, control, and generative modeling of continuous-time dynamical systems across disciplines.