LaSDI Taylor Latent ODE Surrogate

• The paper introduces a Taylor-expanded latent ODE framework that integrates ResNet modules and sparse SINDy regression to efficiently model parameterized PDE dynamics.
• It employs latent space embeddings and multi-stage neural architectures to achieve high-accuracy state reconstruction with relative L2 errors under 2% across varied parameter regimes.
• The surrogate offers significant speedups, mesh-independence, and flexibility, ensuring scalable and interpretable emulation of complex dynamical systems.

A LaSDI Latent ODE Surrogate is a reduced-order modeling framework that leverages latent space embeddings and explicit ordinary differential equation structures to construct fast, interpretable surrogates for parameterized, high-dimensional dynamical systems, especially those governed by partial differential equations (PDEs). The LaSDI (Latent Space Dynamics Identification) paradigm employs nonlinear mappings to encode high-fidelity simulation data into a low-dimensional latent space, where the dynamics governing system evolution are modeled by ODEs with the possibility of parameter dependence. Recent advances have established a wide spectrum of LaSDI surrogates, including variants dominated by Taylor-series expansions and residual networks (e.g., P-TLDINets), methods incorporating uncertainty quantification via Gaussian process interpolation (e.g., GPLaSDI), multi-stage architectures (mLaSDI), and robust weak-form approaches (WLaSDI). This article provides a comprehensive technical account emphasizing the P-TLDINet/LaSDI-Taylor style latent-ODE surrogate, as exemplified in "Parametric Taylor series based latent dynamics identification neural networks" (Lin et al., 2024).

1. Core Model Architecture and Latent ODE Formulation

At the heart of the LaSDI Taylor surrogate lies a set of neural network modules jointly trained to encode, propagate, and decode system states:

• Latent State and Parametric Mapping: The primary latent state, $z(t;\mu) \in \mathbb{R}^{N_s}$, is a low-dimensional embedding ($N_s \ll N_u$, where $N_u$ is the full system dimension), with explicit dependence on the parameter vector $\mu \in \mathbb{R}^{N_D}$.
• Time Evolution via Taylor-Expanded ODE: The evolution of $z$ is governed by a parametric ODE:

$\frac{dz}{dt} = f(z, \mu),$

which is approximated by its truncated Taylor series up to order $p$:

$$z_{m+1} \approx z_m + \Delta t F_1(z_m, \mu) + \frac{(\Delta t)^2}{2!} F_2(z_m, \mu) + \cdots + \frac{(\Delta t)^p}{p!} F_p(z_m, \mu),$$

where in practice $p=2$ suffices, and $F_k(\cdot)$ are network outputs representing derivatives up to the $k$th order.

• Architecture:
  • NN_dyn$(z, \mu) \rightarrow \{ F_1, \ldots, F_p \}$: A multi-layer ResNet-based FCNN outputs the set of time derivatives.
  • NN_$z_0(\mu) \rightarrow z(0;\mu)$: A compact FCNN maps parameters to initial latent states.
  • NN_rec$(z, \mu, x) \rightarrow \hat{u}(x)$: A ResNet-augmented FCNN reconstructs high-dimensional states on arbitrary spatial points.

ResNet skip-connections enable these components to implement both nonlinear Taylor expansion terms and learned integration steps, with network flexibility for representing complex latent flows.

2. Training Methodology and Joint Loss Structure

Training proceeds with simultaneous optimization of all submodules—NN_$z_0$, NN_dyn (including implicit Taylor coefficients), NN_rec, and an auxiliary sparse regression matrix—against a composite loss: $$\text{Loss}_\text{model} = \text{Loss}_\text{rec} + \omega_{z_0} \text{Loss}_{z_0} + \omega_{ID} \text{Loss}_{dz/dt} + \omega_\text{coef} \|\Xi\|_2^2.$$

• $\text{Loss}_\text{rec}$: Enforces accurate reconstruction of high-dimensional fields:

$$\left\langle \|\text{NN}_\text{rec}(z(t_\ell), \mu, x_m) - u(x_m, t_\ell)\|^2 \right\rangle_{i,\ell,m}$$

• $\text{Loss}_{z_0}$: Ensures correct initial condition mapping via NN_rec(NN_$z_0(\mu),\mu,x)$.
• $\text{Loss}_{dz/dt}$: Couples the learned latent time derivative, $d z/dt$, to a SINDy-style sparse regression form:

$\dot{z} \approx \Theta(z)\Xi,$

where $\Theta(z)$ is a library (constant, linear terms), and $\Xi$ is jointly trained.

• $\omega$ coefficients: Control relative weighting; $\omega_\text{coef}$ regularizes $\Xi$.

All latent map parameters and sparse-regression coefficients are optimized via standard mini-batch or full-batch stochastic optimization.

3. Parametric Generalization and KNN-IDW Interpolation

To enable generalization over parameter space, each training point $\mu_i$ receives an associated identified coefficient matrix $\Xi_i$. For unseen parameters $\mu^*$:

1. Neighbor Search: Find $K$ nearest neighbor training points under Euclidean distance.
2. Inverse-Distance Weights:

$$w_i(\mu^*) = \frac{1}{d(\mu_i, \mu^*)^p}, \qquad \phi_i(\mu^*) = \frac{w_i(\mu^*)}{\sum_{j=1}^{K} w_{i_j}(\mu^*)}$$

1. Coefficient Interpolation:

$$\Xi^*(\mu^*) = \sum_{j=1}^{K} \phi_{i_j}(\mu^*) \, \Xi_{i_j}$$

This creates a continuous and locally adaptive latent dynamics model, using the interpolated coefficients within the ODE

$\frac{dz}{dt} = \Theta(z) \Xi^*(\mu^*).$

Time propagation then utilizes either classical integrators (e.g., RK4) or the neural Taylor-expansion integrator.

4. Surrogate Evaluation: Accuracy, Speed, and Mesh Independence

The latent ODE surrogate is quantitatively evaluated using the relative $L_2$ error: $$r_{L_2}(\hat{u}, u) = \frac{ \| \hat{u} - u \|_2 }{ \| u \|_2 } \times 100\%.$$ Benchmark studies show that, for both 2D Burgers and lock-exchange flow problems, P-TLDINets maintain $r_{L_2} < 2\%$ on test parameter regimes beyond training coverage. Training is rapid: e.g., 36 minutes for 25 training points (Burgers), 57 s online inference for 225 points, with orders-of-magnitude ($\sim$94× offline, $>$100× online) speedup compared to GPLaSDI/gLaSDI and high-fidelity solvers.

The decoder NN_rec supports arbitrary spatial discretizations, enabling prediction on grids ranging from $50\times50$ or $60\times60$ uniform meshes to unstructured $1278$–$4900$ node configurations, without retraining—errors remain within the 1–2% regime.

5. Interpretability, Flexibility, and Comparison to Classical LaSDI

Unlike classical LaSDI pipelines, which require explicit autoencoder construction for nonlinear encoding/decoding, this Taylor/ResNet approach is lightweight and grid-independent. The inclusion of a sparse-regression SINDy step for the first-order latent dynamics preserves interpretability (the right-hand side of the ODE is a linear combination of known basis functions with learned coefficients). The absence of strict autoencoder modularity circumvents pitfalls in high-frequency or highly nonlinear data regimes, promoting stability, accuracy, and easier optimization.

Additionally, the parametric KNN-IDW interpolation architecture allows seamless adaptation over broad parameter domains, with smooth generalization even when autoencoder-based methods lose latent stability.

6. Online Inference Pipeline and Deployment

The online prediction pipeline executes as follows:

1. Compute $\Xi^*(\mu^*)$ via KNN-IDW.
2. Set $z(0) =$NN_$z_0(\mu^*)$.
3. Iteratively update:

$$z_{m+1} = z_m + \Delta t \cdot F_1(z_m, \mu^*) + \frac{\Delta t^2}{2} F_2(z_m, \mu^*)$$

1. For each $t_m$ and spatial $x$, predict $\hat{u}(x, t_m)=$NN_rec$(z_m, \mu^*, x)$.

The entire stack provides an adaptable latent-ODE surrogate for parametric PDEs with mesh-independence and interpretable dynamic structure. The framework's generality extends to nonlinear systems, multi-physics, and high-dimensional parameterizations, as explored in recent theory and applications (Lin et al., 2024).

---

Summary Table: Key Structural Components of the LaSDI Taylor Latent ODE Surrogate

Module	Function	Model Type
NN_dyn($z$, $\mu$)	Outputs $[F_1(z,\mu),\dots,F_p(z,\mu)]$ (time derivatives)	ResNet-based FCNN
NN_$z_0(\mu$)	Maps parameters to latent initial condition $z(0;\mu)$	FCNN
NN_rec($z$, $\mu$, $x$)	Reconstructs field $\hat{u}(x)$ from latent and parameter	ResNet-augmented FCNN
SINDy-style sparse coefficient $\Xi$	Enforces sparse interpretable structure in first derivative $F_1$	Linear regression
KNN-IDW ($\mu^*$)	Interpolates $\Xi(\mu^*)$ from training coefficients for new parameters	Nonparametric

---

For additional methodological context and latent-ODE algorithmic variants (classical LaSDI, GPLaSDI, mLaSDI, WLaSDI), see (Bonneville et al., 2024, Bonneville et al., 2023, Anderson et al., 10 Jun 2025, Tran et al., 2023). The LaSDI Taylor (P-TLDINet) approach fundamentally expands the tractable regions of parametric PDE surrogate modeling, offering a scalable, interpretable, and highly efficient solution for scientific emulation tasks.