Stochastic NODE-DMD: Probabilistic Dynamics

• The paper introduces a probabilistic framework that extends DMD by integrating neural ODEs and latent SDEs to model latent nonlinear dynamics and quantify uncertainty.
• It leverages an implicit neural decoder and variational inference to reconstruct continuous spatiotemporal fields from sparse and noisy sensor data.
• Empirical results demonstrate improved mode recovery, accurate eigenvalue estimation, and enhanced spatial resolution flexibility compared to classical methods.

Stochastic NODE-DMD is a probabilistic framework for learning the dynamics of partially observed, continuous spatiotemporal systems from sparse sensor data. It extends Dynamic Mode Decomposition (DMD) by combining linear spectral interpretability with the nonlinear modeling capacity of Neural Ordinary Differential Equations (NODEs) and uncertainty quantification through a latent stochastic differential equation (SDE) formulation. The method enables continuous-time, continuous-space field reconstruction with rigorous predictive uncertainty estimates, interpretable dynamical modes, and the recovery of the underlying linear and nonlinear structure from severely under-sampled and noisy observations (Kim et al., 25 Nov 2025).

1. Latent Nonlinear Dynamics and Stochastic Formulation

The backbone of Stochastic NODE-DMD is a latent evolution law for the mode amplitudes $z(t)\in\mathbb{C}^r$, capturing the temporal dynamics in a reduced dimension. The dynamics are governed by a continuous-time SDE:

$$dz = [\Lambda z(t) + f_\theta(z(t), t)]\,dt + \tau\,dB_t,$$

where:

• $$\Lambda = \text{diag}(\lambda_1,\ldots,\lambda_r) \in \mathbb{C}^{r \times r}$$ is a diagonal matrix containing classical DMD eigenvalues,
• $$f_\theta: \mathbb{C}^r \times \mathbb{R} \rightarrow \mathbb{C}^r$$ is a neural network parameterized by $\theta$, modeling nonlinear residual drift,
• $\tau^2$ is the process noise variance,
• $B_t$ is complex Brownian motion.

This stochastic NODE recovers classical, linear DMD as the limiting case (i.e., $\tau \to 0$ and $f_\theta \to 0$). The neural residual $f_\theta$ is modeled by an MLP (3–4 layers, 64–128 units, ELU activation), equipped with regularization via $L_2$ weight decay or spectral-norm constraints on Jacobians to control expressiveness and prevent overfitting.

2. Observation Model and Field Reconstruction

Field reconstruction from latent dynamics is achieved through a neural implicit decoder—termed “Mode Extractor” $W_\psi$. Given a spatial coordinate $x \in \Omega$, a positional encoding

$$\gamma(x) = [x,\ \sin(2^0\pi x),\ \cos(2^0\pi x),\ldots,\sin(2^{L-1}\pi x),\cos(2^{L-1}\pi x)]$$

is computed and fed into a neural network $g_\psi$ to yield $W_\psi(\gamma(x)) \in \mathbb{C}^{m \times r}$. The field mean at time $t$ is reconstructed as:

$$\hat{y}(t, x) = W_\psi(\gamma(x)) z(t) = \sum_{i=1}^r z_i(t) [W_\psi(\gamma(x))]_i.$$

For sensors $S = \{s_1,\ldots,s_m\}$, observations are modeled as

$$y_k = y(t_k, S) + \eta_k, \quad \eta_k \sim \mathcal{CN}(0, \sigma^2 I_m),$$

yielding a complex Gaussian likelihood:

$$p(y_k | z(t_k)) = \mathcal{CN}(W_\psi(\gamma(S)) z(t_k), \sigma^2 I_m).$$

3. Variational Inference and Training Objective

The intractable posterior over latent trajectories is approximated by a factorized variational distribution, with an amortized encoder providing the initial posterior:

$$q(z(0) | y_{0:T}) = \mathcal{CN}(\mu_\phi, \Sigma_\phi),$$

where $(\mu_\phi, \Sigma_\phi) =$ Encoder$_\phi(y_{0:T},\gamma(S))$.

Temporal evolution of uncertainty is handled via uncertainty-aware Euler–Maruyama integration:

• Means: $$\mu_{k+1} = \mu_k + \Delta t [\Lambda \mu_k + f_\theta(\mu_k, t_k)]$$,
• Covariances:

$$\Sigma_{k+1} = (I + \Delta t J_k) \Sigma_k (I + \Delta t J_k)^H + \Delta t \tau^2 I$$

with $$J_k = \partial[\Lambda z + f_\theta(z, t)]/\partial z \big|_{z = \mu_k}$$.

The evidence lower bound (ELBO) per batch is:

$$\mathcal{L} = \mathbb{E}_q\left[ \sum_{k=0}^{K-1} \log p(y_{k+1}|z(t_{k+1})) \right] - KL[q(z(0)|y_{0:T}) \| p(z(0))] + \text{consistency-loss},$$

involving:

• A Gaussian NLL reconstruction term,
• Latent KL divergence against $\mathcal{CN}(0,I)$,
• A consistency loss aligning encoder and SDE-propagated marginals (using mean square error and a small KL term with weight $\kappa \approx 10^{-3}$).

Total loss weights are typically $w_\text{recon} = 3$, $w_\text{kl} = 10^{-3}$, $w_\text{cons} = 0.15$.

4. Extraction of Dynamical Structure and Spectral Factors

After training, $f_\theta$ and $\Lambda$ encode both the nonlinear and linear dynamics. The local linearization at a nominal state $z^*$ yields a Jacobian $J^*$. The eigenvalues of $J^*$ correspond to continuous-time DMD eigenvalues, and the right eigenvectors $v_i$ are latent-space Koopman modes. Spatial mode functions are reconstructed as $W_i(x) = W_\psi(\gamma(x)) v_i$.

For classical comparison, discrete DMD eigenvalues are obtained via $\mu_i = \exp(\lambda_i \Delta t)$, where $J^* v_i = \lambda_i v_i$. Empirical mode and eigenvalue recovery is quantifiable by comparing to ground truth, with synthetic experiments using the log-ratio estimator:

$$\widehat{\lambda}_i = \operatorname{median}_t \left[ \frac{\log(z_i(t+\Delta t)/z_i(t))}{\Delta t} \right].$$

5. Uncertainty Quantification and Continuous Spatiotemporal Queries

Stochastic NODE-DMD enables principled uncertainty quantification. Samples $z^{(s)}(0)$ from the posterior are propagated through the neural SDE to generate ensemble predictions $\hat{y}^{(s)}(t,x)$, with empirical variance reflecting both epistemic and aleatoric sources. Optionally, a Laplace approximation around MAP $\theta^*$ can quantify parameter uncertainty in $f_\theta$.

The implicit decoder $W_\psi$ enables field reconstruction at arbitrary $x \in \Omega$, affording spatial resolution refinement without retraining.

6. Algorithmic Steps and Training Protocol

The method comprises the following core steps:

1. Encode $y_k$ and $\gamma(S)$ to obtain $q(z(t_k)|y_{0:k})$.
2. Extract mean $\mu_k$ or sample from $q$.
3. Integrate the latent SDE (Euler–Maruyama) to get $(\mu_{k+1}, \Sigma_{k+1})$.
4. Decode $\hat{y}_{k+1}(S)$ using $W_\psi(\gamma(S)) \mu_{k+1}$; compute NLL.
5. Accumulate latent KL and consistency losses.
6. Backpropagate through the sequence and update $(\theta, \phi, \psi)$ to maximize the ELBO.
7. Employ curriculum learning starting with teacher forcing, annealed to full autoregressive prediction.

Critical hyperparameters include mode rank $r$ (4 for synthetic, 8 for PDE flows), positional encoding bands $L=10$, process noise $\tau$ (tuned per dataset), the specified loss weights, Adam optimizer ($\mathrm{lr}=10^{-3}$), batch size 16, and training for $\sim200$ epochs.

7. Benchmark Results and Structural Properties

Empirical evaluation demonstrates significant advantages in reconstructing spatiotemporal fields from sparse and noisy data:

• Synthetic sequence (r=4, 32×32, T=50, 10% sensors):
  • Recovered modes exhibit cosine similarity $\sim0.71$ to true modes.
  • Continuous eigenvalue error $\approx0.61$ versus NDMD's $1.78$.
• Gray–Scott (r=8, 100×100, T=100):
  • 1-step $L_1$ error $\sim4.6\times10^{-3}$ vs. NDMD $4.9\times10^{-3}$ with 10% sensors.
• 2D Navier–Stokes vorticity (r=8, 100×100, T=50):
  • 1-step $L_1$ error $\sim2.4\times10^{-3}$, NDMD $4.7\times10^{-3}$.
• Cylinder flow (r=8, 128×128, T=150):
  • 1-step $L_1$ error $\sim6.7\times10^{-3}$, NDMD $4.6\times10^{-2}$.

The method demonstrates calibrated uncertainty: when trained on multiple realizations, it learns a distribution across latent trajectories matching ensemble variability, avoiding regression to the mean.

Spatial resolution flexibility is evidenced by increased error of $<10\%$ when queried at both coarser or finer resolutions than those used for training, attributable to the continuous-space implicit neural parameterization.

Stochastic NODE-DMD achieves an overview of DMD's interpretability with neural ODE expressivity and Bayesian rigor, delivering continuous spatiotemporal prediction and uncertainty quantification from sparse, noisy sensors (Kim et al., 25 Nov 2025).