GM-SDE/ODE Solvers: Uncertainty in Numerical Integration

• GM-SDE/ODE solvers are probabilistic numerical integrators that use Gauss–Markov processes and Gaussian mixtures to provide uncertainty quantification in solving differential equations.
• They generalize classical solvers by embedding Bayesian filtering and smoothing techniques, yielding adaptive step-size control and robust derivative correction.
• These methods enable efficient forward simulation, Bayesian inference, and generative modeling while balancing high-order accuracy with computational cost.

GM-SDE/ODE solvers are a class of probabilistic numerical integrators for ordinary and stochastic differential equations that propagate uncertainty in the numerical solution using Gauss–Markov processes or Gaussian mixtures. They encompass a spectrum of methods, from ODE filtering/smoothing based on linear time-invariant SDEs to explicit mixture models that generalize classical Itô–Taylor integrators. GM-SDE/ODE approaches have found central application in robust uncertainty quantification for forward simulation, Bayesian inference, and accelerated generative modeling via diffusion or flow-matching techniques.

1. Gauss–Markov SDE/ODE Solver Foundations

The core of many GM-SDE/ODE methods is the embedding of ODE or SDE integration within a Gauss–Markov process prior, commonly formulated as a linear time-invariant SDE over an augmented state: $dX(t) = F X(t)\,dt + L\,dW(t)$ where $X(t)$ collects the solution and its derivatives, $F$ is a companion (shift) matrix, $L$ injects process noise into the highest derivative, and $W(t)$ is a Wiener process. This construction yields a joint Gaussian process over the trajectory, allowing for principled Bayesian filtering or smoothing to produce both mean trajectories and uncertainty estimates (Schober et al., 2016, Kersting et al., 2016).

Classical ODE solvers are recovered as limiting cases:

• The filter mean for Gauss–Markov priors with suitable parameters coincides with Nordsieck multistep (for IWP(q) priors) or with explicit Runge–Kutta methods for appropriate choices of $F, L$ and measurement operators (Schober et al., 2014, Schober et al., 2016).
• The innovation and gain structure in filter updates directly correspond to derivative corrections and adaptive step-size control.

Crucially, these solvers propagate a covariance (posterior uncertainty) at every step: $C_n = C_n^- - K_n (H_1 C_n^- H_1^T) K_n^T$ Quantifying the epistemic numerical uncertainty, with theoretical contraction rates matching $O(h^{2q})$ for IWP(q) priors and global mean error rates $O(h^{q})$.

2. Gaussian Mixture-Based Solvers for SDEs and ODEs

Unlike single-Gaussian GM processes, Gaussian mixture schemes replace each one-step transition of the SDE or ODE with a finite mixture: $$\mu(X^{n+1} \mid X^n = x_n) \approx \sum_{i=1}^M w_i \mathcal{N}(m_i, \Sigma_i)$$ The crucial insight is that, for high weak order of accuracy, the mixture parameters can be constructed to match the generator expansion of the SDE to a desired order with respect to test functions—a constraint unattainable by single Gaussians for large time steps or multimodal data (Li et al., 2018, Guo et al., 2023).

• In the SDE context, mixtures enable high weak order (up to 2) schemes without recourse to high-order Itô–Taylor expansions or multiple iterated integrals.
• For ODEs, mixtures with symmetrically placed "beams" recover collocation-like effects, enabling Runge–Kutta-like integrators with robust uncertainty quantification.
• For score-based diffusion models, fitting a Gaussian mixture to match the first three moments (via a generalized method of moments on extra network heads) delivers significant reduction in discretization error when only a few integration steps are used (Guo et al., 2023).

Empirically, Gaussian mixture solvers outperform strict single-Gaussian approaches in low-NFE (number of function evaluations) regimes and provide more expressive uncertainty (Guo et al., 2023, Li et al., 2018).

3. Filtering, Smoothing, and MAP Statistical Structure

GM-SDE/ODE solvers unify (extended) Kalman filtering, Rauch–Tung–Striebel smoothing, and probabilistic inference. The maximum a posteriori estimate (MAP) over the whole solution path is equivalent to a minimum-norm Sobolev interpolant in the reproducing kernel Hilbert space (RKHS) associated with the SDE prior (Tronarp et al., 2020).

Concretely:

• Explicit solvers: No linearization of $X(t)$0, exact MAP when $X(t)$1 is constant.
• Semi-implicit/linearized solvers: First-order Taylor expansion, exact for affine $X(t)$2.
• Implicit solvers (IEKS): Iterated linearization about the current trajectory, yielding local polynomial convergence rates in fill-distance (Tronarp et al., 2020).

This yields convergence rates: $X(t)$3 where $X(t)$4 is the maximum mesh step and $X(t)$5 is the regularity of the prior.

4. High-Order and Efficiency in Generative Modeling

Recent advances in DM sampling leverage mixed SDE/ODE GM solvers to address the efficiency–effectiveness gap. Key developments:

• Mixed SDE/ODE scheduling (with sliding windows): Integrate SDE sampling and RL updates only within a window, and exploit high-order ODE samplers outside for efficient NFE allocation (Li et al., 29 Jul 2025). This enables nearly 3.4x speed-up in RL-aligned diffusion sampling with minor or negligible generation quality loss.
• Exponential Integrator GM-SDE Solvers (SEEDS): Analytically integrate the linear SDE/SDE portion and match higher-order deterministic corrections in a strong/weak order 1 scheme, outperforming or matching leading ODE/SDE samplers in NFE–FID trade-offs (e.g., 1.38 FID @ 270 NFE on ImageNet 64) (Gonzalez et al., 2023).
• Stochastic Adams (SA-Solver): Linear multi-step stochastic integrators generalizing ODE Adams–Bashforth/Moulton schemes, with controlled stochastic variance and empirical SOTA FID at low NFE (Xue et al., 2023). These methods decouple step size from trajectory diversity and permit stochasticity scheduling.

5. Uncertainty Quantification and Calibration

A major thrust of GM-SDE/ODE is numerical uncertainty quantification. Posterior covariances or mixtures provide explicit epistemic error measures. Challenges and solutions include:

• Naive GP/Kalman approaches may underestimate error due to ignored gradient noise; active uncertainty calibration methods (e.g., Bayesian quadrature filtering) correct the covariance using function evaluations at multiple points, leading to well-calibrated posteriors without prohibitive cost (Kersting et al., 2016).
• In parameter-uncertain systems, solvers must account for both numerical and parametric uncertainty—naively probabilistic integrators do not automatically propagate model parameter uncertainty correctly and require explicit marginalization (e.g., via numerical quadrature) to yield physically meaningful posteriors [(Yao et al., 6 Mar 2025)—abstract only, no methods provided].
• Table 1 summarizes representative solver classes:

Method Class	Order	Uncertainty Type
GM ODE Filter	q	Gaussian (covariance)
GM–Mixture SDE	2 (weak)	Mixture-of-Gaussians
Exponential SEEDS	1	Exact linear/stat. GP
SA-Solver (Adams)	s (strong/weak)	Gaussian; stochastic variance
MAP ODE Smoothing	polynomial	Gaussian (global)

6. Computational Complexity and Practical Considerations

• Classic GM ODE filters: $X(t)$6 per-step cost for IWP(q), dominated by small-matrix multiplications and inversion (Schober et al., 2016).
• Gaussian mixture SDE solvers: $X(t)$7 per eigendecomposition, but linear in $X(t)$8 per sample given that only one mixture component is sampled per trajectory (Li et al., 2018).
• High-order DPM solvers: Cost per step scales with the number of model calls (e.g., $X(t)$9 in SEEDS-k), which is comparable to k-stage Runge–Kutta methods (Gonzalez et al., 2023).
• Implementation for parametric uncertainty requires integration over the parameter posterior, typically using numerical quadrature [(Yao et al., 6 Mar 2025)].

Choice of prior, step-size, mixture order, and kernel smoothness are critical. For ODE filtering, adaptivity can be based on innovation size or posterior covariance. In Gaussian mixture approaches for diffusion models, the number of mixture heads and accuracy of moment estimation directly impact expressiveness and sample quality (Guo et al., 2023).

7. Extensions and Limitations

• GM-SDE/ODE-based samplers are being generalized to boundary value and PDE solvers by extending the process prior over higher-dimensional domains (Barber, 2014).
• For diffusion generative modeling, mixture-based reverse kernels empirically outperform single-Gaussian SDEs and provide robustness in regimes with minimal function evaluations (Guo et al., 2023, Li et al., 29 Jul 2025).
• GM-SDE/ODE solvers inherit the stability and convergence properties of their underlying prior/filter structure; when the prior smoothness matches the true solution, asymptotic order and error rates match classical methods.
• Propagation of parametric uncertainty remains a subtle issue, as standard probabilistic integrators may fail to account for it without explicit marginalization [(Yao et al., 6 Mar 2025)—abstract only].

A plausible implication is that as models and simulation domains increase in complexity and the demand for robust, quantifiable uncertainty grows—alongside efficiency constraints—the GM-SDE/ODE framework and its mixture extensions will become increasingly central in probabilistic numerics and generative modeling.