Neural Stochastic Differential Equations

• NSDEs are continuous-time models where neural networks parameterize drift and diffusion, integrating learnable dynamics with Brownian noise.
• They extend Neural ODEs by incorporating explicit stochastic terms, which improves regularization, generative modeling, and uncertainty quantification.
• NSDE frameworks use numerical solvers like Euler–Maruyama and adjoint methods to efficiently train stable, interpretable models for varied applications.

Searching arXiv for relevant Neural SDE papers to ground the article. arXiv search query: "neural stochastic differential equations" Neural Stochastic Differential Equations (NSDEs) are continuous-time stochastic models in which the drift and diffusion coefficients of an SDE are parameterized by neural networks. They extend Neural ODEs by adding an explicit stochastic term, thereby combining learned vector fields with Brownian forcing, and they have been used for regularization of continuous-depth networks, generative modeling, uncertainty quantification, irregularly sampled time series, interpretable latent dynamics, and data-driven scientific modeling (Liu et al., 2019, Look et al., 2020, Kidger, 2022).

1. Formal definitions and stochastic semantics

A common Itô-form definition writes the hidden state $x(t)\in\mathbb{R}^D$ as

$dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$

where $f_\theta$ is the drift, $L_\phi$ is the diffusion, and $w(t)$ is a standard Wiener process (Look et al., 2020). A closely related formulation used in broader surveys is the Stratonovich system

$$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$

with a readout

$$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$

to produce the observable time series (Kidger, 2022). These formulations differ in stochastic calculus convention, but both place neural parametrizations directly in the coefficients of the stochastic dynamics.

The basic NSDE introduced to stabilize Neural ODEs augments the deterministic flow

$dx_t=f(x_t,t;\theta)\,dt$

by a diffusion term,

$dx_t=f(x_t,t;\theta)\,dt + g(x_t,t;\theta)\,dW_t,$

where $x_t\in\mathbb{R}^n$ is the hidden state, $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$0 is the drift, $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$1 is the diffusion matrix, and $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$2 is Brownian motion (Liu et al., 2019). Special cases include additive noise $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$3 and multiplicative noise $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$4, which already cover several regularization patterns familiar from discrete architectures (Liu et al., 2019).

Across the literature, NSDEs are used in both observable state space and latent space. In latent-variable models, one often places a prior only on the initial latent state $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$5, propagates the latent trajectory with an SDE solver, and reconstructs observations through a decoder (El-Laham et al., 2024, Samota et al., 1 Apr 2026). This places the stochasticity inside the continuous latent dynamics rather than solely in the initial condition.

2. Model classes and structural design choices

One influential interpretation treats NSDEs as the continuous-time analogue of residual architectures with injected noise. In this view, Gaussian noise after each ResNet block is a forward-Euler discretization of an SDE with additive or multiplicative diffusion, and dropout is modeled by matching the Bernoulli mask moments with a Gaussian approximation. In the continuous-time limit, dropout induces a diffusion proportional to the instantaneous drift: $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$6 This framework was introduced precisely to recover regularization mechanisms that were present in discrete networks but absent from the original Neural ODE formulation (Liu et al., 2019).

A second line of work imposes stronger structure on the coefficients. For irregular time series, three stable classes have been proposed: Langevin-type SDEs, Linear Noise SDEs, and Geometric SDEs. The Langevin-type model uses a learned potential and asymptotically constant diffusion; the Linear Noise SDE uses state-multiplicative linear noise; and the Geometric SDE uses the log-linear form

$dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$7

which ensures positivity and makes $dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$8 absorbing (Oh et al., 2024). These constructions are explicitly motivated by stability rather than maximal flexibility.

Interpretability can be strengthened further by restricting the NSDE to overdamped Langevin dynamics in latent space,

$dx(t)=f_\theta(x(t))\,dt + L_\phi(x(t))\,dw(t),$9

where the drift is the gradient flow of a scalar neural potential $f_\theta$0. In this formulation, the learned model admits an energy landscape whose local minima are in one-to-one correspondence with latent states underlying the data, and the same representation supports free-energy visualization, saddle-point analysis, minimum-energy paths, committor functions, and Kramers-type dwell-time analysis (Koop et al., 2022).

Other architectures are more compositional. A constructive framework models function realization through a cascade consisting of a neural SDE for weights, a deterministic dynamical system, and a readout map, with outputs of the form

$f_\theta$1

This shifts emphasis from universal approximation alone to the class of functions realizable under specific structural restrictions (Veeravalli et al., 2023). In heterogeneous-data settings, NSDEs can also be conditioned on static embeddings, so that both drift and diffusion depend on the latent state, time, and an entity embedding such as a district embedding $f_\theta$2 (Samota et al., 1 Apr 2026).

3. Numerical solution, training, and inference

Training NSDEs typically requires discretizing the SDE and differentiating through the solver. The standard first-order Itô integrator is Euler–Maruyama: $f_\theta$3 with $f_\theta$4 (Liu et al., 2019, Kidger, 2022). Surveys of neural differential equations also emphasize Milstein for commutative noise and Stratonovich Heun for Stratonovich systems, alongside the usual discretise-then-optimise and optimise-then-discretise adjoint paradigms (Kidger, 2022).

For memory-efficient gradients, NSDE training adapts the continuous adjoint idea. In the stabilizing Neural SDE framework, the sensitivity $f_\theta$5 satisfies its own SDE,

$f_\theta$6

and jointly solving the state and adjoint-SDE avoids storing the entire forward trajectory, reducing memory to $f_\theta$7 (Liu et al., 2019). In generative settings, training objectives include Wasserstein-GAN losses with neural-CDE discriminators and VAE-style ELBOs with pathwise KL terms derived via Girsanov-type arguments (Kidger, 2022).

A central computational difficulty is that Monte Carlo estimation of transition laws or expected path-functionals is expensive and often poorly calibrated at affordable particle counts. Deterministic Bidimensional Moment Matching (BMM) addresses this by approximating one-step transition kernels with Gaussian moment propagation: horizontal moment matching advances mean and covariance in time, vertical moment matching propagates moments through network layers, and Stein’s lemma approximates input–output cross-covariances. The resulting deterministic kernel can be used for both training and prediction, and the reported calibration quality matches Monte Carlo only after the latter uses tens or hundreds of trajectories (Look et al., 2020).

A different alternative replaces Monte Carlo path averaging by cubature on Wiener space. The expected path-space objective is approximated by a weighted sum of deterministic ODE solutions, gradients are obtained by efficient ODE adjoint methods, and a sparse recombination scheme reduces the number of trajectories needed. Under the assumptions stated in that work, the recombined cubature estimator achieves an $f_\theta$8 rate, improving on standard Monte Carlo $f_\theta$9 and quasi-Monte Carlo $L_\phi$0 (2502.12395).

Related work goes further and removes the solver from inference entirely by learning SDE transition laws directly with conditional normalizing flows. These Neural Stochastic Flows preserve identity at zero gap, the Markov property, and approximate Chapman–Kolmogorov consistency, enabling one-shot sampling between arbitrary times and yielding up to two orders of magnitude speed-ups at large time gaps (Kiyohara et al., 29 Oct 2025). This suggests a broader methodological split between solver-based NSDE learning and direct transition-law modeling.

4. Approximation theory, stability, and other guarantees

The approximation theory of NSDEs has become substantially sharper. For network classes satisfying local uniform approximation under a fixed global linear-growth constraint, the associated neural SDEs can approximate general Itô diffusion SDEs arbitrarily well. In particular, if the target SDE has pathwise uniqueness and a unique strong solution, then for every $L_\phi$1 and compact $L_\phi$2 there exist neural drift and diffusion coefficients such that

$L_\phi$3

The theory covers single-hidden-layer nets with linearly activating activations, deep networks alternating ReLU with another activation, and pure ReLU networks (Kwossek et al., 20 Mar 2025).

A complementary constructive theory studies what kinds of functions can be realized when the NSDE appears inside a cascade. Using both Feynman–Kac representations and Lie-theoretic arguments, that work characterizes realizable function classes, shows how finite-dimensional Lie algebras simplify parametrization, and states that if the activation is nonpolynomial then the Lie algebra generated by the associated controlled vector fields is infinite-dimensional, giving universality, whereas polynomial activation is too restrictive and universality fails (Veeravalli et al., 2023).

Stability results are one of the main reasons NSDEs are studied rather than treated as merely noisy Neural ODEs. For perturbations driven by the same Brownian path, the Neural SDE regularization paper analyzes the difference process and proves that with multiplicative diffusion $L_\phi$4 and drift Lipschitz constant $L_\phi$5,

$L_\phi$6

Whenever $L_\phi$7, small input perturbations are driven back to zero exponentially fast almost surely (Liu et al., 2019). For irregular time-series models, the stable NSDE literature emphasizes that naive coefficient choices may destroy strong existence, induce stochastic destabilization, or make Euler–Maruyama unstable, and it derives Wasserstein contractivity bounds under dissipativity and bounded-noise conditions (Oh et al., 2024).

NSDE theory has also been connected to privacy. For additive Gaussian diffusion,

$L_\phi$8

if $L_\phi$9 is $w(t)$0-Lipschitz in its state, then the map $w(t)$1 is $w(t)$2-differentially private provided

$w(t)$3

With standard composition, the resulting training procedure provides privacy accounting analogous to DP-SGD (Hong et al., 12 Jan 2025).

5. Empirical behavior and domains of application

The earliest empirical studies on NSDE regularization focused on vision models. On CIFAR-10, a shared-parameter Neural ODE reached $w(t)$4 top-1 accuracy, whereas moderate Gaussian or dropout-style SDE noise raised it to $w(t)$5; with test-time stochastic ensemble averaging, SDE+dropout achieved $w(t)$6. The same work also reports $w(t)$7 point average-accuracy gains on CIFAR10-C and Tiny-ImageNet-C and greater resistance to multi-step $w(t)$8-PGD attacks (Liu et al., 2019).

For uncertainty-aware prediction, the deterministic BMM approximation was evaluated on eight UCI regression datasets, time-series classification tasks such as MNIST and IMDB, and time-series modeling benchmarks including Lotka–Volterra, Beijing air-quality, and a 3-DOF robot. It uses MSE, RMSE, NLL, ECPE, and ECE as primary metrics, and reports that BMM reaches the same ECPE/ECE as Monte Carlo only after the latter uses tens or hundreds of trajectories, while also producing more reliable multi-step training (Look et al., 2020).

Irregular and partially observed time series are a major application area. Stable NSDEs were evaluated on interpolation, forecasting, and classification, including PhysioNet Mortality, MuJoCo Hopper, PhysioNet Sepsis, Speech Commands, and 30 UEA/UCR datasets under missingness. The reported results include interpolation MSE of approximately $w(t)$9, AUROC of approximately $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$0 on PhysioNet Sepsis, and accuracy up to $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$1 on Speech Commands, together with minimal degradation under heavy missingness (Oh et al., 2024). A change-point extension models latent dynamics with regime-specific drift and diffusion before and after an unknown change point, alternating between ELBO optimization and change-point updates via particle filtering or a sequential likelihood-ratio detector; on synthetic OU data it reports ELBO values such as $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$2 versus $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$3 when the correct number of change points is used (El-Laham et al., 2024).

Interpretability-driven latent modeling uses Langevin structure. There, minima of the learned energy landscape correspond one-to-one with latent states, simulated exit times are empirically well described by Kramers’ law, and classical molecular-dynamics tools become directly applicable to the learned NSDE (Koop et al., 2022). Adversarial generative learning has also been developed: HGAN-SDEs use a WGAN-style objective with a Hermite-function discriminator and report the lowest MISE, TD, MSE, and MMD across the stated synthetic and real benchmarks, with convergence in approximately $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$4 GPU-hours versus $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$5 for LSTM and $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$6 for CDE-based discriminators (Xu et al., 23 Dec 2025).

In scientific computing and applied probability, NSDEs have been used as learned closure models and control variates. For particle dispersion in large-eddy simulations of turbulence, a neural SDE for the fluid velocity seen by particles matches total particle fluctuating kinetic energy to within $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$7 of DNS data, compared with $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$8 under-prediction when no model is used, and matches the variance of uncorrelated particle velocity to within $$d\mathbf{y}(t)=\mu_\theta(t,\mathbf{y}(t))\,dt+\sigma_\phi(t,\mathbf{y}(t))\circ d\mathbf{W}(t),\qquad \mathbf{y}(0)=\zeta_\theta(v),$$9, compared with $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$0 under-prediction without the model (Williams et al., 2022). In Monte Carlo finance, neural control variates embedded in SDE simulation yield speed-ups up to $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$1 for 1D GBM European calls, $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$2 for 3D call-on-max rainbows, up to $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$3 for Heston, up to $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$4 for Merton jump diffusion, and $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$5 to $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$6 for tempered-stable Lévy examples, while preserving unbiasedness (Hinds et al., 2022).

6. Limitations, misconceptions, and open directions

A recurring misconception is that adding stochasticity automatically improves uncertainty quantification. The deterministic-approximation literature reports the opposite for naive Monte Carlo training: well-calibrated uncertainty estimates from NSDEs can be computationally prohibitive, because finite-sample Monte Carlo corrupts predictive variance unless a very large number of particles is used (Look et al., 2020). Likewise, the stable-NSDE literature emphasizes that diffusion design is not innocuous; poor choices can eliminate strong solutions or destabilize the discretization (Oh et al., 2024).

Another misconception is that generic NSDEs are inherently interpretable. In fact, one motivation for Neural Langevin Dynamics is that unrestricted neural drift and diffusion fields can be hard to interpret or analyze, whereas restricting the model to Langevin structure restores an analyzable energy landscape (Koop et al., 2022). This suggests a trade-off between black-box expressivity and structural analyzability rather than an automatic gain in either direction.

Open problems stated in the literature remain broad. They include sharper bounds on the trade-off between noise strength and drift Lipschitz constants, efficient high-order SDE solvers that preserve stability without blowing up runtime, adaptive or learnable diffusion for regularization, and robustness certificates based on SDE concentration inequalities (Liu et al., 2019). Approximation theory still lacks explicit network-size versus accuracy trade-offs, and extensions to jump SDEs, Lévy noise, rough-path metrics, and quantitative sample-complexity guarantees for learning $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$7 and $$\mathbf{x}(t)=\alpha_\theta\,\mathbf{y}(t)+\beta_\theta$$8 from data remain open (Kwossek et al., 20 Mar 2025). Solver-free transition-law models alleviate some computational burdens, but their Chapman–Kolmogorov property is only approximately enforced and extrapolation beyond the trained time horizon may degrade (Kiyohara et al., 29 Oct 2025).

Taken together, these developments position NSDEs not as a single architecture but as a family of stochastic continuous-depth models spanning regularized residual flows, latent-variable generators, interpretable Langevin systems, stable irregular-time encoders, cubature- and moment-based training schemes, and domain-specific scientific closures. The unifying principle is the replacement of discrete stochastic heuristics or fixed mechanistic coefficients by learned drift–diffusion structure in continuous time, while preserving the analytical vocabulary of stochastic calculus.