Neural Stochastic Differential Equations

• Neural SDEs are continuous stochastic models where neural networks parameterize drift and diffusion, capturing both deterministic trends and random perturbations.
• They employ varied training methods such as simulation-free maximum likelihood, variational inference, and adversarial distribution matching to manage complex dynamics efficiently.
• Applications span fields like turbulence, finance, neuroscience, and spatio-temporal graph modeling, with extensions to handle jump dynamics and memory effects.

A neural stochastic differential equation model is a continuous-time stochastic dynamical model in which the drift and diffusion terms of an SDE are parameterized by neural networks. In the standard Brownian setting, it extends a Neural ODE by replacing deterministic hidden-state evolution with stochastic dynamics of the form $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t$, thereby representing both deterministic trends and stochastic perturbations; later formulations also use time-invariant state-dependent coefficients, latent state-space constructions, jump mechanisms, or Lévy drivers (Liu et al., 2019, Shen et al., 31 Jan 2025, ElGazzar et al., 2024, Yang et al., 2021). The label covers a family of related models rather than a single architecture, and some closely related methods are explicitly presented as alternatives to standard path-simulation-based neural SDEs rather than as neural SDEs in the usual sense (O'Leary et al., 2021).

1. Canonical formulation

The canonical neural SDE writes the hidden state as a stochastic process whose infinitesimal evolution is governed by a neural drift and a neural diffusion. In the basic formulation introduced as a stochastic extension of Neural ODE, the hidden representation follows

$dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$

where $f$ is the drift term, $g$ is the diffusion term, and $W_t$ is standard Brownian motion (Liu et al., 2019). A later continuous-domain sequence-modeling formulation uses a time-invariant state-dependent system

$dx_t=f(x_t)\,dt+g(x_t)\,@\,dW_t,$

with diagonal diffusion

$$g(x_t)=\operatorname{diag}(\sigma_1(x_t),\sigma_2(x_t),\ldots,\sigma_d(x_t)),$$

so that the stochasticity is learned from data rather than fixed a priori (Shen et al., 31 Jan 2025).

Discrete observations are typically connected to the continuous system through Euler–Maruyama. In the time-invariant formulation,

$$X_{t+\Delta t}=X_t+f(X_t)\Delta t+g(X_t)\,@\,\Delta W_t,$$

with $\Delta W_t\sim\mathcal N(0,\Delta t I_d)$, which implies a Gaussian one-step transition

$$p(x_{t+\Delta t}\mid x_t)=\mathcal N(x_{t+\Delta t};\mu_t,\Sigma_t),$$

where $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$0 and $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$1 (Shen et al., 31 Jan 2025). This Markovian transition structure is central to several training procedures: the model conditions on consecutive states rather than on the full history, in contrast to standard autoregressive sequence models (Shen et al., 31 Jan 2025).

A latent-state version treats observed data as partial noisy readouts of an underlying stochastic process. In that setting the latent dynamics are written as

$dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$2

with observation models such as $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$3 and $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$4 (ElGazzar et al., 2024). This formulation shifts the neural SDE from direct state forecasting to continuous-time probabilistic state-space modeling.

2. Learning objectives and inference schemes

For directly observed trajectories, one major training strategy is maximum likelihood based on the Euler–Maruyama transition density. In the continuous-domain sequence model, the negative log-likelihood for a single transition contains a prediction-error term scaled by inverse diffusion and a log-determinant term that penalizes excessively large diffusion. The same work derives a simulation-free maximum likelihood scheme because training uses the closed-form Euler–Maruyama transition density between observed consecutive states rather than forward SDE simulation, backward adjoint SDE solves, or pathwise sampling of continuous trajectories (Shen et al., 31 Jan 2025). In that setting, the diffusion coefficients are analytically matched to the residual error of the flow prediction, and the flow objective becomes a logarithmic squared loss (Shen et al., 31 Jan 2025).

A second major strategy is variational inference over latent paths. In a neuroscience formulation, the approximate posterior is another neural SDE,

$dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$5

which shares diffusion with the prior. This shared diffusion makes it possible to use Girsanov’s theorem to obtain a tractable KL divergence between path measures, yielding an ELBO consisting of expected observation log-likelihood minus KL terms for the initial state and the path measure (ElGazzar et al., 2024). Training uses Euler–Maruyama integration, Monte Carlo estimation of expectations, and backpropagation through the solver (ElGazzar et al., 2024).

A third family uses adversarial distribution matching. In HGAN-SDEs, the generator is a Neural SDE and the learning objective is a Wasserstein GAN-style min–max problem over trajectory distributions,

$dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$6

with a Hermite-guided discriminator designed to approximate path-level dynamics while reducing runtime complexity and improving training stability relative to Neural CDE discriminators (Xu et al., 23 Dec 2025). This is explicitly a likelihood-free training method motivated by the fact that exact likelihoods are generally intractable for SDEs (Xu et al., 23 Dec 2025).

Because Monte Carlo uncertainty propagation can be expensive and noisy, deterministic approximations have also been developed. Bidimensional Moment Matching approximates the transition kernel by propagating Gaussian moments horizontally across time and vertically through neural-network layers, including a backward moment-matching step for the expected Jacobian used in cross-covariance terms. The method is applicable to both training and prediction and is presented as a deterministic approximation to neural SDE transition kernels when Monte Carlo calibration is computationally prohibitive (Look et al., 2020). Earlier work on Neural SDE networks also derived a pathwise gradient estimator and an adjoint-like sensitivity process to train with $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$7 memory rather than storing all intermediate states (Liu et al., 2019).

3. Model families and structural extensions

Brownian neural SDEs are only one branch of the broader class. For event-driven data, Neural Jump Stochastic Differential Equations introduce a point-process jump term,

$dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$8

so that the latent trajectory is piecewise continuous and discontinuities occur at stochastic event times determined by a conditional intensity $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,$9 (Jia et al., 2019). For heavy-tailed and jumpy time series, LDE-Net replaces Brownian motion with symmetric $f$0-stable Lévy motion,

$f$1

with $f$2, in order to model abrupt transitions, heavy tails, and jumps more directly than Brownian-noise neural SDEs (Yang et al., 2021).

Several extensions modify the state space or the dependency structure rather than the noise law. Graph Neural SDEs place the stochastic latent dynamics on graph representations,

$f$3

and use variational latent-path inference to obtain predictive uncertainty for static and spatio-temporal graph tasks (Bergna et al., 2023). CGNSDE constrains the model to a conditional Gaussian form in which hidden variables remain linear in the unobserved state, so that analytic data-assimilation formulae for posterior mean and covariance remain available even after adding neural-network residual terms (Chen et al., 2024). Neural stochastic differential equations on compact polyhedral state-spaces impose viability-preserving constraints on drift and diffusion through a weighted sums parameterization, preserving continuous dynamics and amenability to higher-order solvers inside bounded domains (Liu et al., 23 Aug 2025).

Memory effects lead beyond the Markovian neural SDE. Neural stochastic Volterra equations replace local-in-time coefficients by convolution kernels,

$f$4

and recover the neural SDE as the special case $f$5 and $f$6 (Prömel et al., 2024). This is presented as a strict generalization of neural SDEs for path-dependent dynamics with memory and rough behavior (Prömel et al., 2024).

A distinct boundary case is SPINODE. Although closely related to neural SDE modeling, it is explicitly described as not itself a standard neural SDE model in the usual “learn the stochastic dynamics by simulating sample paths” sense. Instead, it inserts a neural network into a partially known SDE,

$f$7

propagates uncertainty through the known SDE structure by the unscented transform, converts the problem into deterministic ODEs for moments, and trains by matching moment trajectories rather than raw stochastic paths (O'Leary et al., 2021). This places SPINODE closer to a moment-based neural ODE inverse problem for SDEs than to a direct neural SDE solver (O'Leary et al., 2021).

4. Applications across scientific and engineering domains

Neural SDE models have been used as general-purpose continuous-domain sequence models. A 2025 formulation evaluates a time-invariant Neural SDE on multi-modal trajectory generation, Push-T imitation learning, and video prediction on KTH and CLEVRER. In that study, the model achieves comparable or better performance on FVD, JEDI, SSIM, and PSNR, and requires only 2 steps to achieve results comparable to Flow Matching and PFI, which often need 5 to 20 steps (Shen et al., 31 Jan 2025). The same work also reports free temporal interpolation and a strong power-law relationship between model size and validation loss, especially on CLEVRER (Shen et al., 31 Jan 2025).

In turbulence modeling, neural SDEs have been used as closures for unresolved particle-scale velocity fluctuations in large-eddy simulation. For decaying homogeneous isotropic turbulence, a neural SDE closure for the fluid velocity seen by particles learns drift and diffusion coefficients from DNS data. Total particle fluctuating kinetic energy is under-predicted by 40% with no model, whereas the proposed model matches total particle fluctuating kinetic energy to within 5% of the DNS data for low to high-inertia particles; for inertial particles, the variance of uncorrelated particle velocity is matched to within 10% of DNS results, compared to 60-70% under-prediction with no model (Williams et al., 2022). This application illustrates neural SDEs as data-driven stochastic closures rather than generic sequence models (Williams et al., 2022).

Financial modeling has motivated both Brownian and Lévy-driven neural stochastic systems. LDE-Net applies a Lévy-induced stochastic differential equation network to chaotic stock-index forecasting and reports that LDE-Net slightly but consistently outperforms SDE-Net and other baselines; on the SSE Energy Index, the reported MSEs are 0.0011 for SDE-Net and 0.0009 for LDE-Net, with corresponding RMSEs 0.0335 and 0.0295 (Yang et al., 2021). A different finance-oriented system, SDBS, combines simulated SDE sample paths of multi-asset prices with the Black–Scholes PDE to learn option values and Greeks, showing that sufficiently smooth activation functions and the Black–Scholes constraint contribute significantly to accurate option pricing (Chi, 2020).

In neuroscience, latent neural SDEs have been used to model single-trial neural population dynamics as partial observations of an underlying stochastic continuous-time system. A coupled-oscillator SDE hybrid achieves competitive neural and behavioral prediction across three datasets while using only about 465–526 parameters for the generative dynamics, compared with 4,944 for a neural SDE and 33,280 for an LSTM (ElGazzar et al., 2024). On perturbed reach, the coupled oscillator SDE reaches 0.238 ± 0.008 bits per spike and 0.86 ± 0.018 $f$8; on visual decision making it reaches 0.22 ± 0.011 bits per spike and 0.67 ± 0.025 $f$9 (ElGazzar et al., 2024). On graphs, latent Graph Neural SDEs are reported to surpass Graph Convolutional Networks and Graph Neural ODEs especially in confidence prediction and out-of-distribution detection (Bergna et al., 2023). For inverse problems without path supervision, drift and diffusion can instead be identified from stationary distributions by combining a Hellinger-distance density loss with a stationary Fokker–Planck residual (Chen et al., 2023).

5. Theoretical analyses and interpretive frameworks

The theory of neural SDEs has developed along several axes: robustness of stochastic depth-like dynamics, convergence of discretized learning procedures, viability on constrained domains, and constructive characterization of realizable function classes. In the continuous-depth regularization view, neural SDEs were introduced partly to import dropout, additive noise, multiplicative noise, stochastic depth, and shake-shake-like randomization into Neural ODE networks. Under Lyapunov-type conditions, the diffusion can stabilize perturbation dynamics; for the special case

$g$0

if the drift is $g$1-Lipschitz and $g$2, then the zero solution is almost surely exponentially stable (Liu et al., 2019). This is one of the clearest formal statements that stochasticity can improve robustness rather than merely inject noise (Liu et al., 2019).

A constructive geometric perspective treats neural SDEs as cascade systems. One formulation studies

$g$3

and proves the Feynman–Kac representation

$g$4

The same work uses finite-dimensional Lie algebra structure to characterize realizable function classes and shows that sphere-valued and orthogonal-group-valued diffusions arise when the generating matrices are skew-symmetric (Veeravalli et al., 2023). A plausible implication is that neural SDE expressivity can be analyzed through geometry of stochastic flows as much as through parameter count.

Convergence theory has been developed both for specialized architectures and for training algorithms. LDE-Net proves convergence in probability in the Skorokhod space $g$5 to the target Lévy-driven SDE and states that the convergence bound is independent of state dimension, thereby avoiding the curse of dimensionality (Yang et al., 2021). Neural SVEs establish an $g$6-stability result showing that if two Volterra equations have close coefficients and kernels, then their solution processes are close, providing the approximation rationale for kernel-parameterized path-dependent models (Prömel et al., 2024). For compact polyhedral domains, viability theory is used to derive boundary conditions under which the process remains in the domain with probability $g$7, and a weighted sums parameterization is then proved to satisfy these conditions while keeping the dynamics continuous (Liu et al., 23 Aug 2025).

6. Limitations, identifiability, and common misconceptions

A recurring limitation is that exact likelihoods are generally intractable, stochastic simulation can produce noisy gradients, and adversarial SDE learning can be unstable. HGAN-SDEs is motivated precisely by the claim that prior GAN-based discriminators for SDE path distributions, especially Neural CDE discriminators, suffer from high computational costs and exacerbate instability of adversarial training (Xu et al., 23 Dec 2025). Deterministic alternatives such as Bidimensional Moment Matching report that obtaining well-calibrated uncertainty estimations from neural SDEs by Monte Carlo sampling is computationally prohibitive, and that Monte Carlo matches their calibration quality only after introducing high computational cost (Look et al., 2020).

Identifiability is also nontrivial. In stationary-distribution learning, if both drift and diffusion are completely free, the inverse problem is non-unique because the trivial choice $g$8, $g$9 also minimizes the stationary Fokker–Planck residual; partial drift observations are introduced to break this degeneracy (Chen et al., 2023). Moment-based methods face an analogous issue: SPINODE requires repeated trajectories from identical initial conditions to estimate moments accurately, and different parameter sets can match the same moments, so moment fitting may not uniquely identify the true distribution (O'Leary et al., 2021). These are not failures specific to a single architecture; they are structural inverse-problem ambiguities in stochastic system identification.

Another common misconception is that all neural-SDE-related models learn by directly simulating stochastic paths. SPINODE explicitly does not: it learns hidden physics inside an SDE by converting stochastic dynamics into deterministic ODEs for moments and matching mean and covariance trajectories (O'Leary et al., 2021). Neural Laplace for SDEs departs in a different direction, by learning in Laplace space after defining the Laplace transform pathwise for a stochastic process; however, the evidence presented there is restricted to Geometric Brownian Motion in a strong-drift regime, includes no comparison with neural SDE methods, and reports that Neural Laplace is not the best performer in that particular setup (Carrel, 2024).

The simplifying assumptions used in many neural SDE papers remain significant. Reported examples include time-invariant dynamics, diagonal diffusion, Markovian dependence, Euler–Maruyama discretization, or bounded domains that require special viability constructions (Shen et al., 31 Jan 2025, Liu et al., 23 Aug 2025). This suggests that “neural stochastic differential equation model” is best understood not as a single settled methodology, but as a technical family of continuous-time stochastic models whose practical value depends on how drift, diffusion, observation model, state-space geometry, and training objective are matched to the scientific or engineering problem.