---
title: Neural Stochastic Differential Equations
url: https://www.emergentmind.com/topics/neural-stochastic-differential-equation-model
type: topic
---

# Neural Stochastic Differential Equations

A neural stochastic differential equation model is a continuous-time stochastic dynamical model in which the drift and diffusion terms of an SDE are parameterized by neural networks. In the standard Brownian setting, it extends a Neural ODE by replacing deterministic hidden-state evolution with stochastic dynamics of the form $dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t$, thereby representing both deterministic trends and stochastic perturbations; later formulations also use time-invariant state-dependent coefficients, latent state-space constructions, jump mechanisms, or Lévy drivers [1906.02355], [2501.18871], [2412.12112], [2111.13164]. The label covers a family of related models rather than a single architecture, and some closely related methods are explicitly presented as alternatives to standard path-simulation-based neural SDEs rather than as neural SDEs in the usual sense [2109.01621].

## 1. Canonical formulation

The canonical neural SDE writes the hidden state as a stochastic process whose infinitesimal evolution is governed by a neural drift and a neural diffusion. In the basic formulation introduced as a stochastic extension of Neural ODE, the hidden representation follows
$$
dx_t=f(x_t,t)\,dt+g(x_t,t)\,dW_t,
$$
where $f$ is the drift term, $g$ is the diffusion term, and $W_t$ is standard Brownian motion [1906.02355]. A later continuous-domain sequence-modeling formulation uses a time-invariant state-dependent system
$$
dx_t=f(x_t)\,dt+g(x_t)\,@\,dW_t,
$$
with diagonal diffusion
$$
g(x_t)=\operatorname{diag}(\sigma_1(x_t),\sigma_2(x_t),\ldots,\sigma_d(x_t)),
$$
so that the stochasticity is learned from data rather than fixed a priori [2501.18871].

Discrete observations are typically connected to the continuous system through Euler–Maruyama. In the time-invariant formulation,
$$
X_{t+\Delta t}=X_t+f(X_t)\Delta t+g(X_t)\,@\,\Delta W_t,
$$
with $\Delta W_t\sim\mathcal N(0,\Delta t I_d)$, which implies a Gaussian one-step transition
$$
p(x_{t+\Delta t}\mid x_t)=\mathcal N(x_{t+\Delta t};\mu_t,\Sigma_t),
$$
where $\mu_t=X_t+f(X_t)\Delta t$ and $\Sigma_t=g(X_t)g(X_t)^T\Delta t$ [2501.18871]. This Markovian transition structure is central to several training procedures: the model conditions on consecutive states rather than on the full history, in contrast to standard autoregressive sequence models [2501.18871].

A latent-state version treats observed data as partial noisy readouts of an underlying stochastic process. In that setting the latent dynamics are written as
$$
\dd x(t)=\mu_\theta(x(t),u(t))\dd t+\sigma_\theta(x(t),u(t))\dd w(t),
$$
with observation models such as $\mathbf y_i\sim p(\cdot\mid\lambda_\theta(x_i))$ and $\mathbf b_i\sim p(\cdot\mid\rho_\theta(x_i))$ [2412.12112]. This formulation shifts the neural SDE from direct state forecasting to continuous-time probabilistic state-space modeling.

## 2. Learning objectives and inference schemes

For directly observed trajectories, one major training strategy is maximum likelihood based on the Euler–Maruyama transition density. In the continuous-domain sequence model, the negative log-likelihood for a single transition contains a prediction-error term scaled by inverse diffusion and a log-determinant term that penalizes excessively large diffusion. The same work derives a **simulation-free** maximum likelihood scheme because training uses the closed-form Euler–Maruyama transition density between observed consecutive states rather than forward SDE simulation, backward adjoint SDE solves, or pathwise sampling of continuous trajectories [2501.18871]. In that setting, the diffusion coefficients are analytically matched to the residual error of the flow prediction, and the flow objective becomes a logarithmic squared loss [2501.18871].

A second major strategy is variational inference over latent paths. In a neuroscience formulation, the approximate posterior is another neural SDE,
$$
\dd\tilde x=\nu_\phi(\tilde x,u,c)\dd t+\sigma_\theta(\tilde x,u)\dd w,
$$
which shares diffusion with the prior. This shared diffusion makes it possible to use Girsanov’s theorem to obtain a tractable KL divergence between path measures, yielding an ELBO consisting of expected observation log-likelihood minus KL terms for the initial state and the path measure [2412.12112]. Training uses Euler–Maruyama integration, Monte Carlo estimation of expectations, and backpropagation through the solver [2412.12112].

A third family uses adversarial distribution matching. In HGAN-SDEs, the generator is a Neural SDE and the learning objective is a Wasserstein GAN-style min–max problem over trajectory distributions,
$$
\min_\theta\max_\phi\ \mathbb E_v[D_\phi(G_\theta(v))]-\mathbb E_\mu[D_\phi(\mu)],
$$
with a Hermite-guided discriminator designed to approximate path-level dynamics while reducing runtime complexity and improving training stability relative to Neural CDE discriminators [2512.20272]. This is explicitly a likelihood-free training method motivated by the fact that exact likelihoods are generally intractable for SDEs [2512.20272].

Because Monte Carlo uncertainty propagation can be expensive and noisy, deterministic approximations have also been developed. Bidimensional Moment Matching approximates the transition kernel by propagating Gaussian moments horizontally across time and vertically through neural-network layers, including a backward moment-matching step for the expected Jacobian used in cross-covariance terms. The method is applicable to both training and prediction and is presented as a deterministic approximation to neural SDE transition kernels when Monte Carlo calibration is computationally prohibitive [2006.08973]. Earlier work on Neural SDE networks also derived a pathwise gradient estimator and an adjoint-like sensitivity process to train with $\mathcal O(1)$ memory rather than storing all intermediate states [1906.02355].

## 3. Model families and structural extensions

Brownian neural SDEs are only one branch of the broader class. For event-driven data, Neural Jump Stochastic Differential Equations introduce a point-process jump term,
$$
d\mathbf z(t)=f(\mathbf z(t),t;\theta)\,dt+w(\mathbf z(t),\mathbf k(t),t;\theta)\,dN(t),
$$
so that the latent trajectory is piecewise continuous and discontinuities occur at stochastic event times determined by a conditional intensity $\lambda(\mathbf z(t))$ [1905.10403]. For heavy-tailed and jumpy time series, LDE-Net replaces Brownian motion with symmetric $\alpha$-stable Lévy motion,
$$
dx(t)=f(x(t);\theta_f)\,dt+g(x(t);\theta_g)\,dL_t^\alpha,
$$
with $1<\alpha<2$, in order to model abrupt transitions, heavy tails, and jumps more directly than Brownian-noise neural SDEs [2111.13164].

Several extensions modify the state space or the dependency structure rather than the noise law. Graph Neural SDEs place the stochastic latent dynamics on graph representations,
$$
\mathrm dz(t)=f_\phi(z(t),t,\mathcal G)\,\mathrm dt+\sigma(z_t,t)\,\mathrm dW_t,
$$
and use variational latent-path inference to obtain predictive uncertainty for static and spatio-temporal graph tasks [2308.12316]. CGNSDE constrains the model to a conditional Gaussian form in which hidden variables remain linear in the unobserved state, so that analytic data-assimilation formulae for posterior mean and covariance remain available even after adding neural-network residual terms [2404.06749]. Neural stochastic differential equations on compact polyhedral state-spaces impose viability-preserving constraints on drift and diffusion through a weighted sums parameterization, preserving continuous dynamics and amenability to higher-order solvers inside bounded domains [2508.17090].

Memory effects lead beyond the Markovian neural SDE. Neural stochastic Volterra equations replace local-in-time coefficients by convolution kernels,
$$
X_t=\xi\,g(t)+\int_0^tK_\mu(t-s)\mu(s,X_s)\,ds+\int_0^tK_\sigma(t-s)\sigma(s,X_s)\,dB_s,
$$
and recover the neural SDE as the special case $K_{\mu,\theta}(t-s)\equiv1$ and $K_{\sigma,\theta}(t-s)\equiv1$ [2407.19557]. This is presented as a strict generalization of neural SDEs for path-dependent dynamics with memory and rough behavior [2407.19557].

A distinct boundary case is SPINODE. Although closely related to neural SDE modeling, it is explicitly described as **not itself a standard neural SDE model** in the usual “learn the stochastic dynamics by simulating sample paths” sense. Instead, it inserts a neural network into a partially known SDE,
$$
dx=f(x,g(x))\,dt+h(x,g(x))\,dw,
$$
propagates uncertainty through the known SDE structure by the unscented transform, converts the problem into deterministic ODEs for moments, and trains by matching moment trajectories rather than raw stochastic paths [2109.01621]. This places SPINODE closer to a moment-based neural ODE inverse problem for SDEs than to a direct neural SDE solver [2109.01621].

## 4. Applications across scientific and engineering domains

Neural SDE models have been used as general-purpose continuous-domain sequence models. A 2025 formulation evaluates a time-invariant Neural SDE on multi-modal trajectory generation, Push-T imitation learning, and video prediction on KTH and CLEVRER. In that study, the model achieves comparable or better performance on FVD, JEDI, SSIM, and PSNR, and requires only **2 steps** to achieve results comparable to Flow Matching and PFI, which often need **5 to 20 steps** [2501.18871]. The same work also reports free temporal interpolation and a strong power-law relationship between model size and validation loss, especially on CLEVRER [2501.18871].

In turbulence modeling, neural SDEs have been used as closures for unresolved particle-scale velocity fluctuations in large-eddy simulation. For decaying homogeneous isotropic turbulence, a neural SDE closure for the fluid velocity seen by particles learns drift and diffusion coefficients from DNS data. Total particle fluctuating kinetic energy is under-predicted by **40%** with no model, whereas the proposed model matches total particle fluctuating kinetic energy to within **5%** of the DNS data for low to high-inertia particles; for inertial particles, the variance of uncorrelated particle velocity is matched to within **10%** of DNS results, compared to **60-70%** under-prediction with no model [2208.08156]. This application illustrates neural SDEs as data-driven stochastic closures rather than generic sequence models [2208.08156].

Financial modeling has motivated both Brownian and Lévy-driven neural stochastic systems. LDE-Net applies a Lévy-induced stochastic differential equation network to chaotic stock-index forecasting and reports that LDE-Net slightly but consistently outperforms SDE-Net and other baselines; on the SSE Energy Index, the reported MSEs are **0.0011** for SDE-Net and **0.0009** for LDE-Net, with corresponding RMSEs **0.0335** and **0.0295** [2111.13164]. A different finance-oriented system, SDBS, combines simulated SDE sample paths of multi-asset prices with the Black–Scholes PDE to learn option values and Greeks, showing that sufficiently smooth activation functions and the Black–Scholes constraint contribute significantly to accurate option pricing [2007.00937].

In neuroscience, latent neural SDEs have been used to model single-trial neural population dynamics as partial observations of an underlying stochastic continuous-time system. A coupled-oscillator SDE hybrid achieves competitive neural and behavioral prediction across three datasets while using only about **465–526 parameters** for the generative dynamics, compared with **4,944** for a neural SDE and **33,280** for an LSTM [2412.12112]. On perturbed reach, the coupled oscillator SDE reaches **0.238 ± 0.008** bits per spike and **0.86 ± 0.018** $R^2$; on visual decision making it reaches **0.22 ± 0.011** bits per spike and **0.67 ± 0.025** $R^2$ [2412.12112]. On graphs, latent Graph Neural SDEs are reported to surpass Graph Convolutional Networks and Graph Neural ODEs especially in confidence prediction and out-of-distribution detection [2308.12316]. For inverse problems without path supervision, drift and diffusion can instead be identified from stationary distributions by combining a Hellinger-distance density loss with a stationary Fokker–Planck residual [2302.08036].

## 5. Theoretical analyses and interpretive frameworks

The theory of neural SDEs has developed along several axes: robustness of stochastic depth-like dynamics, convergence of discretized learning procedures, viability on constrained domains, and constructive characterization of realizable function classes. In the continuous-depth regularization view, neural SDEs were introduced partly to import dropout, additive noise, multiplicative noise, stochastic depth, and shake-shake-like randomization into Neural ODE networks. Under Lyapunov-type conditions, the diffusion can stabilize perturbation dynamics; for the special case
$$
\mathrm d\epsilon_t=f_\Delta(\epsilon_t,t;)\,\mathrm dt+\sigma\cdot\epsilon_t\,\mathrm dW_t,
$$
if the drift is $L$-Lipschitz and $\sigma^2>2L$, then the zero solution is almost surely exponentially stable [1906.02355]. This is one of the clearest formal statements that stochasticity can improve robustness rather than merely inject noise [1906.02355].

A constructive geometric perspective treats neural SDEs as cascade systems. One formulation studies
$$
\dif W_t=a(W_t;\theta)\dif t+b(W_t;\theta)\circ\dif V_t,\qquad
\dif Z_t=h(W_t,t)Z_t\dif t,\qquad
Y=Z_1\sigma(W_1^\top x),
$$
and proves the Feynman–Kac representation
$$
f(x;\theta)=\mathbb E\!\left[\exp\!\left(\int_0^1h(W_t,t)\,\dif t\right)\sigma(W_1^\top x)\right].
$$
The same work uses finite-dimensional Lie algebra structure to characterize realizable function classes and shows that sphere-valued and orthogonal-group-valued diffusions arise when the generating matrices are skew-symmetric [2307.00215]. A plausible implication is that neural SDE expressivity can be analyzed through geometry of stochastic flows as much as through parameter count.

Convergence theory has been developed both for specialized architectures and for training algorithms. LDE-Net proves convergence in probability in the Skorokhod space $\mathbb D_{\mathbb R^d}[0,T]$ to the target Lévy-driven SDE and states that the convergence bound is independent of state dimension, thereby avoiding the curse of dimensionality [2111.13164]. Neural SVEs establish an $L^p$-stability result showing that if two Volterra equations have close coefficients and kernels, then their solution processes are close, providing the approximation rationale for kernel-parameterized path-dependent models [2407.19557]. For compact polyhedral domains, viability theory is used to derive boundary conditions under which the process remains in the domain with probability $1$, and a weighted sums parameterization is then proved to satisfy these conditions while keeping the dynamics continuous [2508.17090].

## 6. Limitations, identifiability, and common misconceptions

A recurring limitation is that exact likelihoods are generally intractable, stochastic simulation can produce noisy gradients, and adversarial SDE learning can be unstable. HGAN-SDEs is motivated precisely by the claim that prior GAN-based discriminators for SDE path distributions, especially Neural CDE discriminators, suffer from high computational costs and exacerbate instability of adversarial training [2512.20272]. Deterministic alternatives such as Bidimensional Moment Matching report that obtaining well-calibrated uncertainty estimations from neural SDEs by Monte Carlo sampling is computationally prohibitive, and that Monte Carlo matches their calibration quality only after introducing high computational cost [2006.08973].

Identifiability is also nontrivial. In stationary-distribution learning, if both drift and diffusion are completely free, the inverse problem is non-unique because the trivial choice $b(x)=0$, $\sigma(x)=0$ also minimizes the stationary Fokker–Planck residual; partial drift observations are introduced to break this degeneracy [2302.08036]. Moment-based methods face an analogous issue: SPINODE requires repeated trajectories from identical initial conditions to estimate moments accurately, and different parameter sets can match the same moments, so moment fitting may not uniquely identify the true distribution [2109.01621]. These are not failures specific to a single architecture; they are structural inverse-problem ambiguities in stochastic system identification.

Another common misconception is that all neural-SDE-related models learn by directly simulating stochastic paths. SPINODE explicitly does not: it learns hidden physics inside an SDE by converting stochastic dynamics into deterministic ODEs for moments and matching mean and covariance trajectories [2109.01621]. Neural Laplace for SDEs departs in a different direction, by learning in Laplace space after defining the Laplace transform pathwise for a stochastic process; however, the evidence presented there is restricted to Geometric Brownian Motion in a strong-drift regime, includes no comparison with neural SDE methods, and reports that Neural Laplace is not the best performer in that particular setup [2406.04964].

The simplifying assumptions used in many neural SDE papers remain significant. Reported examples include time-invariant dynamics, diagonal diffusion, Markovian dependence, Euler–Maruyama discretization, or bounded domains that require special viability constructions [2501.18871], [2508.17090]. This suggests that “neural stochastic differential equation model” is best understood not as a single settled methodology, but as a technical family of continuous-time stochastic models whose practical value depends on how drift, diffusion, observation model, state-space geometry, and training objective are matched to the scientific or engineering problem.

Source: https://www.emergentmind.com/topics/neural-stochastic-differential-equation-model