Reverse Stochastic Differential Equations

• Reverse SDE is a framework that reverses the dynamics of stochastic processes by incorporating an extra score term derived from the gradient of the log-density.
• It underpins a range of applications including diffusion models, conditional bridge sampling, and Bayesian filtering in high-dimensional systems.
• Numerical solvers such as Euler–Maruyama and high-order ODE integrators enable efficient implementation, reducing neural evaluations in tasks like speech and image restoration.

A reverse stochastic differential equation (SDE) is a stochastic differential equation describing the dynamics of a process when the time parameter is reversed, typically to connect a simple reference distribution (such as a standard Gaussian) to a complex target distribution (such as observed data), or more generally to implement generative, filtering, or assimilation operations in both model-based and data-driven stochastic modeling. The reverse SDE framework underpins score-based diffusion models, conditional bridge constructions, advanced Bayesian assimilation schemes, and the mathematical analysis of pathwise time reversal for diffusion and Volterra-driven SDEs.

1. Mathematical Formulation of Forward and Reverse SDEs

Let $X_t \in \mathbb{R}^d$ be described by a forward Itô SDE over $t \in [0,T]$: $$dX_t = f(X_t, t)\,dt + g(t)\,dW_t, \quad X_0 \sim p_0$$ where $f: \mathbb{R}^d \times [0,T] \to \mathbb{R}^d$ is the drift, $g: [0,T] \to \mathbb{R}_{>0}$ is the scalar or matrix-valued diffusion coefficient, and $W_t$ is a standard Wiener process.

The forward SDE induces a family of marginals $p_t(x)$, whose evolution is governed by the Fokker–Planck (forward Kolmogorov) partial differential equation: $$\frac{\partial}{\partial t}p_t(x) = -\nabla_x \cdot (f(x,t)\,p_t(x)) + \tfrac{1}{2} \nabla_x^2 (g^2(t)\,p_t(x)).$$

The reverse-time SDE, as classically constructed by Anderson (1982), describes the time-reversed process $\widetilde{X}_s = X_{T-s}$, itself an Itô diffusion for which the SDE reads: $$d\widetilde{X}_s = [f(\widetilde{X}_s, T-s) - g^2(T-s)\nabla_x \log p_{T-s}(\widetilde{X}_s)]\,ds + g(T-s)\,d\widetilde{W}_s$$ or, after a time re-indexing $t = T-s$: $$dX_t = [f(X_t, t) - g^2(t)\nabla_x \log p_t(X_t)]\,dt + g(t)\,d\overline{W}_t$$ with $X_T \sim p_T$. The additional drift term involving the score function $\nabla_x \log p_t(x)$ is a direct consequence of the time reversal of the Fokker-Planck equation and connects to score-based generative modeling. Sufficient regularity, non-degeneracy of $g(t)$, and smoothness of $p_t$ guarantee the existence and uniqueness of strong solutions (Wu et al., 2022).

2. Reverse SDEs in Diffusion Models and Generative Modeling

Score-based diffusion models, such as those underlying ItôWave (Wu et al., 2022), exploit the reverse SDE formalism to generate samples from a data distribution. The forward SDE progressively perturbs data toward a tractable distribution (often Gaussian), while the reverse SDE reconstructs high-dimensional structure from noise using an estimated score (the gradient of the log-density with respect to the state variable). In the Variance-Exploding (VE) SDE setting, $f \equiv 0$ and $g(t)$ is exponentially increasing, yielding explicit transition laws: $$X_t \mid X_0 = x_0 \sim \mathcal{N}(x_0, (\sigma_0^2((\sigma_1/\sigma_0)^{2t} - 1))I)$$ The learned score, typically via a deep neural network optimized with denoising-score matching, provides the critical correction in the reverse dynamics: $$dX_t = -g^2(t)\,\mathfrak{S}_\theta(X_t, t, m)\,dt + g(t)\,d\overline{W}_t$$ where $m$ denotes conditioning inputs (e.g., conditional spectrograms in ItôWave). Discretization employs schemes such as the Euler–Maruyama step, often augmented with Langevin correction to sharpen samples.

3. Reverse SDEs for Interpolating and Conditional Diffusion Processes

In speech restoration and conditional generation tasks, interpolating SDEs (iSDEs) (Lay et al., 10 Mar 2026) generalize the standard diffusion SDE by interpolating between a signal and an observation: $dx_t = \gamma(t)\,(y - x_t)\,dt + g(t)\,dw_t$ Reverse-time dynamics for such processes yield a family of SDEs parametrized by $\kappa$: $$dx_t = \left[\gamma(t)(y-x_t) - \frac{1+\kappa^2}{2}g^2(t)\,\nabla_{x_t} \log p_t(x_t\mid y)\right]\,dt + \kappa\,g(t)\,d\bar{w}_t$$ where $\kappa=1$ produces the classical reverse SDE, and $\kappa=0$ yields the probability-flow ODE. The reverse score $\nabla_{x_t}\log p_t(x_t\mid y)$ is replaced by a neural estimator $\hat{s}_\theta(x_t, y, t)$.

These reverse SDEs are contrasted with the unconditional case ($y$ absent), where the forward process ends in a standard Gaussian and the reverse dynamics drift toward the data manifold. In conditional models such as SGMSE+, the interaction between data and observation in the drift structure is essential for stochastic signal interpolation and speech restoration tasks (Lay et al., 10 Mar 2026).

4. Reverse SDEs in Stochastic Filtering and Bayesian Inference

Reverse SDEs are instrumental in recursive Bayesian inference for stochastic (partial) differential equations. In high-dimensional data assimilation for SPDEs, score-based filtering integrates observation likelihoods directly into the reverse SDE drift (Huynh et al., 9 Aug 2025): $$dZ_\tau = [b_\tau\,Z_\tau - \sigma^2_\tau\,S(Z_\tau, \tau)]\,d\tau + \sigma_\tau\,d\bar{W}_\tau$$ Here, the score $S(z, \tau) = \nabla_z \log Q_\tau(z)$ is estimated from either a data ensemble (the Ensemble Score Filter) or a trained network, and a likelihood score term $S_{LH}(z, \tau)$ is added to incorporate observational corrections: $$S_{n+1|n+1}(z,\tau) = S_{n+1|n}(z,\tau) + h(\tau) S_{LH}(z,\tau)$$ The reverse SDE then delivers samples from the Bayesian posterior. This methodology is particularly effective for online data assimilation under model error, with the EnSF yielding real-time, training-free score estimates with satisfactory accuracy for high-dimensional SPDEs.

5. Time-Reversal for SDEs Driven by Volterra Processes

The reverse SDE formalism extends to SDEs driven by Volterra processes, including fractional Brownian motion (Decreusefond, 2010). For a Volterra-driven SDE in Stratonovich form: $X_t = x + \int_0^t \sigma(X_s) \circ dW^V(s)$ where $W^V$ is a (Gaussian) Volterra process, time reversal induces a transformation to an Itô SDE with past-dependent coefficients: $$Z_{r,t}(x) = x - \int_r^t V_T(\sigma \circ Z_{\cdot,t})_{[r,t]}(s) dB^T(s)$$ The operator $V_T = \Theta_T V \Theta_T$ encodes the time-reversed Volterra kernel. Existence and uniqueness of strong solutions for both the original and reversed (past-dependent) SDE are demonstrated under minimal regularity and causality conditions. This approach clarifies the dichotomy in solution theory between different Hurst exponents and links the stochastic calculus of Volterra processes to the classical theory of Itô equations with memory.

6. Numerical Solvers and Efficient Implementation

Reverse SDEs are discretized using either stochastic integrators (Euler–Maruyama) or, for increased efficiency and reduced neural function evaluations (NFEs), high-order ODE solvers such as exponential integrators (iSDE-solver) (Lay et al., 10 Mar 2026) or Heun's method (Wang et al., 2024). Algorithmic accelerations rely on the exact integration of linear drift and Taylor expansion of score terms, allowing for a dramatic reduction in the number of required score network evaluations (e.g., 10 steps vs. 60–100 with Euler–Maruyama), provided that all integrals (e.g., for the integrating factor $\Phi$ and weights $w_n$) can be precomputed.

Specialized posterior sampling, such as the “stochastic start” technique (Wang et al., 2024), is used to avoid singularities in reverse ODEs at the end-point of bridge models. This involves an initial backward step via the closed-form posterior and subsequent integration by a high-order solver. These developments have led to practical samplers achieving state-of-the-art fidelity and efficiency in speech and image restoration tasks.

7. Applications and Theoretical Perspectives

Reverse SDEs provide a mathematical and algorithmic foundation for score-based diffusion modeling, conditional bridge sampling, real-time Bayesian assimilation, and the probabilistic analysis of time-reversed stochastic processes. Their use in generative audio (e.g., ItôWave), speech restoration (SGMSE+, iSDEs), conditional image synthesis (diffusion bridges), and high-dimensional filtering for SPDEs has demonstrated broad impact.

From a theoretical standpoint, time-reversal reveals deep structural relationships between forward- and reverse-time diffusive evolution, highlights the fundamental role of the score function, and enables extensions to non-Markovian or non-Itô environments (e.g., Volterra kernels). The dialogue between numerical innovation and stochastic analysis remains active, with future work expected in non-Gaussian settings, learning for infinite-dimensional systems, and memory-driven processes.

---

References:

• (Wu et al., 2022) ItôWave: Itô Stochastic Differential Equation Is All You Need For Wave Generation
• (Lay et al., 10 Mar 2026) A Fast Solver for Interpolating Stochastic Differential Equation Diffusion Models for Speech Restoration
• (Wang et al., 2024) An Ordinary Differential Equation Sampler with Stochastic Start for Diffusion Bridge Models
• (Huynh et al., 9 Aug 2025) A Score-based Diffusion Model Approach for Adaptive Learning of Stochastic Partial Differential Equation Solutions
• (Decreusefond, 2010) Time reversal of Volterra processes driven stochastic differential equation