---
title: 'Variance-Exploding SDEs: Theory & Applications'
url: https://www.emergentmind.com/topics/variance-exploding-stochastic-differential-equations-ve-sdes
type: topic
---

# Variance-Exploding SDEs: Theory & Applications

Variance-Exploding Stochastic Differential Equations (VE-SDEs) define a class of stochastic processes characterized by increasing variance over time and form a foundational component in generative diffusion models. In VE-SDEs, the forward stochastic differential equation is drift-free with a time-dependent diffusion coefficient, and the backward or reverse-time equation involves a score-based drift. Under the Extended Reverse-Time SDE (ER-SDE) framework, VE-SDEs admit semi-linear solutions with a parameterized noise-schedule, allowing interpolation between fully stochastic and deterministic (ODE) solvers. Approximate closed-form solutions, efficient solvers, and error analyses within this framework provide mathematical and practical insights into the speed, quality, and diversity of diffusion-based sampling algorithms [2309.06169].

## 1. Formal Definition and Structure of VE-SDEs

The Variance-Exploding SDE is defined by a forward equation of the form:
\[
\mathrm{d}x_t = 0\cdot x_t\,\mathrm{d}t + \sqrt{\frac{\mathrm{d}\,\sigma_t^2}{\mathrm{d}t}}\,\mathrm{d}w_t, \qquad x_0 \sim p_0(x_0)
\]
where the drift $f(t, x) = 0$, and the diffusion coefficient is $g(t) = \sqrt{\tfrac{\mathrm{d}\,\sigma_t^2}{\mathrm{d}t}}$. The process is initialized from a data distribution $p_0(x_0)$, and $w_t$ is the standard Wiener process.

For generative modeling tasks, the reverse-time SDE (specialized from Song et al. 2021 to the VE case) is given by:
\[
\mathrm{d}x_t = \left[-g(t)^2 \nabla_x \log p_t(x_t)\right] \mathrm{d}t + g(t) \,\mathrm{d}\bar{w}_t
\]
with reverse drift $f_{\text{rev}}(t, x) = -\frac{\mathrm{d}\,\sigma_t^2}{\mathrm{d}t} \nabla_x \log p_t(x)$ and $\bar{w}_t$ an independent Wiener process.

## 2. Solution Structure in the Extended Reverse-Time SDE Framework

The ER-SDE framework generalizes both SDE and ODE-based solvers by introducing an independent “reverse” noise scale $h(t)$ and substituting the learned data-prediction network $x_\theta(x_t, t)$ for the score term.

Switching to the noise-level parameter $\sigma = \sigma_t$, the ER-SDE reads:
\[
\mathrm{d}x_\sigma = \left[\frac{1}{\sigma} + \frac{\xi(\sigma)}{2\sigma^2}\right](x_\sigma - x_\theta(x_\sigma, \sigma))\,\mathrm{d}\sigma + \sqrt{\xi(\sigma)}\,\mathrm{d}w_\sigma
\]
with $\xi(\sigma) \ge 0$, such that $h^2(t) = \xi(\sigma_t)\,\frac{\mathrm{d}\sigma_t}{\mathrm{d}t}$. Define
\[
\phi(\sigma) = \exp\left(\int\left[\frac{1}{\sigma} + \frac{\xi(\sigma)}{2\sigma^2}\right] \,\mathrm{d}\sigma\right)
\]
with $\phi'(\sigma)/\phi(\sigma) = \frac{1}{\sigma} + \frac{\xi(\sigma)}{2\sigma^2}$.

The exact solution for evolving $x$ from $\sigma_s$ to $\sigma_t$ (Proposition 1) is:
\[
x_t = \frac{\phi(\sigma_t)}{\phi(\sigma_s)}\,x_s + \phi(\sigma_t)\!\int_{\sigma_t}^{\sigma_s}\! \frac{\phi'(\sigma)}{\phi(\sigma)^2}\,x_\theta(x_\sigma, \sigma) \,\mathrm{d}\sigma + \sqrt{\sigma_t^2 - \sigma_s^2\left[\frac{\phi(\sigma_t)}{\phi(\sigma_s)}\right]^2}z
\]
where $z \sim \mathcal{N}(0, I)$. In practice, the nonlinear integral is approximated using Taylor expansion.

## 3. Efficient VE ER-SDE Solvers: Algorithmic Construction

The first-order VE ER-SDE-Solver has the following update for each step:
\[
\tilde x_t = r\,\tilde x_s + (1 - r)x_\theta(\tilde x_s, \sigma_s) + \sqrt{\sigma_t^2 - r^2 \sigma_s^2}z
\]
where $r = \phi(\sigma_t)/\phi(\sigma_s)$, and $z \sim \mathcal{N}(0, I)$. Each update utilizes a single evaluation of $x_\theta$.

Pseudocode for the algorithmic workflow (as described in Algorithm 1) is:

| Step | Description | Details |
|------|-------------|---------|
| 1    | Initialization | $x_0 \gets x_T$ (initial noise sample) |
| 2    | For $i = 1, ..., M$ | $r \gets \phi(\sigma_i)/\phi(\sigma_{i-1})$ |
|      |  | $z \sim \mathcal{N}(0, I)$ |
|      |  | $\hat{x} \gets x_{i-1}$ |
|      |  | drift $\gets x_\theta(\hat{x}, \sigma_{i-1})$ |
|      |  | noise $\gets \sqrt{\sigma_i^2 - r^2 \sigma_{i-1}^2} \cdot z$ |
|      |  | $x_i \gets r\cdot\hat{x} + (1 - r)\cdot$ drift $+ $ noise |
| 3    | Output | $x_M$ |

The method is tunable via the choice of $\phi(\sigma)$, the number of steps $M$, and the schedule $t_i \mapsto \sigma_i$. Higher-order versions reuse evaluations and add finite-difference corrections at the cost of additional network calls per step.

## 4. Local Discretization Error and Theoretical Analysis

Discretization error is governed by the First-Order Euler Integral (FEI) coefficient:
\[
\mathrm{FEI} = 1 - \frac{\phi(\sigma_t)}{\phi(\sigma_s)}
\]
which quantifies the dominant local one-step error:
\[
\tilde x_t - x_t = \mathcal{O}((\sigma_t - \sigma_s)^2)
\]
The minimum FEI is achieved by choosing $\phi(\sigma) = \sigma$, corresponding to the deterministic probability-flow ODE, resulting in the lowest discretization error among the ER-SDE family. Any larger $\phi(\cdot)$ increases FEI and the corresponding global error for equal step-size.

A change of variable demonstrates that VE and VP ER-SDEs share the same FEI coefficient, establishing parity between these formulations for a given pretrained model and fixed number of function evaluations (NFE).

## 5. Stochasticity, Sample Quality, and Diversity

ODE-based samplers with $\phi(\sigma) = \sigma$ have minimal local error but no injected noise, leading to less sample diversity. Choosing $\phi(\cdot)$ close to $\sigma$ but sufficiently large to inject controlled noise allows ER-SDE-based VE solvers to achieve near-ODE fidelity without sacrificing diversity. This stochasticity-efficiency tradeoff is central: VE-SDE solvers interpolate between pure SDE and ODE processes, balancing rapid low-NFE sampling with high sample quality and output variability.

## 6. Practical Considerations and Empirical Findings

VE ER-SDE-Solvers are parameterized by the noise-scale function $\phi(\sigma)$, number of steps $M$, and schedule mapping $t_i \mapsto \sigma_i$. First-order solvers require just one network evaluation per step. Advanced higher-order variants increase per-step cost for potentially improved empirical accuracy.

Empirical evaluation on the ImageNet $128 \times 128$ benchmark demonstrates that ER-SDE-Solvers attain state-of-the-art performance across stochastic samplers while maintaining the efficiency typical of deterministic samplers (e.g., $8.33$ FID in $20$ function evaluations) [2309.06169]. This suggests that appropriate tuning of $\phi(\cdot)$ enables simultaneous optimization of sample quality and computational efficiency.

## 7. Significance and Theoretical Summary

The ER-SDE framework unifies ODE and SDE sampling methodologies for VE-SDEs, providing a family of semi-linear solutions whose error and stochasticity are parametrically controlled by $\phi(\sigma)$. The key theorem asserts that, among all extended reverse-time SDEs with a given drift-score model, the ODE ($h(t) = 0, \phi(x) = x$) uniquely minimizes the local discretization error. A plausible implication is that careful functional choice allows constructing samplers that closely approach ODE performance while preserving the stochastic effects essential for output variability and model robustness [2309.06169].

Source: https://www.emergentmind.com/topics/variance-exploding-stochastic-differential-equations-ve-sdes