---
title: First-Order VE ER-SDE-Solver
url: https://www.emergentmind.com/topics/first-order-ve-er-sde-solver
type: topic
---

# First-Order VE ER-SDE-Solver

A first-order VE ER-SDE-Solver (often denoted as "VE ER-SDE-Solver-1") is a stochastic differential equation (SDE) solver for diffusion models, derived within the framework of Extended Reverse-Time SDEs. This method generalizes and interpolates between ordinary differential equation (ODE) and SDE-based diffusion samplers by introducing a tunable reverse-process noise scale. Specifically, it targets the variance-exploding (VE) case, delivering a computationally lightweight, training-free algorithm that attains state-of-the-art performance among first-order stochastic solvers in diffusion-based generative modeling [2309.06169].

## 1. Mathematical Formulation

The Variance-Exploding Extended Reverse-Time SDE (VE ER SDE) is defined by setting $f(t)=0$ and $g(t)=\sqrt{d\,\sigma^2(t)/dt}$, where $\sigma(t)$ describes the forward-process noise schedule. An additional reverse-process noise scale $h(t)$, with $h^2(t)=\xi(t)\,(d\,\sigma^2(t)/dt)/(2\sigma(t))$, is introduced. The ER SDE is then:
\[
d x_t = \left[ \frac{d\sigma^2(t)/dt + h^2(t)}{2\sigma^2(t)} (x_t - x_\theta(x_t, t)) \right]dt + h(t)\, d\overline{w}_t,
\]
where $\overline{w}_t$ is a reverse-time Wiener process, $x_\theta(\cdot, \cdot)$ is the pretrained data model, and $\xi(\sigma), \varphi(\sigma)$ control the reverse-process scale. After time-reparameterization ($\sigma \leftrightarrow t$), the SDE becomes
\[
d x_\sigma = \left[ \left( \frac{1}{\sigma} + \frac{\xi(\sigma)}{2\sigma^2} \right)(x_\sigma - x_\theta(x_\sigma, \sigma)) \right]d\sigma + \sqrt{\xi(\sigma)}\, d\overline{w}_\sigma.
\]
The function $\varphi(\sigma)$ is strictly increasing and satisfies $\partial_\sigma \ln\varphi(\sigma) = 1/\sigma + \xi(\sigma)/(2\sigma^2)$. This parameterization allows interpolating between deterministic and stochastic sampling regimes [2309.06169].

## 2. First-Order Euler-Style Update Rule

An explicit first-order ("Euler-style") scheme is derived via a one-step Taylor expansion. The formal exact solution (Proposition 3.2) is:
\[
x_{t_i} = \frac{\varphi(\sigma_{t_i})}{\varphi(\sigma_{t_{i-1}})}\, x_{t_{i-1}} + \varphi(\sigma_{t_i}) \int_{\sigma_{t_i}}^{\sigma_{t_{i-1}}} \frac{\varphi'(\sigma)}{\varphi^2(\sigma)} x_\theta(x_\sigma, \sigma)\, d\sigma + \sqrt{\sigma_{t_i}^2 - \sigma_{t_{i-1}}^2 \big[\varphi(\sigma_{t_i})/\varphi(\sigma_{t_{i-1}})\big]^2} \; z,
\]
where $z\sim \mathcal{N}(0, I)$. To make this practical, $x_\theta(x_\sigma, \sigma)$ in the integral is replaced with $x_\theta(x_{t_{i-1}}, t_{i-1})$ (left-endpoint evaluation). The resulting update is:
\[
\tilde{x}_{t_i} =
\underbrace{r_i \tilde{x}_{t_{i-1}}}_{\text{linear term}} +
\underbrace{(1 - r_i) x_\theta(\tilde{x}_{t_{i-1}}, t_{i-1})}_{\text{data-model term}} +
\underbrace{\sqrt{ \sigma_{t_i}^2 - r_i^2 \sigma_{t_{i-1}}^2 }\, z_i}_{\text{noise injection}}, \quad z_i \sim \mathcal{N}(0, I)
\]
where $r_i := \varphi(\sigma_{t_i})/\varphi(\sigma_{t_{i-1}})$.

## 3. Algorithmic Execution

The VE ER-SDE-Solver-1 operates in discrete time over a user-supplied time grid $T = t_0 > t_1 > \ldots > t_M = 0$ with precomputed noise schedules and reverse-process scaling arrays. The requisite steps are summarized in the following table:

| Step                         | Operation                                             | Notes                              |
|------------------------------|------------------------------------------------------|-------------------------------------|
| Initialization               | Set $x_{t_0} \leftarrow x_T$                         | $x_T$: standard normal noise        |
| For each $i = 1..M$          | $r_i \leftarrow \varphi(\sigma_{t_i})/\varphi(\sigma_{t_{i-1}})$ |                                   |
|                              | $z_i \sim \mathcal{N}(0, I)$                         | Gaussian noise                     |
|                              | $m \leftarrow x_\theta(x_{t_{i-1}}, t_{i-1})$        | One forward network eval            |
|                              | $x_{t_i} \leftarrow r_i x_{t_{i-1}} + (1-r_i)m + \sqrt{\sigma_{t_i}^2 - r_i^2 \sigma_{t_{i-1}}^2} z_i$ | Euler-style update     |
| Return                       | $x_{t_M}$                                            | Output sample                      |

This scheme induces a one-step truncation error of $O((\Delta\sigma)^2)$, and global order of accuracy $O(\Delta\sigma)$, classifying it as a first-order method [2309.06169].

## 4. Computational Considerations

Each step in VE ER-SDE-Solver-1 requires a single evaluation of the pretrained data model $x_\theta(\cdot,\cdot)$, generation of one Gaussian noise sample, and $O(D)$ vector-scalar arithmetic (with $D$ the data dimension). Its computational footprint matches that of ancestral SDE sampling and single-evaluation ODE solvers. It is more lightweight compared to high-order solvers, which involve derivative evaluations or multi-stage integration.

Runtime is similar to the EDM-Stochastic or DDIM ($\eta = 1$) samplers, and marginally slower than deterministic ODE-only solvers due to the need for a Gaussian draw. The method is suitable for batch or parallel settings, leveraging modern accelerators effectively [2309.06169].

## 5. Empirical Performance Metrics

Empirical evaluation on the Guided-diffusion ImageNet $128\times128$ model (linear noise schedule) establishes the competitiveness of VE ER-SDE-Solver-1. As measured by Fréchet Inception Distance (FID) at $NFE$ (network function evaluations):

- $NFE=10$: FID $\approx 15$
- $NFE=20$: FID $\approx 12$
- $NFE=30$: FID $\approx 10$
- $NFE=50$: FID $\approx 9$

At $NFE=20$:
- DDIM ($\eta=1$): FID $\approx 21.23$
- SDE-DPM-Solver++(2M): FID $\approx 9.73$
- VE ER-SDE-Solver-1: FID $\approx 11$--$12$

Compared to higher-order VE ER-SDE-Solver-3, which achieves FID $\approx 8.33$ at $NFE=20$, VE ER-SDE-Solver-1 situates performance midway between DDIM and advanced stochastic solvers. This suggests that the first-order solver offers substantial improvements over previous methods while retaining full stochasticity [2309.06169].

## 6. Theoretical and Practical Insights

A tunable scaling function $\varphi(\sigma)$ allows VE ER-SDE-Solver-1 to interpolate continuously between deterministic ODE sampling ($\varphi(\sigma) = \sigma$) and high-variance SDE sampling ($\varphi(\sigma) = \sigma^2$). By choosing an optimal $\varphi(\sigma)$, the discretization error term $1 - \varphi(\sigma_t)/\varphi(\sigma_s)$ can be minimized, enhancing both quality and speed. The injected noise term corrects for bias accumulated early in simulation, yielding improved sample diversity as a "Langevin-style" correction.

Advantages include:
- Substantial narrowing of the quality gap to ODE-based methods, even with single network function evaluations per step.
- Maintenance of the benefits of stochastic sampling (greater diversity).
- Outperforming all previously published first-order SDE samplers.

A plausible implication is that VE ER-SDE-Solver-1 constitutes a practical, "turn-key," training-free sampler, suitable for diverse application settings requiring a balance of sampling speed and generative quality [2309.06169].

## 7. Applications and Significance

The first-order VE ER-SDE-Solver plays a central role in the toolkit of modern diffusion-based generative frameworks, enabling efficient sampling at a level of quality previously associated with higher-order methods or increased function evaluations. Its lightweight computational profile, coupled with tunable stochasticity, makes it well-suited for scenarios demanding high-throughput sampling or ensemble generation. The methodology unifies ODE and SDE approaches, allowing flexible traversal of the speed-diversity-quality tradeoff landscape in generative modeling [2309.06169].

Source: https://www.emergentmind.com/topics/first-order-ve-er-sde-solver