---
title: Score SDE Formulation
url: https://www.emergentmind.com/topics/score-sde-formulation
type: topic
---

# Score SDE Formulation

Score SDE Formulation

The Score SDE (Stochastic Differential Equation) formulation provides a rigorous mathematical framework for diffusion-based generative modeling, centered on the evolution and utilization of the time-dependent score function $\nabla \log p_t(x)$, i.e., the gradient of the log-density of an evolving distribution under an SDE. This article details the mathematical structure, estimation approaches, and algorithmic integration of score-based SDEs, including results on analytic computation via Malliavin calculus, connections with the Fokker–Planck equation, and practical implications for generative modeling.

## 1. Forward SDEs and the Marginal Score

Consider an $m$-dimensional Itô SDE:
\[
dX_t = b(t, X_t)\,dt + \sigma(t, X_t)\,dB_t, \qquad X_0 \sim p_0,
\]
where $B_t$ is standard Brownian motion in $\mathbb{R}^d$, $b$ is the drift, and $\sigma$ the (possibly matrix-valued, state-dependent) diffusion coefficient. Under regularity and nondegeneracy, the law of $X_t$ is absolutely continuous with density $p_t(x)$:
\[
p_t(x) = \frac{d}{dx}\,\mathbb{P}(X_t \le x).
\]
The time-$t$ score is defined as
\[
\text{Score}_t(x) := \nabla_x \log p_t(x),
\]
i.e., the spatial gradient of the log-density. This score describes the local direction of log-likelihood increase and is central to reverse-time generative modeling [2503.16917].

## 2. Analytical Score Representation via Malliavin–Bismut Calculus

Classical integration-by-parts in Malliavin calculus yields an exact analytical expression for the score. Let $F = X_T$. The Malliavin derivative $D_sF$ encodes the infinitesimal sensitivity of the terminal random variable to perturbations in $B_s$. Define the Malliavin covariance matrix:
\[
\gamma_F = \int_0^T D_s F\, D_s F^\top\,ds \in \mathbb{R}^{m\times m}.
\]
The covering process for the $k$th coordinate is
\[
u_k(s) = \sum_{j=1}^m [\gamma_F^{-1}]_{k,j} D_s F^j,
\]
satisfying $\int_0^T D_s F^i \cdot u_k(s)\,ds = \delta_{ik}$. The Bismut–Malliavin formula gives:
\[
\partial_{x_k} \log p_T(x) = - \mathbb{E}[\delta(u_k)\mid X_T = x],
\]
where $\delta(u_k)$ denotes the Skorokhod integral of $u_k$. This is a fully constructive method for score computation using only forward SDE paths and their Malliavin derivatives [2503.16917, 2507.05550].

## 3. Closed-Form Scores: Linear and Nonlinear SDEs

For linear SDEs:
\[
dX_t = b(t) X_t\,dt + \sigma(t)\,dB_t,\quad X_0\sim p_0,
\]
let $Y_t$ be the solution to $dY_t = b(t) Y_t\,dt$, $Y_0 = I$, and
\[
D_s X_T = Y_T Y_s^{-1} \sigma(s), \quad \gamma_{X_T} = Y_T\bigl(\int_0^T Y_r^{-1} \sigma(r) \sigma(r)^\top (Y_r^{-1})^\top dr\bigr) Y_T^\top.
\]
The score admits the explicit formula:
\[
\nabla_x \log p_T(x) = -\gamma_{X_T}^{-1} \bigl(x - Y_T\,\mathbb{E}[X_0 | X_T = x]\bigr),
\]
which coincides with the Fokker–Planck Gaussian solution [2503.16917]. For nonlinear drift but state-independent diffusion $\sigma(t)$, the score is given by the conditional expectation of the Skorokhod integral determined by first and second variation processes (see below for explicit computational schemes):
\[
\partial_{x_k} \log p_T(x) = -\mathbb{E}[\delta(u_k) \mid X_T = x],
\]
with $\delta(u_k)$ comprising Itô integrals over random variation processes parameterized by the endpoint $x$ [2503.16917, 2507.05550].

## 4. First and Second Variation Processes and Their Role

The analytic score formula for nonlinear drift relies on the explicit SDEs for the first ($Y_t = \partial_x X_t$) and second ($Z_t = \partial_x^2 X_t$) variation processes:
\[
\begin{aligned}
& dY_t = \partial_x b(t, X_t) Y_t\,dt + \sum_{l=1}^d \partial_x \sigma^l(t, X_t) Y_t\,dB_t^l,\quad Y_0 = I_m, \\
& dZ_t = [\partial_{xx}b(t,X_t)(Y_t \otimes Y_t) + \partial_x b(t,X_t) Z_t]dt
    + \sum_{l=1}^d [\partial_{xx}\sigma^l(t,X_t)(Y_t \otimes Y_t) + \partial_x \sigma^l(t,X_t) Z_t]\,dB_t^l.
\end{aligned}
\]
The cover process $u_k(t)$ and its Skorokhod divergence $\delta(u_k)$ are then expressible entirely in terms of $Y_t,\,Z_t$, and deterministic time-integrals, with all Malliavin derivatives resolved via explicit variation process expansions [2507.05550].

## 5. Integration of Analytical Scores into Training and Sampling

Training (score matching) proceeds by forward simulation of $(X_t, Y_t, \gamma_{X_t})$ under the SDE, with direct computation of the analytic score for each $X_t$. The neural network $s_\theta(t, x)$ is trained by minimizing the mean-square error to the analytic or Malliavin–Bismut target:
\[
\mathcal{L}(\theta) = \mathbb{E}_{t, x} \bigl\| s_\theta(t, x) + \gamma_{X_t}^{-1}(x - Y_t\,\mathbb{E}[X_0|X_t = x]) \bigr\|^2.
\]
In practice, $\mathbb{E}[X_0|X_t = x]$ is implemented via regression or a second auxiliary network in the nonlinear case. For sampling, the reverse-time SDE is numerically integrated:
\[
dx = [b(t, x) - \sigma^2(t)\,s_\theta(t, x)]\,dt + \sigma(t)\,d\bar{B}_t,
\]
using the trained $s_\theta$ as the score estimator [2503.16917].

In cases where analytical/computable closed forms are available (e.g., via Malliavin–Bismut representations), direct plugin of the computed score enables Monte Carlo evaluation of the conditional Skorokhod integral. This approach bypasses neural score parameterization and can be inserted directly into samplers such as Langevin-type or predictor–corrector procedures [2507.05550].

## 6. Theoretical Guarantees, Consistency, and Extensions

The Malliavin–Bismut analytic scheme rigorously coincides with the solution derived via the Fokker–Planck PDE in the linear–Gaussian case [2503.16917]. For general nonlinear drift and state-independent diffusion, the closed analytic score expressed through random variation processes remains free from unknown densities, enabling broader generalization to complex SDEs.

Key theoretical conditions include invertibility of the Malliavin covariance and sufficient smoothness (drift/diffusion at least $C^2$), ensuring existence and computability of the requisite variation processes. In high dimensions, computational cost scales with the need to simulate $Y_t, Z_t$ and their integrals along SDE sample paths; Monte Carlo averaging is a practical solution, and the resulting scores can be used without recourse to denoising or sliced score matching methodologies [2507.05550].

Extensions to more general SDE forms, including state-dependent diffusion (using generalized Skorokhod and Malliavin differential machinery), are developing, paving the way for systematic score computation in increasingly broad generative modeling contexts [2503.16917].

## 7. Implications for Diffusion Generative Modeling

Malliavin–Bismut-based score SDEs enable the direct, tractable computation of $\nabla \log p_t(x)$ for a wide class of SDE-driven generative models—including both linear (Gaussian) and nonlinear regimes. These analytical formulas tightly connect the forward SDE, the time-evolving density's score, and the reverse-time sampling process crucial to diffusion modeling. This analytical viewpoint refines and generalizes the backbone of training and sampling pipelines in score-based generative models, providing higher accuracy and deeper theoretical justification for the use of SDEs in generative modeling workflows [2503.16917, 2507.05550].

Source: https://www.emergentmind.com/topics/score-sde-formulation