Score-Based Diffusion Models (SGMs)

• Score-based diffusion models (SGMs) are generative models that reverse a stochastic diffusion process via score function gradients to transform simple reference measures into complex data distributions.
• They leverage advanced tools like Malliavin and Γ-calculus to rigorously extend analysis from finite-dimensional spaces to infinite-dimensional Hilbert spaces and manifold-valued data.
• Dimension-independent entropic convergence bounds and an L2 regression formulation ensure robust denoising and optimal filtering in modeling complex random fields.

Score-based diffusion models (SGMs) are a class of generative models that transform a simple reference distribution into a complex data distribution by reversing a stochastic diffusion process, guided by the score function—the gradient of the log-density of the evolving distribution. Recent advances have placed the theory on increasingly rigorous mathematical ground, extending the analysis and modeling capabilities from finite-dimensional data to random fields and infinite-dimensional Hilbert spaces through the application of Malliavin and Γ (“Gamma”) calculus. This generalization enables SGMs to model complex functional data, including random fields on manifolds such as the sphere, and provides robust theoretical guarantees for both the forward and reverse processes.

1. Infinite-Dimensional Hilbert Space Formalism for SGMs

SGMs can be formulated directly on an infinite-dimensional separable Hilbert space $\mathcal{H}$ (e.g., $L^2(S^2)$ for random fields on the sphere) by defining both the forward diffusion (noising) and the reverse (denoising) processes using tools from functional analysis.

A Gaussian reference measure $m$ with trace-class covariance operator $C$ on $\mathcal{H}$ is specified, with spectral decomposition $C Y_\ell = C_\ell Y_\ell$ for an orthonormal basis $\{Y_\ell\}$. The associated Cameron–Martin space $\mathcal{H}_m$ is

$$\mathcal{H}_m = \left\{ h \in \mathcal{H} : \sum_\ell C_\ell^{-1} |\langle h, Y_\ell \rangle |^2 < +\infty \right\},$$

with inner product $$\langle h, k\rangle_{\mathcal{H}_m} = \sum_\ell \frac{\langle h, Y_\ell\rangle \langle k, Y_\ell\rangle}{C_\ell}$$.

Cylindrical functions $p(\phi) = F(I(f_1)(\phi), ..., I(f_k)(\phi))$, where $f_i \in \mathcal{H}_m$ and $F$ is smooth with polynomial growth, serve as test functions for the development of the infinite-dimensional calculus. Here, $I$ maps the Cameron–Martin element to a Paley–Wiener integral.

The gradient (Malliavin derivative) of $p$ in the direction of the Cameron–Martin space is

$$\bar{\nabla} p = \sum_{i=1}^k (\partial_i F)(I(f_1), ..., I(f_k)) f_i \in \mathcal{H}_m.$$

This is used to define the carré du champ operator (Γ operator) as $$\Gamma(p, q) = \langle \bar{\nabla} p, \bar{\nabla} q \rangle_{\mathcal{H}_m}$$. The associated Dirichlet form is

$$\mathcal{E}(p, q) = \frac{1}{2} \int_{\mathcal{H}} \Gamma(p, q) \, dm.$$

The forward diffusion process is then given by the infinite-dimensional Ornstein–Uhlenbeck dynamics,

$$X_t = X_0 - \frac{1}{2} \int_0^t X_s \, ds + W_t^{(m)},$$

where $W_t^{(m)}$ is an $\mathcal{H}$-valued Brownian motion with covariance given by $\mathcal{H}_m$.

2. Malliavin Derivative as the Score Function: Conditional Expectation Representation

In this functional framework, the reverse SGM process is driven by a score function defined as a suitable Malliavin (gradient) operator. Specifically, for the time-marginal density $\rho_t = d\mu_t/dm$, the critical quantity is

$\frac{\Gamma(\rho_t, u)}{\rho_t}$

where $u$ is a test function.

A principal result (see Theorem 3.1) identifies this quantity as a conditional expectation: $$\left( \frac{\Gamma(\rho_t, u)}{\rho_t} \right)(\Phi) = \frac{1}{e^{t/2} - e^{-t/2}} \mathbb{E}\left[ I(\bar{\nabla} u (X_t)) \cdot (X_0 - e^{-t/2} X_t) \mid X_t = \Phi \right].$$ This formalizes the Malliavin derivative as the abstract infinite-dimensional analog of the finite-dimensional score function $\nabla_x \log p_t(x)$. The identification with a conditional expectation $E[X_0 | X_t = \Phi]$ positions learning the score in SGMs as a form of optimal $L^2$ regression in Hilbert space.

This conditional expectation property is essential for training, as it implies that the best $L^2$ estimator of the initial condition given the observed influx of noise is precisely the conditional mean—an underpinning of optimal filtering and denoising.

3. Entropic Convergence in Infinite Dimensions and Fisher Information

The established framework allows for rigorous extension of finite-dimensional entropic convergence results (e.g., Kullback–Leibler divergence bounds) to infinite-dimensional settings. If $J$ denotes a projection onto a finite set of coordinates (subsystem), the convergence bound holds uniformly in $J$: $$\text{KL}(\mu^J \,||\, \mathcal{L}(Y_{\text{gen}}^J)) \leq e^{-T/2} \text{KL}(\hat{m}||m) + T\epsilon^2 + 2h\max\{4,h\} I(m).$$ Here $I(m)$ is the Fisher information relative to $m$, with the gradient taken in the Cameron–Martin norm, and $\epsilon$ quantifies the $L^2$ score approximation error. These bounds are dimension-free in the sense that they persist under arbitrary finite-dimensional projections, highlighting the centrality of the Cameron–Martin norm for regularity properties and control of the KL divergence. Γ-calculus and functional inequalities underlie the proof, leveraging the structure of infinite-dimensional Dirichlet forms.

4. Specialization to Random Fields on the Sphere and the Whittle–Matérn Construction

Taking $\mathcal{H} = L^2(S^2)$, the machinery specializes to random fields over the sphere. An isotropic random field $T(x)$ admits the Karhunen–Loève expansion

$$T(x) = \sum_{\ell=0}^{\infty} \sum_{m=-\ell}^{\ell} a_{\ell,m} Y_{\ell,m}(x)$$

with $Y_{\ell,m}$ the spherical harmonics and $a_{\ell,m}$ uncorrelated random coefficients.

For Whittle–Matérn fields (reference noise model), one uses the stochastic PDE

$(\kappa^2 - \Delta_S)^{\beta} u = W,$

where $W$ is spherical white noise, $\kappa > 0$, $\beta > 1/2$. The eigenfunctions are the spherical harmonics, with eigenvalues $$C_{\ell}^{(\kappa,\beta)} = (\kappa^2 + \ell(\ell+1))^{-2\beta}$$.

The Cameron–Martin space becomes

$$\mathcal{H}_{m^{(\kappa,\beta)}} = \left\{ u = \sum_{\ell,m} a_{\ell,m} Y_{\ell,m} \;\Bigg|\; \sum_{\ell,m} a_{\ell,m}^2 (\kappa^2 + \ell(\ell+1))^{2\beta} < +\infty \right\}.$$

Each harmonic coefficient $a_\ell$ undergoes an independent infinite-dimensional Ornstein–Uhlenbeck process: $$da(s) = -\frac{1}{2} a(s) ds + (\kappa^2 + \ell(\ell+1))^{-\beta} dB_s.$$ Thus, modeling and generation can be performed in the spectral domain, with denoising implemented via neural predictors for the conditional expectation of the coefficients given noisy observations.

5. Malliavin–Gamma Calculus and Dirichlet Forms in the Theory of SGMs

The synthesis of Malliavin calculus and Γ-calculus (the paper of Dirichlet forms) allows for the expression of both the forward and reverse SDEs, the score function, and information-theoretic quantities in a dimension-independent, analytically robust language.

• The Dirichlet form, essentially capturing energy dissipation or the “diffusive” nature of the process, is used to define the generator of the forward SDE.
• The Malliavin derivative, acting as the proper “score” in the infinite-dimensional space, is simultaneously used to define optimal regression predictors and to link SGMs with conditional expectation-based denoising schemes.
• The carré du champ operator $\Gamma$ and Fisher information in the Cameron–Martin norm replace classical finite-dimensional analytic quantities, enabling generalization to abstract Wiener spaces and manifold-valued data.

A critical implication is that key convergence bounds (e.g., for relative entropy, marginal law matching) no longer scale with the ambient dimension, but only with regularity properties of the data as measured in the Cameron–Martin space—a phenomenon crucial for modeling high-dimensional functional data and complex random fields.

6. Summary and Significance

This Malliavin–Gamma calculus framework robustly generalizes SGMs:

• Lifting the SGM paradigm from $\mathbb{R}^d$ to infinite-dimensional Hilbert spaces enables modeling of challenging data types such as random fields and functional observations.
• The score function is characterized as a Malliavin derivative, with a concrete conditional expectation representation directly connected to $L^2$ regression.
• Entropic (KL) convergence bounds uniform over finite-dimensional projections are achieved, governed by Fisher information in the Cameron–Martin space.
• For random fields, particularly on manifolds such as the sphere, specialization to Whittle–Matérn fields and spectral decompositions provides a concrete computational pathway.
• This approach paves the way for theoretically grounded, dimension-independent generative modeling of infinite-dimensional data and augments the deployment of SGMs in scientific, functional, and geometric data settings (Greco, 19 May 2025).