Score-Based Diffusion Models

• Score-based diffusion models are generative frameworks that transform structured data into noise and then reverse the process using an explicitly computed score function.
• They directly model function-valued data in separable Hilbert spaces through stochastic partial differential equations without relying on finite-dimensional discretization.
• Techniques from Malliavin calculus and operator theory yield a closed-form score function that preserves the geometry of the data and enhances generative stability and scalability.

Score-based diffusion models are a class of generative models that define a stochastic process to continuously transform structured data into noise and then generate new samples by simulating a learned reversal of this process. In the infinite-dimensional context, these models are constructed and analyzed within separable Hilbert spaces, enabling the modeling of function-valued data (e.g., solutions to partial differential equations, images seen as elements of $L^2$) and circumventing the limitations of finite-dimensional or discretized approaches. The core innovation is the explicit computation of the score function—the Fréchet derivative of the log-density—in infinite-dimensional settings, leveraging tools from Malliavin calculus and operator theory to maintain consistency with the underlying geometry of the data space, accommodate spatial correlations in noise, and connect with modern approaches in functional data analysis (Mirafzali et al., 27 Aug 2025).

1. Infinite-Dimensional Mathematical Framework

Score-based diffusion modeling in infinite dimensions is formulated on a separable Hilbert space $H$, such as $L^2$ over a spatial domain. The forward process is modeled as a linear stochastic partial differential equation (SPDE):

$$du(t) = A u(t) dt + Q^{1/2} dW_t, \quad u(0) = u_0 \in H,$$

where $A$ is a densely defined, unbounded (typically elliptic) operator generating a strongly continuous semigroup $S(t) = e^{tA}$, $Q^{1/2}$ is a Hilbert–Schmidt operator encoding the spatial covariance of the (possibly colored) noise, and $W_t$ is an $H$-valued cylindrical Wiener process. The mild solution to this equation is

$u(t) = S(t) u_0 + \int_0^t S(t-s) Q^{1/2} dW_s.$

The Malliavin covariance operator is defined as

$$\gamma_{u(t)} = \int_0^t S(s) Q^{1/2}(Q^{1/2})^* S(s)^* ds,$$

which plays a central role in the derivation of the infinite-dimensional score.

2. Rigorous Treatment of the Forward Diffusion Process

The use of a trace-class noise operator ($Q$ is trace class) ensures well-posedness and finite variance of the solution in the infinite-dimensional space $H$. The stochastic convolution

$\int_0^t S(t-s) Q^{1/2} dW_s$

is then well-defined in $L^2(\Omega; H)$ even in arbitrary spatial dimensions. The noise term generalizes white noise to colored (correlated) noise, accommodating physical systems with spatial structure. Importantly, the framework is constructed without invoking any finite-dimensional projection or discretization, in contrast to standard approaches that would rely on grid-based representations or truncation to $N$-dimensional subspaces.

3. Malliavin Calculus and Explicit Score Formula

Malliavin calculus provides the machinery to compute the sensitivity of the solution $u(t)$ with respect to perturbations in the driving noise. The explicit form of the Malliavin derivative is

$D_r u(t) = S(t-r) Q^{1/2} \,\mathbf{1}_{[0,t]}(r),$

which quantifies the impact of a “noise impulse” at time $r$. The covariance operator $\gamma_{u(t)}$ aggregates this sensitivity over all $r \in [0,t]$. Through an infinite-dimensional generalization of the Bismut–Elworthy–Li formula, the score—the Fréchet derivative of the log-density—is given by

$$\nabla_h \log p_{u(t)}(u) = -\langle \gamma_{u(t)}^\dagger (u - S(t) u_0), h \rangle_H,$$

where $h$ is any Cameron–Martin direction (i.e. in the effective range of $\gamma_{u(t)}^{1/2}$) and $\gamma_{u(t)}^\dagger$ is the Moore–Penrose pseudoinverse of the covariance operator. This analytic, closed-form formula does not require discretization or projection, and is intrinsic to the Hilbert space structure.

4. Operator-Theoretic Analysis and Spatially Correlated Noise

The analysis relies on operator-theoretic constructions:

• The semigroup $S(t)$, encoding the flow induced by the drift $A$, serves as the first variation process, respecting the geometry of $H$.
• Bounded operators and their adjoints structure the progression and sensitivity of the stochastic process.
• The framework accommodates trace-class and thus spatially correlated noise, and is not dependent on the invertibility of $S(t)$.
• When the semigroup is invertible, the covariance can alternatively be represented as

$$\gamma_{u(t)} = Y_t C_t Y_t^*, \quad C_t = \int_0^t Y_r^{-1} Q^{1/2}(Q^{1/2})^*(Y_r^{-1})^* dr,$$

allowing further flexibility in treating the stochastic structure.

5. Connections to Functional Data Analysis and Learning Strategies

Score estimation in this infinite-dimensional setting is naturally linked to methodologies in functional data analysis:

• The score function is framed as a regression operator acting on the space $H$, enabling the application of kernel methods in reproducing kernel Hilbert spaces and neural operator architectures for efficient function-to-function learning.
• This operator perspective allows the deployment of advanced computational tools for estimation and sampling in high-dimensional or function-valued data without ad hoc discretization, and aligns with recent developments in neural operator learning.

6. Implications for Generative Modeling and Applications

The framework enables generative modeling of infinite-dimensional random elements, such as function-valued data and solutions to SPDEs, directly in Hilbert space. Potential impacts include:

• Rigorous algorithms for infinite-dimensional data synthesis, sampling, and denoising that respect the geometry and statistical properties of the underlying functional space.
• The analytic form of the score function provides a principled basis for constructing reverse-time SDEs for generative modeling in function space, extending the reach of diffusion models to scientific computing domains, climate and materials simulation, and medical imaging, where the data is intrinsically infinite-dimensional or requires resolution independence.
• By not requiring discretization, the approach improves theoretical guarantees, scalability, and stability, especially in applications where spatial correlations are significant.

7. Outlook and Future Research Directions

This framework lays a rigorous mathematical foundation for score-based diffusion modeling in infinite-dimensional settings, suggesting several avenues for future investigation:

• Extending the methodology to nonlinear SPDEs and to systems with multiplicative (state-dependent) noise, broadening the class of physical and scientific processes that can be modeled.
• Integrating advanced regularization and kernel-based methods from functional data analysis for improved score estimation and computational tractability.
• Applying neural operator and kernel architectures to leverage the operator-theoretic structure, thus enabling efficient learning and inference in large-scale functional datasets.
• Exploring the impact of the infinite-dimensional framework on the expressivity and generalization capacity of generative models in practical applications.

In summary, the operator-theoretic use of Malliavin calculus yields a closed-form, infinite-dimensional score formula that preserves the intrinsic geometry of Hilbert spaces, accommodates spatially correlated noise, and bridges generative modeling with modern functional analytic and computational techniques (Mirafzali et al., 27 Aug 2025).