Score-Based Generative Models

• Score-Based Generative Models are defined via stochastic differential equations that iteratively refine noisy samples using learned score functions.
• They use time-reversed dynamics with neural networks to estimate the gradient of log-probability, enabling effective conditional and unconditional synthesis.
• Recent advances extend these models to structured data and accelerated sampling methods, achieving state-of-the-art performance in image and speech generation.

Score-Based Generative Model (SGM) refers to a class of generative models that synthesize samples by leveraging the score function, i.e., the gradient of the log-probability density, estimated for a sequence of noise-perturbed intermediate distributions via stochastic differential equations (SDEs). SGMs formulate data generation as a stochastic process of progressively denoising a sample from a simple prior (such as a Gaussian) using the time-reversed dynamics of a carefully chosen forward diffusion, underpinned by rigorous mathematical theory and expressive neural estimation of the score field.

1. Mathematical Foundations of Score-Based Generative Modeling

Score-based generative models are built on a Markovian noising process defined by a forward SDE on $z_t \in \mathbb{R}^d$: $dz_t = f(z_t, t) \, dt + g(t) \, dW_t,$ where $f$ is the drift, $g$ the (typically time-dependent) diffusion, and $W_t$ a standard Wiener process. The marginal distribution $p_t(z)$ is designed to converge at $t \to T$ to a tractable prior $p_T$ (commonly isotropic Gaussian).

The most general recipe for constructing such SDEs follows from the parameterization of scalable Bayesian posterior samplers: $$f(z, t) = -[D(z, t) + Q(z, t)] \nabla H(z) + \tau(z, t), \quad g(t)g(t)^\top = 2D(z, t),$$ with $H(z)$ the Hamiltonian (often quadratic), $D(\cdot) \succeq 0$ a positive-semidefinite diffusion matrix, $Q(\cdot)$ a skew-symmetric matrix, and $\tau$ a divergence correction term. This construction ensures that the SDE's stationary distribution is $p_s(z) \propto e^{-H(z)}$ under general assumptions, providing a theoretically complete parameterization of all diffusion samplers converging to a prescribed prior (Pandey et al., 2023).

Reversing the SDE yields the generative process. The backward (data-generating) SDE for $x_t$ is given by

$$dx_t = \Bigl[ f(x_t, t) - g(t)g(t)^\top \nabla_x \log p_t(x_t) \Bigr] dt + g(t) d\bar W_t,$$

where $\bar W_t$ is a reverse-time Wiener process. The time-dependent score field $\nabla_x \log p_t(x)$ is unknown and estimated by a neural network $s_\theta(x, t)$.

Classic reductions recover discrete DDPMs and continuous-time variance-preserving (VP) or variance-exploding (VE) SDEs as special cases within this parameterization. However, certain forward processes (notably VE SDEs that do not converge to an isotropic Gaussian) do not fit this unifying structure.

2. Phase Space Langevin Diffusion and Algorithmic Implementation

Extending the standard SGM recipe, Phase Space Langevin Diffusion (PSLD) augments the state space to include auxiliary "momentum" variables $v$ paired with the original space $x$. The augmented state $z = (x, v)$ evolves under a Hamiltonian $H(x, v) = U(x) + \frac{1}{2}v^\top M^{-1}v$, leading to forward dynamics: $$\begin{pmatrix} dx_t \ dv_t \end{pmatrix} = \frac{\beta}{2} \begin{pmatrix} -\Gamma I & M^{-1} \ -I & -\nu I \end{pmatrix} \begin{pmatrix} x_t \ v_t \end{pmatrix} dt + \sqrt{\beta} \begin{pmatrix} \sqrt{\Gamma}I & 0 \ 0 & \sqrt{M\nu}I\end{pmatrix} dW_t.$$ The reverse SDE follows the general recipe and involves two neural score fields: $s_x$ for $x$ and $s_v$ for $v$. The model is trained with a Gaussian conditional for $p(z_t\mid z_0)$ using a hybrid score-matching loss: $$\mathcal{L} = \mathbb{E} \left[ \Gamma \beta \|s_x(z_t, t) - \nabla_{x_t} \log p(z_t|z_0)\|^2 + M\nu \beta \|s_v(z_t, t) - \nabla_{v_t} \log p(z_t|z_0)\|^2 \right].$$ Sampling is performed with an operator-splitting scheme (SSCS splitting), conducting analytic Ornstein–Uhlenbeck (OU) flow for the linear (A) step and explicit Euler steps for the score-based (S) dynamics, enabling lower discretization errors.

Empirically, PSLD achieves state-of-the-art FID scores on image generation tasks (FID ≈ 2.10 on CIFAR-10 with 200 ODE steps; FID ≈ 2.01 on CelebA-64 with 250 steps), outperforming previous baselines in the trade-off between sample quality and computational cost (Pandey et al., 2023).

3. Conditional Generation, Fine-Tuning, and Task Transfer

Conditional synthesis in SGMs leverages a pretrained unconditional score network $s_\theta(z, t)$ while augmenting the reverse SDE drift with a classifier $c_\phi(x_t, t)$ trained on noisy data: $$\nabla_{z_t} \log p(z_t \mid y) \approx s_\theta(z_t, t) + \lambda \nabla_{z_t}\log c_\phi(y \mid x_t).$$ This yields flexible conditional sampling and is effective for class-conditional generation and inpainting within the same algorithmic skeleton. Both classifier guidance and prompt-based conditioning integrate naturally within this SGM framework.

The generic conditional SGM approach is broadly applicable: for instance, in time-series synthesis, a conditional score network $M_\theta(h_n^s, h_{n-1}^0, s)$ models the conditional distribution of each time step in the latent space, trained with a denoising score-matching loss derived for the autoregressive structure (Lim et al., 26 Nov 2025, Lim et al., 2023). In speech enhancement, the score network operates on the complex STFT domain, trained via an SDE-based objective without assumptions on noise distributions (Welker et al., 2022).

4. Convergence Theory and Statistical Guarantees

Recent works establish rigorous convergence rates for SGMs under various metrics and data assumptions:

• For sufficiently smooth, log-concave data (e.g., when $-\log p_0$ is strongly convex and smooth), 2-Wasserstein convergence is polynomial in the inverse accuracy and dimension. For the Euler-discretized process with $K$ steps:

$$W_2(\operatorname{Law}(y_K), p_0) \leq O(\text{prefactor}(d), 1/K)$$

with minimax lower bounds showing no process beats the $O(d/\epsilon^2)$ rate (Gao et al., 2023).

• For smooth, sub-Gaussian densities whose log-relative density is locally approximable by a bounded neural network, SGMs achieve dimension-independent total variation error $O(\epsilon)$ using polynomial sample and network complexity (Cole et al., 2024).
• Detailed studies show that $L^2$-accurate score estimates guarantee convergence in Wasserstein and total variation distances, for high-dimensional, non-smooth, multimodal, and even manifold-supported data, provided step sizes and annealing schedules are appropriately chosen (Lee et al., 2022, Lee et al., 2022, Pidstrigach, 2022).
• Recent analysis demonstrates robustness of SGMs under finite-sample error, model misspecification, early stopping, and in the choice of score-matching objective, with explicit uncertainty propagation bounds in Wasserstein-1 and other IPMs (Mimikos-Stamatopoulos et al., 2024).

A notable theoretical nuance is that minimizing $L^2$ score error without further constraints can result in purely memorizing models that only reproduce blurred training data, as these may constitute the optimal solution when score matching is performed over an empirical measure (Li et al., 2024). This highlights a need for generative criteria that promote generalization (i.e., coverage beyond the empirical support), beyond naive score-matching.

5. Algorithmic Efficiency, Acceleration, and Generalization

SGM sampling, especially with Euler discretization, incurs substantial computational demands due to thousands of iterative steps. Data-adaptive preconditioned diffusion sampling (PDS) addresses this bottleneck by introducing matrix-based preconditioning—constructed from pixel- and frequency-domain statistics—that standardizes coordinate scales and enables reduction of the iteration count by up to 28×, without sacrificing sample quality or requiring network retraining (Ma et al., 2023).

Empirical and theoretical work emphasizes the crucial influence of optimizer hyperparameters (learning rate, batch size) and algorithmic trajectory on the generalization gap of the score network. Generalization bounds for SGMs now incorporate both sample size and algorithmic details, with SGLD- and topology-based techniques yielding concrete, data- and optimizer-dependent estimates. Late-stage optimizer dynamics (e.g., flatness of the solution basin, trajectory topology) afford diagnostics for model generalization and stability (Dupuis et al., 4 Jun 2025).

Noise scheduling is another key axis: the variance schedule $\beta(t)$ fundamentally modulates mixing, convergence rates, and the impact of discretization. Theoretical bounds recommend making $\beta$ a tunable parameter and jointly optimizing it with the score network, as time-inhomogeneous schedules are shown to yield significant improvements in KL divergence and FID across both synthetic and real-data tasks (Strasman et al., 2024).

6. Extensions to Structured, Functional, and Geometric Data

Score-based generative modeling generalizes naturally to non-Euclidean data:

• In Riemannian score-based generative models (RSGMs), the forward diffusion and reverse SDEs are intrinsically defined on manifolds, with the score field replaced by the Riemannian gradient of the log density. This approach enables modeling on spheres ($\mathbb{S}^2$), tori, groups such as $\mathrm{SO}(3)$, and the hyperbolic plane, using geodesic random walks for SDE discretization. RSGMs match or outperform Moser flows and CNF baselines while remaining efficient and scalable (Bortoli et al., 2022).
• Functional data can be modeled through spectral SGMs which represent processes via truncated Karhunen–Loève expansions and apply finite-dimensional SGMs in coefficient space. The algorithm is competitive on multivariate and high-dimensional function datasets and supports modeling of multimodal and long-range correlated phenomena (Phillips et al., 2022).
• In biophysical and molecular structure design (e.g., LoopGen for peptide backbone modeling), SGMs operate over rigid-body frames ($SE(3)$), using equivariant architectures and specialized variance schedules, with reverse SDEs defined on the product manifold of rotations and translations (Boom et al., 2023).

Finally, the underlying mathematical structure of SGMs can be characterized as the Wasserstein proximal operator (WPO) for cross-entropy to data, connecting SGM sampling to the solution of coupled mean-field games (MFGs)—a forward Fokker-Planck and a backward Hamilton-Jacobi-Bellman PDE—thereby providing a principled framework for kernel-based score estimation, direct control of inductive bias, and mitigation of memorization pathologies (Zhang et al., 2024).

7. Practical Impact, Limitations, and Ongoing Developments

SGMs have set state-of-the-art sample quality metrics (e.g., FID $\sim 2.1$ on CIFAR-10, competitive performance on CelebA-64 and CelebA-HQ-256 (Pandey et al., 2023, Vahdat et al., 2021)) and demonstrated versatility in conditional generation, regular and irregular time-series synthesis (Lim et al., 26 Nov 2025, Lim et al., 2023), speech enhancement (Welker et al., 2022), and more.

Key practical considerations and active directions include:

• Avoiding memorization by constructing score models that generalize beyond empirical data points, enforced via kernel mixtures or through regularization of the score at the final time.
• Acceleration through preconditioning, adaptive noise scheduling, and sampler distillation.
• Conditionality and task transfer through reusable unconditional score networks and systematic classifier guidance.
• Extensions to structured domains such as manifolds, functional spaces, and biomolecular structures.
• Algorithmic diagnostics and hyperparameter selection informed by data- and optimizer-dependent generalization bounds.

Outstanding challenges remain in balancing sample efficiency, generative diversity, computational speed, and theoretical guarantees, particularly as models are deployed across increasingly high-dimensional, structured, and non-Euclidean domains. SGMs continue to evolve both as a powerful modeling paradigm and as a theoretical subject with deep connections to stochastic analysis, optimal transport, and PDE theory.

---

References

• "A Complete Recipe for Diffusion Generative Models" (Pandey et al., 2023)
• "TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models" (Lim et al., 26 Nov 2025)
• "Preconditioned Score-based Generative Models" (Ma et al., 2023)
• "Score-Based Generative Models Detect Manifolds" (Pidstrigach, 2022)
• "Algorithm- and Data-Dependent Generalization Bounds for Score-Based Generative Models" (Dupuis et al., 4 Jun 2025)
• "Moderating the Generalization of Score-based Generative Model" (Jiang et al., 2024)
• "Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models" (Gao et al., 2023)
• "Convergence for score-based generative modeling with polynomial complexity" (Lee et al., 2022)
• "Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions" (Cole et al., 2024)
• "A Good Score Does not Lead to A Good Generative Model" (Li et al., 2024)
• "An analysis of the noise schedule for score-based generative models" (Strasman et al., 2024)
• "Score-based Generative Models for Designing Binding Peptide Backbones" (Boom et al., 2023)
• "Riemannian Score-Based Generative Modelling" (Bortoli et al., 2022)
• "Spectral Diffusion Processes" (Phillips et al., 2022)
• "Score-based Generative Modeling in Latent Space" (Vahdat et al., 2021)
• "Wasserstein proximal operators describe score-based generative models and resolve memorization" (Zhang et al., 2024)
• "Score-based generative models are provably robust: an uncertainty quantification perspective" (Mimikos-Stamatopoulos et al., 2024)
• "Convergence of score-based generative modeling for general data distributions" (Lee et al., 2022)