Diffusion-Based Generative Models

• Diffusion-based generative models are deep learning frameworks that reverse a multi-step noise injection process to accurately reconstruct complex data samples.
• They use a discrete-time Markov chain with a forward corruption process and a learned backward denoising step via architectures like U-Net and Transformers.
• Recent advances extend these models with stochastic differential equations, domain-specific conditioning, and accelerated sampling techniques to enhance generation quality.

A diffusion-based generative model is a class of deep generative model that constructs samples by learning to reverse a multi-step stochastic process that gradually destroys data structure through noise injection. Formulated originally in the context of image synthesis, but now spanning diverse application domains, these models define a tractable forward corruption process and a parameterized neural backward or denoising process that successively "removes" noise to reconstruct complex samples. This paradigm has established new performance frontiers in unconditional and conditional generation across images, sequences, scientific data, and more.

1. Mathematical Formulation of Diffusion-Based Generative Models

The prototypical framework defines a discrete-time Markov chain for the forward (noising) and reverse (generation) processes. The forward process corrupts a data sample $x_0 \in \mathbb{R}^d$ using a sequence of conditional Gaussian distributions:

$$q(x_t|x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t} x_{t-1},\, \beta_t I)$$

Given $T\gg1$ timesteps per sample, the cumulative effect transforms $x_0$ into nearly isotropic Gaussian noise:

$$q(x_t|x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t}\, x_0,\, (1 - \bar{\alpha}_t) I), \quad \bar{\alpha}_t = \prod_{i=1}^t (1-\beta_i)$$

The reverse process models conditional transitions via a parameterized neural network (often a U-Net or Transformer, depending on data domain):

$$p_\theta(x_{t-1}|x_t) = \mathcal{N}(x_{t-1};\, \mu_\theta(x_t, t),\, \Sigma_\theta(x_t, t))$$

The network typically learns either the conditional mean $\mu_\theta$ (often re-parameterized via noise-prediction) or, equivalently, the noise added at each step. The training objective is derived from maximizing the data likelihood via a variational lower bound, simplifying to an L2 loss between true and predicted noise:

$$\mathcal{L}_{\text{simple}} = \mathbb{E}_{x_0,\,\epsilon,\,t}\left[\|\epsilon - \epsilon_\theta(\sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t}\epsilon,\, t)\|^2\right]$$

This objective is theoretically justified by the exact decomposition of the variational bound into tractable KL-divergences between Gaussian distributions (Deja et al., 2022, Higham et al., 2023, Zhen et al., 14 Dec 2024).

2. Backward Diffusion Process: Generation as Inexact Denoising

The backward, or reverse, diffusion is conceptually the “unrolling” of the forward process. It is initiated from pure Gaussian noise $x_T \sim \mathcal{N}(0, I)$. At each step, the model updates the sample by “removing” noise, reconstructing structure incrementally.

The process exhibits a fluid transition: in initial reverse steps (high noise), the model acts as a generator, synthesizing coarse structure from randomness. As the noise decays, it becomes a denoiser, refining corruption into high-fidelity content. Empirically, this transition occurs in the first ~10-20% of steps, as seen in reconstruction error, SNR, and MS-SSIM trends (Deja et al., 2022). This motivates a division into two phases, which can be exploited to design hybrid models (see Section 4).

The reverse Markov chain is

$$p(x_0, ..., x_T) = p(x_T) \prod_{t=1}^T p_\theta(x_{t-1} | x_t)$$

with $p(x_T)$ a standard Gaussian. Approximating $p_\theta$ for all $t$ is essential for sample fidelity; imperfect modeling leads to accumulation of error and degraded output.

3. Extensions: Flexible and Structured Diffusion Mechanisms

Subsequent research generalizes the original framework along several axes:

• SDE Parameterization and Geometry: Recent formulations interpret the forward and backward processes as solutions to coupled stochastic differential equations (SDEs), permitting learnable, spatially-varying metrics and Hamiltonian components (Du et al., 2022). This includes extensions such as sub-VP SDEs and critically-damped Langevin SDEs. For a forward SDE

$dX_t = f(X_t, t)dt + g(t)dW_t,$

the reverse is

$$dY_t = [f(Y_t, t) - g^2(t) \nabla \log p_t(Y_t)]dt + g(t) d\tilde{W}_t.$$

Parameterizations by Riemannian geometry and Hamiltonian structures (via the metric $R(x)$ and anti-symmetric matrix $\omega$) allow "forward processes" tuned to data geometry, improving density estimation and sample diversity.

• Bridged and Constrained Domains: The connection to latent variable modeling allows for algorithmic extensions to discrete, structured, or constrained domains. Constructing "diffusion bridges" (i.e., SDEs conditioned to hit specified endpoints or constraints) enables generation of segmentation maps, discrete-valued samples, and point clouds with domain-imposed structure. Theoretical error analysis quantifies how discretization and sample size impact distributional accuracy (Liu et al., 2022).
• PDE-Driven and Spectral Corruption: Generalizations to the forward process include PDE-driven advection-diffusion-reaction operators (incorporating both diffusion and advection terms) (Gruszczynski et al., 20 Jun 2025), and frequency-domain approaches leveraging Fourier or DCT transforms for scale-dependent, energy-aware noise injection, inspired by renormalization group flow in physics (Sheshmani et al., 26 Feb 2024). These approaches enable multi-scale and physically plausible corruption processes that can improve synthesis quality and sampling efficiency.

4. Architectural Developments: Model Division, Conditioning, and Acceleration

A key empirical insight is the value in dividing the generative model into two functional phases, demarcated by a fluid transition step:

Phase	Function	Network Type
Generator	Structure synthesis from noise	Diffusion U-Net / Score Model
Denoiser	Refinement/removal of moderate corruption	Denoising Autoencoder

Such an explicit separation (the DAED architecture) can improve performance and generalization, particularly in transfer scenarios where artifacts from shared parameterization are undesirable. The main trade-off is a potential reduction in diversity when using non-variational objectives in the denoiser segment (Deja et al., 2022).

Conditioning on additional signals—such as geometry in flow field prediction (obstacle-conditioned via cross-attention and U-Net) (Hu et al., 30 Jun 2024) or multi-modality in unified generation (shared latent space, modality-specific decoders) (Chen et al., 24 Jul 2024)—broadens applicability.

Speeding up diffusion sampling is a parallel theme. Analytical approximations allow omitting early reverse steps, exploiting closed-form Gaussian solutions to accelerate generation (teleporting or skipping steps) (Wang et al., 2023). Image-aware, pixel-wise schedules (exponential SNR decay per-pixel via a water-filling analogy) and autoencoder-predicted diffusion coefficients further reduce required steps, with parallel reverse-time networks eliminating MCMC post-processing (Asthana et al., 15 Aug 2024).

5. Empirical Evaluation and Sample Quality Metrics

Performance of diffusion-based generative models is typically benchmarked using metrics such as the Fréchet Inception Distance (FID), Inception Score (IS), and negative log-likelihood (NLL):

• FID: Evaluates the Wasserstein-2 distance between the Gaussian feature statistics (mean and covariance) of generated and real images, measuring perceptual similarity (Masuki et al., 15 Jan 2025). Lower FID indicates higher quality and diversity.
• MAE, MS-SSIM: Used to probe fidelity at various diffusion steps, help locate the generator/denoiser transition, and measure mode collapse or smoothing.
• Empirical Outcomes: Renormalization group-inspired diffusion models show that FID and sample quality can be maintained or improved with an order of magnitude fewer generation steps, reducing computational cost substantially (e.g., 200-500 steps vs. 1000+) (Masuki et al., 15 Jan 2025, Sheshmani et al., 26 Feb 2024, Asthana et al., 15 Aug 2024).

6. Domain-Specific and Multimodal Generalizations

Diffusion-based generative modeling has been extended far beyond imagery:

• Fluid Dynamics and Physics: For flow field prediction, models learn geometry-conditioned denoising, outperforming CNN and VAE baselines in accuracy, robustness, and preservation of nonlinear physical invariants (Hu et al., 30 Jun 2024). Surface structure generation with rotationally equivariant neural networks enables discovery and generation of atomic surface phases far exceeding the size and complexity seen in training, leveraging physical constraints such as substrate registry and domain periodicity (Rønne et al., 27 Feb 2024).
• Bayesian Inference: Multimodal and high-dimensional distributions (as in Bayesian inverse problems) are addressed by decomposing the target measure into locally unimodal domains, training diffusion models for each, and using bridge sampling for correct mode mixing (Tran et al., 20 Apr 2025). This "divide and conquer" approach yields scalable, high-fidelity posterior sampling in previously intractable settings.
• Multi-Modality: Unified multi-modal diffusion architectures learn common latent spaces for disparate types (images, labels, representations), facilitating multi-task supervision, better cross-modality transfer, and simultaneous multi-output generation (Chen et al., 24 Jul 2024).

7. Historical Evolution and Future Directions

Diffusion-based models originated with noise-injection Markov processes (Sohl-Dickstein et al., 2015), with DDPM (Ho et al., 2020) popularizing the discrete-time Gaussian setup and U-Net parameterization. Later, continuous-time SDEs (Song et al., 2020) generalized the theory, bridging score matching, VAEs, and denoising autoencoders under a unified mathematical umbrella (Zhen et al., 14 Dec 2024, Cao et al., 2022).

Trends and challenges highlighted in comprehensive surveys (Cao et al., 2022) include:

• Strategies for accelerating sampling (e.g., knowledge distillation, advanced SDE/ODE solvers, pixel-wise schedules)
• Improved noise schedules and training protocols
• Enhanced data efficiency, especially in low-data or domain-transfer regimes
• Extension to discrete, structured, and physics-constrained domains
• Integration with multi-modal, multi-task, and conditional frameworks
• Deeper connections with physical principles (e.g., renormalization group, optimal transport, physically informed PDEs)
• Theoretical investigations of sample quality, convergence rates, and error bounds

Future research is expected to further automate schedule optimization, merge with reinforcement and graph learning, exploit new physical and theoretical principles (fluid/advection-driven corruption, RG flows, optimal transport), and generalize generation to ever more complex, multimodal data landscapes.

---

References (arXiv ids):

• (Deja et al., 2022): On Analyzing Generative and Denoising Capabilities of Diffusion-based Deep Generative Models
• (Du et al., 2022): A Flexible Diffusion Model
• (Liu et al., 2022): Let us Build Bridges: Understanding and Extending Diffusion Generative Models
• (Cao et al., 2022): A Survey on Generative Diffusion Model
• (Wang et al., 2023): Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later
• (Higham et al., 2023): Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians
• (Sheshmani et al., 26 Feb 2024): Renormalization Group flow, Optimal Transport and Diffusion-based Generative Model
• (Rønne et al., 27 Feb 2024): Generative diffusion model for surface structure discovery
• (Hu et al., 30 Jun 2024): Generative prediction of flow fields around an obstacle using the diffusion model
• (Bae et al., 9 Jul 2024): A Very Effective and Simple Diffusion Reconstruction for the Diluted Ising Model
• (Chen et al., 24 Jul 2024): Diffusion Models For Multi-Modal Generative Modeling
• (Asthana et al., 15 Aug 2024): Accelerated Image-Aware Generative Diffusion Modeling
• (Zhen et al., 14 Dec 2024): Diffusion Model from Scratch
• (Le, 14 Dec 2024): Generative Modeling with Diffusion
• (Masuki et al., 15 Jan 2025): Generative diffusion model with inverse renormalization group flows
• (Tran et al., 20 Apr 2025): Diffusion-based supervised learning of generative models for efficient sampling of multimodal distributions
• (Gruszczynski et al., 20 Jun 2025): Beyond Blur: A Fluid Perspective on Generative Diffusion Models
• (Kim et al., 25 Jul 2025): A diffusion-based generative model for financial time series via geometric Brownian motion