The Principles of Diffusion Models (2510.21890v1)

Abstract: This monograph presents the core principles that have guided the development of diffusion models, tracing their origins and showing how diverse formulations arise from shared mathematical ideas. Diffusion modeling starts by defining a forward process that gradually corrupts data into noise, linking the data distribution to a simple prior through a continuum of intermediate distributions. The goal is to learn a reverse process that transforms noise back into data while recovering the same intermediates. We describe three complementary views. The variational view, inspired by variational autoencoders, sees diffusion as learning to remove noise step by step. The score-based view, rooted in energy-based modeling, learns the gradient of the evolving data distribution, indicating how to nudge samples toward more likely regions. The flow-based view, related to normalizing flows, treats generation as following a smooth path that moves samples from noise to data under a learned velocity field. These perspectives share a common backbone: a time-dependent velocity field whose flow transports a simple prior to the data. Sampling then amounts to solving a differential equation that evolves noise into data along a continuous trajectory. On this foundation, the monograph discusses guidance for controllable generation, efficient numerical solvers, and diffusion-motivated flow-map models that learn direct mappings between arbitrary times. It provides a conceptual and mathematically grounded understanding of diffusion models for readers with basic deep-learning knowledge.

• The paper presents a unifying framework linking variational, score-based, and flow-based approaches in diffusion models.
• It introduces a conditioning strategy that enables tractable training by replacing intractable marginal objectives with closed-form regression targets.
• The work demonstrates that continuous-time stochastic processes underpin efficient sampling, controllable generation, and algorithmic flexibility.

The Principles of Diffusion Models: A Technical Synthesis

This essay provides a comprehensive, technical summary of "The Principles of Diffusion Models" (2510.21890), focusing on the mathematical foundations, unifying perspectives, and practical implications of modern diffusion-based generative modeling. The discussion is organized around the three principal frameworks—variational, score-based, and flow-based—tracing their origins, formal equivalence, and the role of continuous-time stochastic processes. Emphasis is placed on the conditioning strategies that enable tractable training, the centrality of the time-dependent velocity field, and the implications for efficient sampling and controllable generation.

---

Foundations of Deep Generative Modeling

The monograph begins by formalizing the deep generative modeling (DGM) problem: learning a model distribution $p_{\bm{\phi}}$ to approximate an unknown data distribution $p_{\mathrm{data}}$ from i.i.d. samples. The standard approach is to minimize a divergence $\mathcal{D}(p_{\mathrm{data}}, p_{\bm{\phi}})$, with maximum likelihood estimation (MLE) corresponding to minimizing the forward KL divergence. The tractability of the normalization constant (partition function) is a central challenge, motivating the development of explicit models (e.g., NFs, ARs, VAEs, DMs) and implicit models (e.g., GANs).

Figure 1: Illustration of the target in DGM. Training a DGM minimizes the discrepancy between the model and data distributions using finite i.i.d. samples.

A taxonomy is established, distinguishing explicit models (with tractable or approximate likelihoods) from implicit models (defined only via sampling). The connection to diffusion models is made explicit: DMs inherit variational objectives from VAEs, score-matching from EBMs, and continuous-time transformations from NFs.

---

Three Core Perspectives on Diffusion Models

The monograph systematically develops three complementary frameworks for diffusion modeling:

1. Variational View: From VAEs to DDPMs

Diffusion models are cast as hierarchical latent variable models, with the forward process (encoder) fixed as a Markovian noising chain and the reverse process (decoder) learned as a sequence of denoising steps. The training objective is a variational lower bound (ELBO) on the log-likelihood, decomposing into a sum of tractable denoising subproblems. The key innovation is the conditioning strategy: by conditioning the reverse transition on the clean data, the intractable marginal KL divergence is replaced by a tractable conditional KL, enabling closed-form regression targets.

Figure 2: Illustration of the DDPM forward process, wherein Gaussian noise is incrementally added to corrupt a data sample into pure noise.

Figure 3: Illustration of DDPM reverse (denoising) process. The model sequentially denoises from pure noise to data.

Figure 4: Illustration of DDPM sampling with clean prediction: estimate $\mathbf{x}_{\bm{\phi}^\times}(\mathbf{x}_i, i)$ from $\mathbf{x}_i$, then update to $\mathbf{x}_{i-1}$.

The DDPM framework is shown to be a special case of this variational principle, with the forward process defined by a fixed Gaussian kernel and the reverse process parameterized as a Gaussian with learned mean. The $\epsilon$-prediction and $x$-prediction parameterizations are proven equivalent, both corresponding to conditional expectation minimizers.

2. Score-Based View: From EBMs to NCSN

The score-based perspective originates from energy-based models, focusing on learning the gradient of the log-density (the score function) rather than the density itself. Score matching circumvents the intractable partition function, and denoising score matching (DSM) further enables tractable training by injecting noise and regressing against the known conditional score.

Figure 5: Illustration of EBM training. The model adjusts energy to increase density at data points and decrease it elsewhere.

Figure 6: Illustration of score vector fields. The score points toward regions of higher probability.

Figure 7: Illustration of Langevin sampling. The score guides samples toward high-density regions.

Figure 8: Illustration of Score Matching. The neural network score is trained to match the ground truth score using MSE.

Figure 9: Illustration of DSM via the conditioning technique. Noise injection enables closed-form regression targets.

Figure 10: Illustration of SM inaccuracy. Low-density regions may yield inaccurate score estimates due to limited sample coverage.

Figure 11: Illustration of NCSN. The forward process perturbs data with multiple levels of additive Gaussian noise; generation proceeds via annealed Langevin dynamics.

The NCSN framework extends DSM to multiple noise levels, training a noise-conditional score network and generating samples via annealed Langevin dynamics. The equivalence between noise prediction in DDPMs and score estimation in NCSN is established via Tweedie's formula.

3. Flow-Based View: From NFs to Flow Matching

The flow-based perspective generalizes normalizing flows and neural ODEs, treating generation as following a continuous path governed by a learned velocity field. The change-of-variables formula is extended to continuous time, with the probability flow ODE (PF-ODE) providing a deterministic counterpart to the stochastic reverse SDE.

Figure 12: Illustration of sample movement of NF under an invertible map. The change in density is computed via the change-of-variables formula.

The flow matching (FM) framework further generalizes this approach, learning a velocity field to transport between arbitrary source and target distributions. The continuity equation (deterministic Fokker–Planck) ensures that the ODE flow matches the prescribed marginal densities. Conditional flow matching enables simulation-free training by regressing against tractable conditional velocities.

---

Unification via Continuous-Time Stochastic Processes

The monograph demonstrates that the variational, score-based, and flow-based perspectives are mathematically equivalent in the continuous-time limit. The forward process is formalized as a stochastic differential equation (SDE), with the reverse-time SDE and PF-ODE providing generative processes that match the evolving marginals.

Figure 13: Illustration of the discrete-time noise-adding step. Noise is added from $t$ to $t+\Delta t$ with mean drift and diffusion.

Figure 14: (1D) Visualization of the forward process in a diffusion model. The process evolves from a complex data distribution to a simple Gaussian prior.

Figure 15: Visualization of the reverse-time stochastic process for data generation. Samples evolve from the prior to the data distribution.

Figure 16: (2D) Illustration of sampling from the Score SDE. Both SDE and ODE trajectories terminate near the data manifold.

Figure 17: Illustration of Lemma on the equivalence between forward perturbation kernels and linear SDEs.

The Fokker–Planck equation is shown to govern the evolution of the marginals, ensuring that both the stochastic and deterministic flows yield the same family of distributions. The conditioning strategy is revealed as a unifying principle: by conditioning on clean data or latent variables, intractable marginal objectives are replaced by tractable conditional ones, enabling efficient regression-based training.

---

Practical Implications: Sampling, Controllability, and Acceleration

Sampling from diffusion models is equivalent to numerically integrating the reverse SDE or PF-ODE, which is computationally expensive due to the need for many small steps to maintain accuracy. The monograph discusses advanced numerical solvers and distillation-based methods for accelerating sampling, as well as guidance techniques (e.g., classifier-free guidance) for controllable generation.

The flow-based perspective, particularly flow matching, enables direct learning of solution maps (flow maps) between arbitrary times, supporting anytime-to-anytime generation and efficient distribution-to-distribution translation.

---

Theoretical and Practical Implications

The unification of diffusion models under the continuous-time, velocity-field-driven framework has several implications:

• Theoretical Clarity: The equivalence of variational, score-based, and flow-based formulations provides a rigorous foundation for understanding and analyzing diffusion models. The centrality of the time-dependent velocity field and the role of the Fokker–Planck equation clarify the relationship between stochastic and deterministic generative processes.
• Algorithmic Flexibility: The conditioning strategy enables tractable training across diverse formulations, supporting both simulation-based and simulation-free approaches. The flow matching framework generalizes diffusion modeling to arbitrary source and target distributions, broadening the applicability of these methods.
• Sampling Efficiency: The recognition that sampling is a differential equation problem motivates the use of advanced numerical solvers and direct flow-map learning for acceleration. The deterministic PF-ODE enables exact likelihood computation and bidirectional mapping, supporting applications in inversion and controllable generation.
• Future Directions: The principled foundation established by this monograph suggests several avenues for further research, including the design of more expressive velocity fields, improved numerical integration schemes, and the extension of flow-based methods to high-dimensional, multimodal, or discrete data domains.

---

Conclusion

"The Principles of Diffusion Models" provides a mathematically rigorous, conceptually unified, and practically relevant account of diffusion-based generative modeling. By tracing the origins and formal equivalence of variational, score-based, and flow-based perspectives, the monograph establishes the time-dependent velocity field as the backbone of modern diffusion models. The conditioning strategy emerges as a key enabler of tractable training, while the continuous-time framework clarifies the relationship between stochastic and deterministic generation. These insights lay a stable foundation for both the application and further development of diffusion models in generative modeling and beyond.