Generative Diffusion Models

• Generative Diffusion Models are deep generative methods that reverse a gradual noise corruption process using stable likelihood- or score-based training.
• They employ a paired forward-reverse stochastic process with Gaussian denoising and score estimation to enable versatile conditional generation across images, audio, text, and more.
• Despite producing high-quality outputs, their iterative sampling is computationally intensive and poses challenges in controllability and efficiency.

Generative Diffusion Models (GDMs) are deep generative models that learn to synthesize data by reversing a gradual corruption process: a forward stochastic transformation maps structured data to nearly Gaussian noise, and a learned reverse process reconstructs samples from that noise. In the standard formulation, GDMs encompass discrete denoising diffusion probabilistic models, score-based models, continuous-time SDE/ODE formulations, conditional and latent variants, and more recent structured extensions. Across the literature, they are characterized by stable likelihood- or score-based training, strong conditional generation capability, and broad applicability across images, audio, text, graphs, inverse problems, and network optimization, while also retaining well-documented limitations in sampling efficiency, controllability, and structural fidelity (Cao et al., 2022, Ding et al., 2024).

1. Foundational formulation

At the core of a GDM is a pair of coupled stochastic processes. In the discrete DDPM formulation, the forward process is a Markov chain that incrementally adds Gaussian noise,

$$q(x_t\mid x_{t-1})=\mathcal{N}\!\left(x_t;\sqrt{1-\beta_t}\,x_{t-1},\beta_t I\right),$$

so that after sufficiently many steps the sample approaches isotropic Gaussian noise. The reverse process is learned as

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$

and generation starts from $x_T\sim\mathcal N(0,I)$ and iterates denoising until $x_0$ is obtained (Cao et al., 2022).

A standard closed form for the noised sample is

$$q(x_t \mid x_0)=\mathcal{N}(x_t;\sqrt{\bar{\alpha}_t}x_0,(1-\bar{\alpha}_t)I), \qquad \bar{\alpha}_t=\prod_{i=1}^{t}(1-\beta_i),$$

which makes training practical because any noise level can be sampled directly from a clean example rather than by simulating the full chain (Torre, 2023). The same source emphasizes that corruption must be very gradual for reversibility; if noise is added too aggressively, information is destroyed too quickly for the inverse mapping to remain learnable, which is why practical schedules often use on the order of a thousand steps (Torre, 2023).

This “diffuse-then-denoise” perspective is one reason the survey literature contrasts GDMs favorably with earlier families. Compared with VAEs, diffusion models avoid difficult posterior alignment; compared with GANs, they replace adversarial training with a stable likelihood- or score-based objective; compared with EBMs, they avoid expensive MCMC during training; and compared with normalizing flows, they relax the requirement of exact invertibility and rigid architectural constraints (Cao et al., 2022).

2. Score estimation, objectives, and continuous-time theory

A major unification in the literature is that reverse diffusion can be parameterized either as Gaussian denoising or as score estimation. In the common noise-prediction parameterization,

$$x_t=\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\,\varepsilon, \qquad \varepsilon\sim\mathcal N(0,\mathbf I),$$

and the denoiser predicts $\epsilon_\theta(x_t,t)$. The practical clean-sample estimator is then

$$\hat{x}_0=\frac{x_t-\sqrt{1-\bar{\alpha}_t}\,\epsilon_\theta(x_t,t)}{\sqrt{\bar{\alpha}_t}},$$

while the score is related by

$$s_\theta(x_t,t)=-\frac{\epsilon_\theta(x_t,t)}{\sqrt{1-\bar{\alpha}_t}}.$$

The handbook presents these parameterizations as largely interchangeable views of the same generative process (Ding et al., 2024).

Training is usually framed as denoising regression. A standard objective is the simplified DDPM loss

$$L_{\text{simple}}=\mathbb{E}_{x_0,\epsilon,t}\left[\|\epsilon-\epsilon_\theta(x_t,t)\|^2\right],$$

which is the practical form of the variational objective used in many implementations (Torre, 2023). In the score-based formulation, the corresponding target is the score field $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$0, and the survey literature treats DDPM noise prediction and denoising score matching as reparameterizations of the same learning problem (Cao et al., 2022).

Continuous-time formulations make this connection explicit. The forward corruption can be written as

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$1

with reverse-time dynamics

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$2

and a deterministic probability flow ODE

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$3

These SDE/ODE views underlie many fast samplers and also provide a common language for score-based, DDPM-style, and bridge-based models (Cao et al., 2022).

A separate theoretical line casts reverse diffusion as stochastic optimal control. “Generative Diffusion From An Action Principle” derives score matching from an action principle, interprets the score as a force field, and shows that reverse diffusion is the least-cost control that steers one distribution to another. In that formulation, score matching is not merely a heuristic denoising loss but the variational condition associated with minimizing a pathwise KL divergence relative to a reference diffusion process (Premkumar, 2023).

3. Model families and structured extensions

The contemporary GDM literature includes several closely related families. A review focused on wireless systems organizes six representative models: DDPM, score-based generative models, SDE/ODE formulations, DDIM, conditional diffusion models, and latent diffusion models (Fan et al., 22 Jul 2025).

Family or variant	Defining mechanism	Characteristic role
DDPM	Fixed Gaussian forward chain and learned reverse denoising	Foundational discrete-time formulation
SGM	Direct estimation of $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$4	Score-field view of generation
SDE/ODE	Continuous-time stochastic or deterministic dynamics	Unifying solver-based formulation
DDIM	Non-Markovian shortcut preserving forward marginals	Faster deterministic or lightly stochastic sampling
CDM	Denoising conditioned on auxiliary information	Task-aware controllable generation
LDM	Diffusion in compressed latent space	Reduced compute and memory cost

Conditioning is central to modern GDMs. The survey literature describes conditional diffusion probabilistic models $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$5 in which $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$6 may be a class label, text prompt, image, segmentation map, depth map, skeleton, or other side information. Guidance is often implemented either by classifier gradients or by classifier-free guidance, with the latter combining conditional and unconditional predictions as

$$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$7

This mechanism has become standard in large text-to-image systems (Cao et al., 2022).

Architecturally, the denoising backbone is often a U-Net with timestep embeddings, self-attention, and, in many practical systems, latent-space operation rather than pixel-space operation (Torre, 2023). The handbook emphasizes that many apparent differences across implementations reduce to output parameterization—predicting $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$8, $$p_\theta(x_{t-1}\mid x_t)=\mathcal{N}\!\left(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)\right),$$9, a score, or a velocity $x_T\sim\mathcal N(0,I)$0—plus differences in scheduler and sampler choice (Ding et al., 2024).

Several extensions explicitly modify the structure of the forward process. The Groupwise Diffusion Model divides data coordinates into groups and diffuses one group at one time interval. With suitable grouping and order, it generalizes autoregressive models and cascaded diffusion models, and in its frequency-domain extension yields a hierarchical latent representation in which low-frequency groups encode coarse semantics and later groups encode finer detail (Lee et al., 2023). A different extension, the renormalization group-based diffusion model, replaces isotropic white-noise corruption with inverse renormalization-group flows that remove fine-scale information first and reconstruct samples coarse-to-fine. That work reports improved sample quality and/or sampling speed by an order of magnitude while tying the schedule to a regulator rather than heuristic tuning (Masuki et al., 15 Jan 2025).

4. Domains of use and functional roles

The application surface of GDMs is unusually broad. The major survey literature covers image generation, image editing, video generation, 3D generation, speech and audio, text generation, time series, biology and chemistry, healthcare, and graphs (Cao et al., 2022). In those domains, diffusion models function both as unconditional generators and as highly structured conditional models for inpainting, super-resolution, restoration, multimodal translation, molecular conformation generation, and biomedical reconstruction (Cao et al., 2022).

A recurring misconception is that GDMs are primarily image generators. The applied literature instead treats them as a general mechanism for learning complex conditional distributions over signals, decisions, or structured objects. In network optimization, a tutorial paper explicitly positions diffusion models as optimization engines that iteratively refine candidate solutions conditioned on network state rather than as one-shot content generators (Du et al., 2023). A later study formalizes this view by learning a conditional solution distribution $x_T\sim\mathcal N(0,I)$1 for network optimization and argues that repeated sampling from a learned high-quality solution distribution is more robust to prediction error than direct discriminative regression to a single optimal point (Liang et al., 2024).

This broader role is visible in domain-specific systems. In dense Wi-Fi control, D3PG replaces the deterministic DDPG actor with a conditioned diffusion model and jointly adjusts the contention window and aggregation frame length for each station (Liu et al., 2024). In seismic inversion, DiffusionVel combines a seismic-data conditional GDM, a well-log conditional GDM, an unconditional geology-oriented GDM, and a low-pass background model so that generated velocity fields can be constrained simultaneously by seismic observations, geological priors, well data, and background trends (Zhang et al., 2024). In training-free semantic communication, a pretrained Stable Diffusion model is repurposed as the encoder and decoder by using DDIM inversion at the transmitter and DDIM sampling at the receiver, with a two-stage forward diffusion process split between transmitter and receiver to improve robustness against channel noise (Tang et al., 2 May 2025).

Wireless systems literature has further elevated GDMs to cross-layer infrastructure components. A dedicated review organizes GDM-enabled wireless networking into a sensing layer, transmission layer, application layer, and security plane, assigning to diffusion models roles such as channel estimation, channel generation, radio map construction, semantic denoising, multimodal transmission, resource allocation, and privacy or anomaly defense (Fan et al., 22 Jul 2025). This suggests a shift from viewing GDMs as modality-specific generators toward viewing them as reusable probabilistic operators for denoising, synthesis, augmentation, and sequential decision support.

5. Sampling efficiency, distillation, and implementation practice

The principal practical weakness of vanilla diffusion is slow sampling. Reverse diffusion usually needs many function evaluations, and the survey literature treats sampling acceleration as a major research axis. Methods cited in that survey include DDIM, PNDM, DEIS, DPM-Solver, DPM-Solver++, EDM, gDDIM, restart sampling, progressive distillation, consistency models, and trajectory-straightening approaches such as Rectified Flow and stochastic interpolants (Cao et al., 2022).

The practical handbook systematizes these developments by treating diffusion probabilistic models, score-based models, consistency models, rectified flow, and related methods under a unified notation intended to bridge the “paper-to-code” gap (Ding et al., 2024). In that treatment, most families are variations on the same ingredients: a noising law, a denoiser or vector field, a timestep parameterization, and a numerical procedure for traversing the learned trajectory. Distillation and post-training are therefore central. The handbook discusses progressive distillation, consistency distillation, distribution matching distillation, adversarial diffusion distillation, reward-based fine-tuning, and related methods as mechanisms for converting slow many-step samplers into few-step or one-step samplers while preserving quality or alignment (Ding et al., 2024).

Efficiency improvements can also come from changing the forward process itself. DDIM reduces the number of reverse steps by using non-Markovian shortcuts that preserve the same forward marginals (Fan et al., 22 Jul 2025). Latent diffusion reduces cost by operating in a compressed latent space rather than raw signal space (Fan et al., 22 Jul 2025). RG-based diffusion uses projection layers to remove Fourier modes that have already been eliminated by coarse-graining, which the authors identify as a reason for both sample-quality improvements and faster sampling (Masuki et al., 15 Jan 2025).

Implementation practice remains sensitive to design details. The introductory Spanish review stresses that performance depends heavily on U-Net design, timestep representation, noise schedule, and objective weighting, and that variance learning can be difficult and unstable (Torre, 2023). The survey adds that ODE samplers are often more stable, while SDE samplers can recover quality better by injecting noise that corrects accumulated discretization error (Cao et al., 2022). A practical implication is that “diffusion model” names a family of related training and sampling systems rather than a single canonical algorithm.

6. Limitations, failure modes, and research frontiers

The literature is explicit that GDMs are not universally reliable. The most obvious limitation is computational: iterative sampling is expensive, and both training and deployment can be difficult in high-dimensional or resource-constrained settings (Cao et al., 2022). The survey literature also notes that conditional generation often depends on guidance heuristics or external models that can be brittle or costly; discrete and non-Euclidean domains require special adaptation; and, in text generation, diffusion still lags autoregressive LLMs in semantic coherence (Cao et al., 2022).

A more specific controversy concerns what diffusion objectives actually optimize. In graph generation and graph sparsification, “Generative Myopia: Why Diffusion Models Fail at Structure” argues that standard graph diffusion models behave like frequency filters, becoming good at reproducing abundant substructures while ignoring statistically rare but structurally mandatory ones. The paper terms this mismatch “Generative Myopia” and attributes it to “Gradient Starvation,” an optimization effect in which rare structural signals are suppressed regardless of model capacity (Siami, 23 Nov 2025). In the barbell-graph stress test at target density $x_T\sim\mathcal N(0,I)$2, standard diffusion achieves 0% connectivity, while the proposed spectrally weighted method reaches 89.6% connectivity, close to the Spectral Oracle at 88.2% and above random sampling at 46.0%; in the asymmetric chain SBM at density $x_T\sim\mathcal N(0,I)$3, standard diffusion again yields 0.0% connectivity, whereas the weighted method reaches 100.0%, matching the Spectral Oracle exactly (Siami, 23 Nov 2025). The authors’ remedy, Spectrally-Weighted Diffusion, reweights the training objective by effective resistance and is described as having zero inference overhead because the spectral computation is amortized offline (Siami, 23 Nov 2025).

This result sharpens a general caution already present in the broader diffusion literature: a model can be statistically accurate under likelihood or denoising metrics yet still be structurally wrong for downstream objectives governed by connectivity, constraints, or worst-case topology. A plausible implication is that diffusion objectives may need domain-specific priors when rare global structure matters more than frequent local patterning.

Current research directions therefore emphasize both breadth and correction. The survey literature highlights better controllable distribution-based generation, tighter integration with LLMs for multimodal systems, and deeper integration with semi-supervised learning and reinforcement learning (Cao et al., 2022). Wireless-network reviews emphasize lightweight GDMs, edge intelligence, federated learning, differential privacy, and stronger security against backdoor and privacy attacks (Fan et al., 22 Jul 2025). Across these lines of work, the central theme is consistent: GDMs are a flexible probabilistic framework rather than a finished recipe, and their continued expansion depends on making them faster, more controllable, more structurally faithful, and more secure.