Conditional Diffusion Probabilistic Models

• Conditional Diffusion Probabilistic Models are generative methods that reverse a noising process using neural networks conditioned on auxiliary signals.
• They employ techniques such as cross-attention, classifier-free guidance, and adaptive normalization to integrate diverse contextual information.
• Empirical results show state-of-the-art performance in tasks like image synthesis, time series forecasting, and graph learning with competitive metrics.

Conditional diffusion probabilistic models (CDPMs) are a class of generative models extending denoising diffusion probabilistic models (DDPMs) and score-based diffusion frameworks by incorporating conditioning mechanisms that guide sample generation toward distributions specified by external variables, side-information, or observed data. These models provide a principled approach to modeling complex conditional distributions, support expressive conditioning for diverse tasks, and admit both rigorous theoretical analysis and competitive empirical performance across imaging, time series, graph data, physics-driven systems, and beyond.

1. Foundational Principles and Model Formulation

CDPMs define a generative process by reversing a forward noising procedure (Markov chain or stochastic differential equation, SDE) that incrementally corrupts structured data into pure noise. The reverse process is parameterized by neural networks and explicitly incorporates a conditioning variable, $y$ or $c$, to reflect desired dependencies.

Forward Process:

For data $x_0$ and $T$ discrete steps, the forward chain is

$$q(x_t | x_{t-1}) = \mathcal{N}\left(x_t ; \sqrt{1-\beta_t} x_{t-1}, \beta_t I\right)$$

with marginal

$$q(x_t | x_0) = \mathcal{N}\left(x_t ; \sqrt{\bar{\alpha}_t} x_0, (1-\bar{\alpha}_t) I\right),\quad \bar{\alpha}_t = \prod_{i=1}^t (1 - \beta_i)$$

Reverse Process (Conditional):

The reverse-time chain is learned as

$$p_\theta(x_{t-1} | x_t, y) = \mathcal{N}\left(x_{t-1} ; \mu_\theta(x_t, t, y), \Sigma_\theta(x_t, t, y)\right)$$

Common parameterizations (as in [Ho et al. 2020]) yield

$$\mu_\theta(x_t, t, y) = \frac{1}{\sqrt{\alpha_t}} \left( x_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}} \epsilon_\theta(x_t, t, y) \right)$$

where $\epsilon_\theta$ is a neural network estimating the noise added at time $t$.

Conditioning is injected by making $\epsilon_\theta$ or the mean/variance parameters functions of $y$. The form and timing of this conditioning are crucial distinguishing features across CDPM variants (Tashiro et al., 2021, Peng et al., 2022, Niu et al., 2023, Dong et al., 29 Jun 2025).

2. Conditioning Mechanisms and Architectural Variants

CDPMs deploy a spectrum of conditioning strategies, spanning explicit variable injection, cross-attention, classifier-free guidance, and forward-process trajectory modification:

• Input Concatenation / Early Fusion: The conditioning variable $y$ (e.g., an image, label, history, or feature vector) is concatenated with the current state $x_t$ at each denoising step and supplied as input to the neural network (Corley et al., 2023, Letafati et al., 2023).
• Cross-Attention and Adaptive Normalization: For high-dimensional or multi-modal $y$, models use cross-attention modules or adaptive normalization (AdaLN, FiLM) in the U-Net or graph network layers, enabling flexible context integration (Huang et al., 22 Mar 2025, Chen et al., 8 May 2025, Peng et al., 2022).
• Classifier-Free Guidance: Two networks are trained—one with and one without conditioning. Linear combination at inference boosts conditional fidelity (Fu et al., 2024, Huang et al., 22 Mar 2025).
• Shifted/Guided Forward Processes: Some models alter the forward noising path as a function of $y$, allocating distinct diffusion trajectories for each condition [(Zhang et al., 2023), "ShiftDDPMs"]. Others implement guidance via gradient-based drift corrections in the SDE for hard-constrained or rare-event sampling (Guo et al., 5 Feb 2026).
• Physics-Based Conditioning and Hybrid Priors: Conditioning variables can arise from explicit physics solvers (e.g., ODEs on graphs for air quality), which are fused with data-driven denoising residuals (Dong et al., 29 Jun 2025).

Architectural designs reflect task demands, including U-Nets, Transformer blocks, Graph Neural Networks, and spatio-temporal networks. The choice of fusion—at which layers and scales conditioning is applied—directly affects the model's ability to leverage structure in $y$.

3. Training Objectives, Theoretical Guarantees, and Information-Theoretic Perspectives

CDPMs are typically trained by minimization of a conditional denoising-score matching objective or variational ELBO tailored to the inclusion of $y$:

$$L(\theta) = \mathbb{E}_{x_0, t, \epsilon, y} \| \epsilon - \epsilon_\theta(x_t, t, y) \|^2$$

where $$x_t = \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1 - \bar{\alpha}_t} \epsilon$$. Variants exist for problems such as imputation (conditioning on observed features), single-view estimation, sequence-to-sequence tasks, and superresolution (Tashiro et al., 2021, Niu et al., 2023, Corley et al., 2023).

Recent work gives the first statistical sample-complexity bounds for CDPMs, showing minimax-optimal convergence rates under total variation and Wasserstein metrics (Tang et al., 2024, Fu et al., 2024). The error rates depend on the smoothness of $p(x|y)$, intrinsic data and covariate dimension, and cover both Euclidean and (joint) manifold structures. The conditional denoising loss is shown to maximize a lower bound on the conditional mutual information, tying latent representation informativity to predictive sample quality (Chen et al., 8 May 2025).

4. Empirical Performance and Applications

CDPMs have demonstrated state-of-the-art empirical results in a range of benchmarks and domains:

• Image-to-image Translation and Restoration: CDPMs for superresolution, deraining, and colorization consistently outperform GANs and prior diffusion models, showing superior FID, LPIPS, PSNR/SSIM, and perceptual realism (Niu et al., 2023, Mei et al., 2022, Peng et al., 2022).
• Time Series Forecasting and Imputation: Conditional and channel-aware architectures achieve best-in-class MSE/CRPS on electricity, traffic, and environmental datasets, offering interpretable uncertainty estimates and fine-grained control (Tashiro et al., 2021, 2410.02168, Dong et al., 29 Jun 2025).
• Physics and Domain Knowledge Fusion: Explicitly incorporating physical solvers or expert priors as conditioning inputs yields superior scenario and uncertainty prediction in power load and air quality (Dong et al., 29 Jun 2025, Huang et al., 22 Mar 2025).
• Graph and Representation Learning: Graph-encoded conditions in diffusion enable mutual-information–driven embeddings that set new accuracy records in node/graph classification (Chen et al., 8 May 2025).
• Inverse Problems and Medical Imaging: Conditional architectures generate high-resolution digital surface models from single-view images, reconstruct realistic brain MRIs, and robustly impute missing clinical data (Peng et al., 2022, Corley et al., 2023, Mei et al., 2022).

Representative results are summarized below:

Task	Key Metric	CDPM Best Result	Baseline	Paper
Single-image SR (Set5, ×4)	LPIPS ↓	0.0564 (cDPMSR+SwinIR)	0.0596 (ESRGAN)	(Niu et al., 2023)
Air Quality Prediction	CRPS (Beijing)	0.3317	0.3649 (DiffSTG)	(Dong et al., 29 Jun 2025)
Graph Node Classification	Acc. (Computers)	91.3%	89.9% (BGRL)	(Chen et al., 8 May 2025)
Speech Enhancement	PESQ (CHiME-4)	1.66 (CDiffuSE Large)	1.38 (Demucs)	(Lu et al., 2022)
Prob. Load Forecasting	MAPE (%)	7.19 (ECDM)	10–16 (others)	(Huang et al., 22 Mar 2025)

Ablation studies reveal that strong conditioning pathways are essential; removing the conditional input or cross-attention typically leads to substantial degradation in both accuracy and uncertainty calibration.

5. Specialized Methodologies: Hard Constraints, Training-Free Conditionality, and Informative Guidance

Advanced CDPM variants address regimes where standard soft/likelihood-based conditioning is insufficient:

• Hard Constraint Conditioning:

Martingale and Doob's $h$-transform–based conditioning implement explicit drift corrections in the reverse SDE, enforcing prescribed rare events or safety-critical constraints exactly, with non-asymptotic error bounds in total variation and Wasserstein distances. Off-policy learning algorithms are used to estimate the conditioning function from trajectories under a pre-trained diffusion (Guo et al., 5 Feb 2026).

• Training-Free Conditional Generation:

Posterior sampling and flow matching approaches ("FMPS") enable conditional generation without explicit retraining, by rewriting the SDE velocity field in terms of the score and differentiable condition–sample distances. These approaches extend flexible posterior guidance to flow-based models (Song et al., 2024).

• Mutual Information and Contrastive Guidance:

Combining denoising and auxiliary contrastive losses can explicitly maximize predictive mutual information, enhancing out-of-distribution robustness and generalization in forecasting and representation learning (2410.02168, Chen et al., 8 May 2025).

6. Theoretical Limitations, Pitfalls, and Model Selection

Recent analyses show that maximizing the (conditional) likelihood alone can be misleading: for example, in text-to-speech, the diffusion log-likelihood is insensitive to the text prompt; in text-to-image, it weakly correlates with semantic alignment between prompt and image (Cross et al., 2024). Model selection and evaluation must be augmented with explicit conditionality metrics (CLIP score, WER, task accuracy) and possibly mutual information or cross-entropy objectives.

Computational bottlenecks remain in iterative sampling, though strides have been made via acceleration, residual-fusion, deterministic samplers (e.g., DDIM), and learned approximation to drift corrections.

7. Outlook and Research Frontiers

CDPMs constitute a rigorous, versatile family for conditional generative modeling, unifying expressive conditioning, strong uncertainty quantification, and theoretical support for distribution estimation:

• Statistical and Manifold Adaptivity: CDPMs provably attain minimax-optimal rates in TV and Wasserstein distances and adapt to low-dimensional manifold structure in both covariate and target domains (Tang et al., 2024).
• Hybrid and Multi-Speed Diffusion: Ongoing developments in multi-speed and physics-informed models promise further advances in accuracy, speed, and sample diversity.
• Evaluation Protocols: Closing the gap between high generative likelihood and actual conditional fidelity is an open challenge, motivating the integration of mutual information objectives, adversarial or contrastive critics, and domain-specific quality metrics.
• Flexible, Modular Inference: Training-free adaptation and flow-matching guidance extend conditionality to a broader range of pre-trained generative models, facilitating wider deployment and transferability (Song et al., 2024).

In summary, conditional diffusion probabilistic models deliver principled, statistically grounded, and empirically competitive solutions to conditional generation, probabilistic inference, and representation learning tasks across scientific and real-world domains (Tashiro et al., 2021, Zhang et al., 2023, Niu et al., 2023, Chen et al., 8 May 2025, Song et al., 2024, Dong et al., 29 Jun 2025, Huang et al., 22 Mar 2025, 2410.02168, Mei et al., 2022, Corley et al., 2023, Fu et al., 2024, Tang et al., 2024, Guo et al., 5 Feb 2026, Peng et al., 2022, Peng et al., 2022, Cross et al., 2024).