Classifier-Free Guidance in Diffusion Models

Updated 22 December 2025

Classifier-free guidance is an inference technique that blends conditional and unconditional predictions to control the trade-off between sample fidelity and diversity.
It utilizes a convex combination parameterized by a guidance scale to produce sharper, semantically aligned samples, though it may introduce biases in sample variability.
The Gibbs-like iterative approach refines samples through noise injection and guided denoising, effectively mitigating mode collapse and preserving diversity.

Classifier-free guidance (CFG) is an inference-time procedure for steering conditional generative models, especially diffusion models, by linearly combining predictions from both a conditional and an unconditional model. Introduced as a practical alternative to classifier-based guidance, CFG enables fine-grained control over the trade-off between sample fidelity and diversity across modalities such as images, audio, and language. Despite its empirical success, ongoing research has revealed subtle theoretical limitations and motivated numerous extensions targeting both the quality/diversity trade-off and the geometric and statistical properties of guided samples.

1. Mathematical Foundations and Theoretical Formulation

CFG operates by interpolating between a generative model’s conditional prediction (given a context $c$ ) and its unconditional prediction (no context), usually by a convex or affine combination parameterized by a guidance scale $w$ . In diffusion models, this is applied to either the network's predicted noise or score function at each denoising step. The classic formula is: $\hat{\epsilon}_\theta(x_t, c) = \epsilon_\theta(x_t, \emptyset) + w\bigl[\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t, \emptyset)\bigr]$ where $\epsilon_\theta(x_t, \emptyset)$ is the unconditional prediction and $\epsilon_\theta(x_t, c)$ is the conditional prediction. In the continuous score-based formalism, the guidance-modified score is: $\widetilde s(x, t \mid c) = (1-\gamma)\,\nabla_x \log p_t(x) + \gamma\,\nabla_x \log p_t(x|c),\quad \gamma > 0$ CFG typically uses $w, \gamma > 1$ to over-emphasize the conditional component, yielding sharper samples more aligned with the conditioning signal but at the expense of reduced diversity (Ho et al., 2022, Bradley et al., 16 Aug 2024, Pavasovic et al., 11 Feb 2025).

2. Theory: Optimality, Bias, and Missing Repulsive Terms

CFG as widely implemented does not generally correspond to sampling from the exact desired conditional distribution, especially for $w > 1$ . In particular, recent work demonstrates that the population of samples from the standard CFG reverse process does not match the power-tilted conditional target

$p_0^{c;w}(x_0) \propto \pi_0(c|x_0)^w \,p_0(x_0)$

unless a specific correction is included. The precise score for this distribution involves not only the CFG term but also a repulsive gradient of the order- $w$ Rényi divergence $R_w$ between the conditional and unconditional posteriors: $\nabla_x \log \pi_\sigma^{c;w}(x) = \nabla_x \log \hat p_\sigma^{c;w}(x) + (w-1)\nabla_x R_w\bigl(p_{0|x,c}\|p_{0|x}\bigr)$ Here, $R_w(\mu \|\nu) = \frac{1}{w-1}\log\int (d\mu/d\nu)^w d\nu$ . This extra term acts as a repulsion that mitigates excessive concentration and mode collapse of standard CFG, especially at moderate noise levels. As noise vanishes ( $\sigma\to0$ ), the contribution of $\nabla_x R_w$ scales as $O(\sigma^2)$ and becomes negligible (Moufad et al., 27 May 2025).

Thus, standard CFG (without the Rényi correction) leads to overly sharp, under-diverse samples in medium- to high-noise diffusion steps and is not a theoretically faithful approximation to the desired distribution.

3. Algorithm: Classifier-Free Gibbs-Like Guidance

To address the theoretical deficiency, a Gibbs-like iterative noising/denoising approach is proposed. The procedure starts with a sample from a (possibly lightly-guided) DDM, then repeatedly:

Adds small noise to the sample: $X_{\sigma_*}^r = X_0^{r-1} + \sigma_* Z$ , with $Z \sim N(0, I)$ and $\sigma_* \ll 1$ .
Applies CFG-guided denoising with a stronger guidance scale from $\sigma_*$ down to $0$.

This two-step Markov kernel preserves sample diversity via the injected noise (counteracting over-concentration) while harnessing strong guidance for semantic alignment. In the idealized setting (with truly consistent denoisers), this process converges to the power-tilted target posterior $\pi_0^{c;w}(x_0)$ . In practice, using learned denoisers with the CFG update is sufficiently accurate for small $\sigma_*$ (Moufad et al., 27 May 2025).

Simplified pseudocode:

for r in range(R):
    x_noise = x_prev + sigma_star * torch.randn_like(x_prev)
    x_prev = run_cfg_denoising(x_noise, w, start_sigma=sigma_star, end_sigma=0)
return x_prev

Here,

R

is the number of refinement rounds, and

w

is chosen larger than 1. The approach reintroduces stochasticity at each step, counterbalancing the mode-collapsing bias.

4. Empirical Results: Quality, Diversity, and Modalities

Empirical evaluation demonstrates that classifier-free Gibbs-like guidance improves both sample quality and diversity over standard and interval CFG schemes across multiple domains:

ImageNet-512 (EDM-S, EDM-XXL):

Standard CFG achieves lower FID at optimal $w$ but is outperformed in aggregate metrics (FID, FD $_{\text{DINOv2}}$ , precision, recall, density, coverage) by the Gibbs-like approach.
For EDM-XXL, Gibbs-like achieves FID 1.48, FD $_{\text{DINOv2}}$ 42.87, precision 0.70, recall 0.68—beating static and interval CFG on all axes.

AudioCaps $\rightarrow$ AudioLDM 2-Full-Large:

Gibbs-like matches or betters the best FAD, KL, and Inception Score (IS) of CFG at similar or better guidance strengths, across several refinement schedules.

Ablations:

Two refinement rounds ( $R=2$ ) yield the best diversity-quality trade-off; more rounds reduce per-round step count and may degrade results.
Increasing $\sigma_*$ enhances recall (diversity) at the expense of quality; smaller $\sigma_*$ makes behavior closer to classic CFG.

5. Impact and Interpretations

The absence of the repulsive Rényi-divergence correction in standard CFG leads to excessive semantic sharpening and loss of sample variability, especially at moderate guidance strengths. The Gibbs-like method, by cycling noise injection and guided denoising, empirically restores the balance between diversity and fidelity, closely approximating the desired power-tilted conditional law $p_0^{c;w}$ .

This approach also provides a practical, architecture-agnostic mechanism for improving sample variety in conditional diffusion models regardless of modality, and does not require explicit computation of the Rényi gradient term.

6. Limitations, Open Directions, and Future Work

Although the added Rényi divergence contribution is theoretically critical for exact conditional sampling, it diminishes in the final stages of denoising. Thus, the bias introduced by classic CFG is mainly relevant in the noisier, earlier (or middle) steps. The effectiveness of the Gibbs-like sampler depends on accurately approximating the ideal denoisers and careful tuning of parameters such as $\sigma_*$ and the number of refinement rounds $R$ .

Open avenues include:

Incorporating explicit or learned estimators for the missing repulsive term, potentially via energy-based modeling.
Analysis of convergence rates and practical sample efficiency.
Extension to highly multimodal or non-Euclidean targets, where divergence corrections may play a larger role.

In summary, CFG is broadly effective but theoretically incomplete; the addition of stochastic refinement cycles approximates the missing statistical corrections and yields improved generative performance in both image and audio conditional diffusion models (Moufad et al., 27 May 2025).