Predictor-Corrector Samplers in DPMs

• Predictor-corrector samplers are advanced numerical algorithms that alternate structured predictor and corrector steps to enable efficient, high-fidelity sample generation in diffusion models.
• They leverage techniques like multi-step finite differences and Lagrange interpolation to enhance integration order and reduce the number of function evaluations.
• Recent innovations, including dynamic compensation and classifier-free guidance integration, improve sample quality while mitigating misalignment issues at high guidance scales.

Predictor-corrector samplers are advanced numerical schemes for accelerating generation in diffusion probabilistic models (DPMs), achieving high sample quality with minimal function evaluations (NFE). These algorithms alternate structured predictor and corrector updates, which can be formulated from deterministic ODE solvers and/or stochastic correctors such as Langevin dynamics. Recent developments in predictor‐corrector diffusion sampling unify theoretical guarantees, introduce adaptive error-correction strategies, and clarify the foundations of widely used guidance paradigms.

1. Theoretical Foundations of Predictor-Corrector Sampling

Diffusion probabilistic models (DPMs) proceed from a forward noising process

$$q_{t|0}(x_t|x_0) = \mathcal N(x_t | \alpha_t x_0, \sigma_t^2 I)$$

with learned denoising network $\epsilon_\theta(x_t, t)$. Sampling corresponds to integrating the probability‐flow ODE

$$\frac{dx_t}{dt} = f(t) x_t - \tfrac{1}{2} g^2(t) \nabla_x \log q_t(x_t), \qquad \nabla_x \log q_t(x_t) = -\frac{1}{\sigma_t} \epsilon_\theta(x_t, t)$$

Efficient sampling requires discretization of this ODE in a minimal number of steps. Predictor-corrector (PC) frameworks formalize a layered update scheme: each time step is split into a predictor phase and a corrector phase, where the latter improves the local accuracy or sharpens the distributional properties of the sample trajectory.

A canonical instantiation is the UniPC framework, which provides multi-order accurate predictors (UniP) and correctors (UniC) through explicit exponential-integrator or Taylor-based constructions. For a $p$th-order scheme, the buffer $Q$ stores the latest $p$ noise predictions. The predictor step leverages multi-step finite difference or Lagrange interpolation; the corrector step, without any additional network evaluation, raises the integration order by including the just-predicted model output (Zhao et al., 2023).

2. Classification-Free Guidance and Predictor-Corrector Interpretation

Classifier-free guidance (CFG) is viewed as a predictor-corrector mechanism in recent theoretical work (Bradley et al., 2024). For conditional DPMs, CFG applies a learned mixture of conditional and unconditional scores,

$$\bar\epsilon_\theta(x_t, t, c) = s \epsilon_\theta(x_t, t, c) + (1-s) \epsilon_\theta(x_t, t, \varnothing), \quad s>1$$

Standard CFG sampling alternates a deterministic denoising (predictor: DDIM-type) with a sharpening step (corrector: Langevin dynamics targeting a $\gamma$-powered density $$p_{t,\gamma}(x|c) \propto p_t(x|c)^\gamma p_t(x)^{1-\gamma}$$). In the SDE limit, CFG is exactly decomposed into a predictor (DDIM) for the conditional marginal and a corrector (Langevin) targeting the $\gamma$-powered distribution, with explicit correspondence between guidance weights and corrector step size (Theorem 3.1) (Bradley et al., 2024).

Pseudocode for such predictor-corrector guidance (PCG) alternates deterministic DDIM updates and stochastic Langevin correction, providing a lens to interpret the empirical success of CFG as principled annealed Langevin dynamics.

3. High-Order and Multi-Step Predictor-Corrector Samplers

Modern predictor-corrector DPM samplers (DDIM, DPM-Solver++, DEIS, UniPC) have advanced from single-step Euler–Maruyama and Heun analogues to arbitrary-order multistep schemes. The core formulation for $p$-step UniPC reads:

1. Predictor update:

$$\tilde x_{t_i} = \mathrm{Pred}(\tilde x_{t_{i-1}}^c, Q)$$

2. Corrector update (optional, reuses buffer):

$$\tilde x_{t_i}^c = \mathrm{Corr}(\tilde x_{t_i}, \epsilon_\theta(\tilde x_{t_i}, t_i), Q)$$

3. Buffer is updated with the most recent $\epsilon_\theta$.

Theoretical guarantees include local truncation error $\mathcal O(h^{p+1})$ for predictors and $\mathcal O(h^{p+2})$ for correctors. Multistep variants utilize previous $p$ evaluations for maximal efficiency with minimal NFE. Plug-in correctors can be applied to a wide range of existing solvers, consistently raising their integration order (Zhao et al., 2023).

4. Misalignment and Dynamic Compensation in Corrector Steps

A critical challenge in predictor-corrector sampling under large guidance scales (high $s$ in CFG) is misalignment. The corrector step updates the sample but the buffer $Q$ holds the predicted $\epsilon_\theta$ at the non-corrected point, which can mislead subsequent predictions, particularly worsening sample quality for large CFG scales ($s=7.5$). While re-evaluating the network on the corrected state partially restores alignment, it doubles computation. This systemic misalignment is empirically characterized in UniPC and related frameworks (Zhao et al., 2024).

5. Dynamic Compensation and DC-Solver

DC-Solver introduces dynamic compensation (DC) to address misalignment in the predictor-corrector loop (Zhao et al., 2024). Dynamic compensation constructs a Kth-order Lagrange interpolation for the model output at the corrected state:

$t_i' = \rho_i t_i + (1-\rho_i) t_{i-1}$

$$\hat\epsilon^{\rho_i}(\tilde x_{t_i}^c, t_i) = \sum_{k=0}^K \left[ \prod_{0 \le l \le K, l \neq k} \frac{t_i' - t_{i-l}}{t_{i-k} - t_{i-l}} \right] \epsilon_\theta (\tilde x_{t_{i-k}}, t_{i-k})$$

Optimal compensation ratios $\rho_i^*$ are learned by minimizing the deviation from ground-truth sampling trajectories across several seeds ($N=10$), using an AdamW optimizer over $L=40$ iterations. To generalize across configuration grids (varying NFE, CFG), DC-Solver fits a three-stage cascade polynomial regression (CPR), enabling instant, configuration-aware compensation ratio prediction via least squares (Zhao et al., 2024).

DC-Solver is plug-and-play: dynamic compensation can be integrated into predictor-only samplers to boost performance, not limited to corrector-equipped frameworks.

6. Experimental Performance and Comparative Evaluation

Empirical benchmarks indicate substantive gains for predictor-corrector and DC-enhanced samplers. For unconditional FFHQ (latent DPM, $64\times64$, NFE=5), reported FIDs are:

Method	FID (NFE=5)
DPM-Solver++	27.15
DEIS	32.35
UniPC	18.66
DC-Solver	10.38

For conditional Stable-Diffusion-2.1 ($768\times768$, CFG=7.5, NFE=5):

Method	MSE
DPM-Solver++	0.443
DEIS	0.436
UniPC	0.434
DC-Solver	0.394

These results demonstrate that with only 5 model evaluations, DC-Solver achieves state-of-the-art sample quality and substantially reduces misalignment errors without incurring extra forward passes. The cascading polynomial regression enables robust compensation across sampling regimes, making DC-Solver particularly effective for extremely low-NFE scenarios (Zhao et al., 2024).

7. Conceptual Generalizations and Design Space

PCG formalism embeds CFG and related sampling approaches into a broader class of score-based predictor-corrector methods. The predictor can encompass DDIM, DPM-Solver++, or higher-order schemes; the corrector can be overdamped Langevin, Hamiltonian MCMC, or other stochastic updates. The guidance schedule ($\gamma$) itself may be annealed over time to balance sharpness and coverage.

Bradley & Nakkiran (Bradley et al., 2024) note that DDPM vs DDIM with CFG yield distinct distributions (except when $\gamma=1$), and provide explicit counterexamples. Viewing CFG as alternating denoising and sharpening steps clarifies both its empirical effectiveness and exposes principled avenues for further sampler design—extending predict-correct loops beyond the prevailing single-step recipe.

Overall, predictor-corrector samplers represent a unified numerical stratagem for fast, accurate DPM generation and their recent advances (including dynamic compensation and theory-grounded guidance) establish new standards for efficient, high-fidelity synthesis in both unconditional and guided conditional regimes.