DPM-Solver: Efficient Diffusion Sampling

• DPM-Solver is a family of training-free, high-order ODE solvers that leverage the semilinear structure of diffusion models to drastically reduce function evaluations.
• Its variants, such as DPM-Solver++ and DPM-Solver-v3, achieve high-quality sampling with as few as 10 evaluations while enhancing stability and accuracy.
• Recent advancements incorporate quantum adaptations and exact inversion techniques, expanding applications to efficient image editing and scalable generative modeling.

DPM-Solver is a family of training-free high-order ODE solvers designed for accelerated sampling from diffusion probabilistic models (DPMs). DPMs, which parameterize the forward noising process and learn its reversal, have established state-of-the-art results in image and audio generation but have historically required hundreds to thousands of function evaluations (NFE) for high-fidelity sampling. DPM-Solver leverages the semilinear structure of the diffusion ODE, integrating the linear component in closed form and using Taylor-expansion-based quadrature for the learned neural network term, thereby enabling high-quality samples with as few as 10 function evaluations, without retraining or altering the pretrained DPM (Lu et al., 2022). The DPM-Solver methodology has evolved through several iterations—DPM-Solver, DPM-Solver++, and DPM-Solver-v3—each introducing improved stability, accuracy, and adaptability for guided, multistep, and low-NFE sampling. Further, DPM-Solver algorithms have inspired quantum computing adaptations capable of polynomial or exponential savings in the amplitude-encoded system dimension (Wang et al., 20 Feb 2025).

1. Mathematical Foundations and Solver Construction

DPM-Solver is built around the continuous-time ODE induced by the reverse (denoising) process in DPMs. For standard variance-preserving schedules, the reverse ODE can be written as

$$\frac{dx_t}{dt} = f(t) x_t + \frac{g^2(t)}{2\sigma_t} \varepsilon_\theta(x_t, t)$$

where $f(t)$, $g(t)$, $\sigma_t$, and $\alpha_t$ are schedule functions, and $\varepsilon_\theta$ is the noise-predicting neural network. Changing variables to the log-SNR domain, $\lambda_t = \log(\alpha_t/\sigma_t)$, the solution for $x_t$ at time $t$ from $x_s$ at time $f(t)$0 reduces to

$f(t)$1

DPM-Solver-$f(t)$2, for order $f(t)$3, is constructed by Taylor-expanding $f(t)$4 up to order $f(t)$5 around $f(t)$6, yielding a high-order update that integrates the leading linear terms and approximates the neural term using only $f(t)$7 network evaluations per step (Lu et al., 2022, Wang et al., 20 Feb 2025).

2. DPM-Solver Family: Algorithmic Advances

The original DPM-Solver introduced first-, second-, and third-order exponential integrator schemes, exhibiting provable $f(t)$8th-order convergence and enabling drastic reductions in NFE compared to black-box ODE solvers and prior training-free samplers. The update rule for DPM-Solver-2, for example, involves a midpoint evaluation: $f(t)$9 where $g(t)$0 is an intermediate step.

DPM-Solver++ extends this by reformulating the ODE in a data-prediction parameterization and introducing both single-step and multistep second-order solvers. The multistep variant (2M),

$g(t)$1

improves stability under large guidance scales and allows full use of the model evaluation budget with fewer artifacts at low NFE (Lu et al., 2022).

DPM-Solver-v3 further generalizes the approach by introducing an optimal parameterization of the ODE that minimizes first-order discretization error. This is achieved by computing empirical model statistics (EMS)—coefficients $g(t)$2 estimated from the pretrained model—which parameterize and flatten the nonlinear term, enhancing accuracy and stability, especially at very small NFE or under large classifier-free guidance (Zheng et al., 2023). DPM-Solver-v3 incorporates arbitrary-order multistep predictor-corrector schemes and several heuristics (pseudo-order correctors, half-corrector application) for further robustness.

3. Sampling Frameworks and Solver Scheduling

All high-order exponential-integral-based solvers, including DPM-Solver, DPM-Solver++, and related methods (e.g., UniPC, DEIS), can be unified under a modular meta-algorithm within a Unified Sampling Framework (USF). USF decomposes each ODE solve step into independent components: time schedule, prediction type (noise or data), order, derivative estimation, and corrector usage. This exposes a combinatorial space of solver schedules.

Recent work proposes S³, a predictor-guided search method for automatically selecting per-step solver strategies under NFE constraints. By leveraging a learned performance predictor $g(t)$3 over schedules $g(t)$4, S³ finds customized schedules that outperform fixed-strategy DPM-Solver variants, especially for tight NFE budgets (e.g., 5–10 steps), yielding significant improvements in FID and convergence speed across multiple datasets (Liu et al., 2023).

Method/Schedule	5 NFE FID (CIFAR-10)	10 NFE FID (ImageNet-256, s=8.0)
DPM-Solver++(2M)	33.8	13.66
S³ (USF + search)	6.86	9.06

This scheduling flexibility demonstrates that adaptively changing solver parameters per timestep can achieve close to minimal truncation error for a fixed compute budget.

4. Exact Inversion and Editing Applications

Standard DPM-Solver sampling is explicit and non-invertible, challenging direct reconstruction of initial noise vectors from generated images, especially under strong classifier-free guidance. Recent advances employ implicit optimization—gradient descent or forward-step methods—to exactly invert each DPM-Solver denoising step, yielding orders-of-magnitude reduction in reconstruction error compared to naïve or fixed-point inversion methods. These techniques robustly handle guided sampling (e.g., guidance $g(t)$5) and preserve background content during prompt-based image editing workflows by accurately recovering and reusing initial noise latents (Hong et al., 2023).

A typical inversion strategy for DPM-Solver-2M is as follows: for each step, solve for $g(t)$6 by treating the forward update as an implicit equation in $g(t)$7 and minimize $g(t)$8 until convergence. Experimental results show normalized MSE for recovered noise drops from $g(t)$9 (naïve) to $\sigma_t$0 (exact inversion).

5. Quantum Algorithms for DPM-Solver

Quantum adaptations of DPM-Solver (DPM-Solver-$\sigma_t$1) and related high-order diffusion-ODE solvers (UniPC) have been proposed using Carleman linearization and quantum linear system solvers (QLSS). The primary innovation is the embedding of the nonlinear Taylor expansion into a system of linear difference equations via polynomial truncation and auxiliary variables, leading to a quantum Carleman matrix formulation. Quantum block-encoding and linear combination of Hamiltonian simulation (LCHS) enable the preparation of amplitude-encoded solutions such that the gate complexity scales only polylogarithmically in the effective system dimension $\sigma_t$2, rather than linearly as in classical sampling. This framework anticipates exponential savings for generative modeling in high-dimensional settings, provided fault-tolerant quantum hardware (Wang et al., 20 Feb 2025).

A plausible implication is that end-to-end quantum pipelines for large-scale image and audio generation may soon become practical, with resource overheads dominated by quantum state tomography and block-encoding rather than by the core ODE solve.

6. Empirical Performance and Practical Limitations

DPM-Solver and its variants have demonstrated state-of-the-art sample efficiency, achieving, for instance, FID = 2.87 on CIFAR-10 using only 20 function evaluations with DPM-Solver (Lu et al., 2022), FID = 8.40 at 20 NFE with DPM-Solver++ (with classifier guidance $\sigma_t$3) (Lu et al., 2022), and further 15–30% speedup in DPM-Solver-v3 versus preceding training-free methods (Zheng et al., 2023). S³ scheduling improves 5-NFE FID on CIFAR-10 from 33.8 (DPM-Solver++(2M)) to 6.86 (Liu et al., 2023).

Limitations include reduced stability under large guidance scales (requiring dynamic thresholding or smaller per-step intervals), increased complexity of inversion (causing 10–100× more network evaluations compared to naïve inversion), and code overhead for partitioning the solver into quantum subroutines. The quantum approaches remain theoretical until large-scale fault-tolerant QPUs become available. For inversion, background preservation and watermark extraction improve substantially with implicit corrector methods, but at the cost of significant runtime increase (Hong et al., 2023).

7. Significance and Outlook

DPM-Solver provides a mathematically rigorous, efficient, and flexible framework for solving diffusion ODEs in generative modeling. Its modularity enables integration of advanced scheduling and inversion strategies, and its design has made it a standard backbone for rapid DPM sampling, prompt-to-prompt editing, latent inversion, and, recently, quantum generative algorithms. Future directions involve further generalization of per-step solver adaptation, lowering quantum and classical resource overheads, and extending inversion and editing methods for more general conditional and stochastic DPM sampling (Liu et al., 2023, Wang et al., 20 Feb 2025).