Dual-Solver: Learned ODE Sampler

• Dual-Solver is a learned ODE sampler that uses dual prediction by interpolating between noise, velocity, and data predictions with learnable parameters.
• It generalizes multistep solvers by optimizing the integration domain and residual terms, preserving second-order local accuracy in low-NFE regimes.
• Empirical results on ImageNet and text-to-image generation show significant improvements in FID and CLIP scores with fewer function evaluations.

Dual-Solver is a learned ODE sampler for diffusion and flow-matching generative models that is designed for the low-NFE regime, where sampling quality must be maintained with only a few function evaluations. It generalizes multistep samplers through learnable parameters that continuously interpolate among prediction types, select the integration domain, and adjust residual terms. The method retains a standard predictor-corrector structure while preserving second-order local accuracy, and its parameters are learned with a classification-based objective using a frozen pretrained classifier such as MobileNet or CLIP. In reported ImageNet class-conditional generation with DiT and GM-DiT, and text-to-image generation with SANA and PixArt-$\alpha$, it improves FID and CLIP scores for $3 \le \mathrm{NFE} \le 9$ (Park et al., 4 Mar 2026).

1. Numerical setting and motivation

Dual-Solver is formulated in the standard continuous-time view of diffusion and flow-matching generation. The forward process is written as

$$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$

with common schedules including VP, VE, and OT. Sampling is then interpreted through the probability-flow ODE associated with the diffusion SDE, so acceleration becomes a numerical integration problem rather than a purely stochastic one (Park et al., 4 Mar 2026).

The method is motivated by two discretization effects. First, although noise prediction, data prediction, and velocity prediction are algebraically equivalent in continuous time, they are not equivalent after discretization; the paper explicitly states that Euler updates in the same domain differ at $O(\Delta \lambda^2)$ depending on whether the model predicts noise, data, or velocity. Second, the integration domain matters: linear-time and log-SNR-like domains induce different integral weightings, and those differences become pronounced in the low-NFE regime, where quadrature error dominates. Dual-Solver addresses these effects by treating prediction type and integration domain as learnable numerical design variables rather than fixed solver choices (Park et al., 4 Mar 2026).

2. Dual prediction and generalized ODE parameterization

A central construction is the conversion between prediction types. The paper writes

$$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$

and

$$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$

This means that even when a backbone is trained for one parameterization, the others can be recovered at sampling time (Park et al., 4 Mar 2026).

The “dual” aspect of Dual-Solver refers to using the backbone’s data prediction $x_\theta$ and noise prediction $\epsilon_\theta$ as separate ingredients rather than collapsing them into a single velocity field. The solver introduces a learnable interpolation parameter $y$ that continuously spans the three standard formulations. The paper states that $y=-1$ recovers noise prediction, $3 \le \mathrm{NFE} \le 9$0 recovers velocity prediction, and $3 \le \mathrm{NFE} \le 9$1 recovers data prediction. In this parameterization, the ODE is written in a unified form whose coefficients depend on $3 \le \mathrm{NFE} \le 9$2, and the resulting integral update includes separate integrals of $3 \le \mathrm{NFE} \le 9$3 and $3 \le \mathrm{NFE} \le 9$4. This makes the prediction geometry itself part of the learned solver rather than a fixed modeling assumption (Park et al., 4 Mar 2026).

3. Integration domain, residual terms, and predictor-corrector structure

The second learnable component is the integration domain. Dual-Solver introduces a log-linear transform

$3 \le \mathrm{NFE} \le 9$5

with inverse

$3 \le \mathrm{NFE} \le 9$6

The limiting cases are explicit: as $3 \le \mathrm{NFE} \le 9$7, the transform approaches a linear domain, while $3 \le \mathrm{NFE} \le 9$8 gives the logarithmic form used by prior diffusion solvers. Separate domain parameters $3 \le \mathrm{NFE} \le 9$9 and $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$0 are used for the $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$1-integral and $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$2-integral (Park et al., 4 Mar 2026).

The third component is a learnable residual term

$$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$3

with an analogous form for $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$4. This term is constrained to be $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$5 or $$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$6, so it adds flexibility without changing the formal local order of the method (Park et al., 4 Mar 2026).

Dual-Solver keeps a standard predictor-corrector architecture. The first-order predictor is

$$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$7

and the second-order corrector is

$$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$8

$$x_t = a_t x_0 + o_t \epsilon, \qquad x_0 \sim p_{\text{data}},\ \epsilon \sim \mathcal{N}(0, I),$$9

The paper proves that the first-order predictor has truncation error $O(\Delta \lambda^2)$0, while the second-order predictor and the second-order corrector both have truncation error $O(\Delta \lambda^2)$1. This is the basis for the claim that Dual-Solver preserves second-order local accuracy despite its learnable parameterization (Park et al., 4 Mar 2026).

4. Learnable parameters and classifier-based training

Dual-Solver learns per-step solver parameters for both predictor and corrector. For each step, the predictor has

$O(\Delta \lambda^2)$2

and the corrector has

$O(\Delta \lambda^2)$3

Thus each step has 10 learnable parameters, except the last step, which has only predictor parameters. The timestep schedule is also learned: the paper uses unnormalized step variables, applies a softmax to obtain normalized step sizes, and then computes timesteps by cumulative summation (Park et al., 4 Mar 2026).

The paper contrasts this with regression-based solver learning, specifically trajectory regression, sample regression, and feature regression. Those methods require supervision from a teacher solver running at high NFE, whereas Dual-Solver uses a classification-based objective. For class-conditional generation, a sample is decoded and passed through a frozen classifier $O(\Delta \lambda^2)$4, and the loss is cross-entropy against the ground-truth label. For text-to-image generation, the classification loss is replaced by a CLIP loss. The reported frozen models include MobileNetV3-Large for ImageNet experiments with DiT and GM-DiT, and CLIP RN101 for text-to-image experiments with SANA and PixArt-$O(\Delta \lambda^2)$5. The paper further notes that classifier quality and downstream FID are not monotonically related; it reports a V-shaped trend in which a moderately accurate classifier can yield the best FID (Park et al., 4 Mar 2026).

5. Reported empirical performance in the low-NFE regime

The experimental setting covers ImageNet class-conditional generation with DiT-XL/2-256$O(\Delta \lambda^2)$6256 and GM-DiT 256$O(\Delta \lambda^2)$7256, and text-to-image generation with SANA 600M-512px and PixArt-$O(\Delta \lambda^2)$8 XL-2-512. Baselines are DDIM, DPM-Solver++, BNS-Solver, and DS-Solver. Evaluation uses FID on ImageNet and both FID and CLIP score on MSCOCO 2014 validation captions, with NFEs $O(\Delta \lambda^2)$9 (Park et al., 4 Mar 2026).

On DiT, the paper reports especially large gains at very low NFE. At NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$0, the FID values are 89.33 for DDIM, 88.46 for DPM-Solver++, 103.26 for BNS-Solver, 67.31 for DS-Solver, and 24.91 for Dual-Solver. At NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$1, Dual-Solver reports 3.52 versus 32.91 for DPM-Solver++, 14.53 for BNS-Solver, and 7.66 for DS-Solver. At NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$2, Dual-Solver reports 2.75 versus 7.42, 4.25, and 3.79 for those same baselines (Park et al., 4 Mar 2026).

On GM-DiT, the best reported variant is the classification-trained Dual-Solver, with FID 6.81 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$3, 3.76 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$4, 3.09 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$5, and 2.97 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$6. The paper states that the regression-trained variant is much weaker. For text-to-image, Dual-Solver also improves both FID and CLIP. On PixArt-$$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$7, it reports FID 66.61, 24.68, and 22.01 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$8, with CLIP 0.4499, 0.4784, and 0.4815 at the same NFEs. On SANA, it reports FID 21.79, 18.81, and 19.77 at NFE $$x_\theta(x_t,t) = \frac{x_t - o_t \epsilon_\theta(x_t,t)}{a_t},$$9, with CLIP 0.4795, 0.4821, and 0.4849 (Park et al., 4 Mar 2026).

6. Ablations, interpretation, and scope

The ablation studies identify the best-performing architectural combination as a first-order predictor with a second-order corrector. This outperforms a first-order predictor alone, a second-order predictor alone, and a second-order predictor combined with a second-order corrector. The paper also reports that allowing all three parameter families $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$0, $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$1, and $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$2 to remain learnable gives the best results at the smallest NFEs, while special parameter choices recover known solver families: $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$3 gives data prediction, $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$4 gives velocity prediction, $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$5 gives noise prediction, $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$6 gives the logarithmic domain, and $$v_\theta(x_t,t) = \frac{d a_t}{dt} x_0(x_t,t) + \frac{d o_t}{dt}\epsilon_\theta(x_t,t).$$7 removes the residual term (Park et al., 4 Mar 2026).

The paper further reports that the proposed log-linear transform performs better than a naive linear interpolation between linear and logarithmic transforms, attributing this to numerical stability near zero. It also finds that learned parameter curves are similar across different NFE budgets, and therefore interpolates parameters from neighboring NFEs to obtain parameters for unseen budgets; those interpolated solvers do not fully match directly optimized parameters, but still remain better than competing solvers (Park et al., 4 Mar 2026).

Within the broader literature, the name “dual solver” has also been used for optimization algorithms in settings such as model predictive control, quadratic programming, and conic programming. This suggests that “Dual-Solver” in the diffusion context functions as a domain-specific proper name rather than a generic label for one algorithmic family (Zhang et al., 2017, Arnström et al., 2021, Lin et al., 1 May 2025).