Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures

Abstract: iffusion-based generative models increasingly rely on inference-time guidance, adding a drift term or reweighting mixture of experts, to improve sample quality on task-specific objectives. However, most existing techniques require repeated score or gradient evaluations, introducing bias, high computational overhead, or both. We introduce \texttt{URGE}, Unbiased Resampling via Girsanov Estimation, a derivative-free inference-time scaling algorithm that performs path-wise importance reweighting via a Girsanov change of measure. Instead of computing gradient-based particle weights in previous work, \texttt{URGE} attaches a simple multiplicative weight to each simulated trajectory and periodically resamples. No score, no Hessian, and no PDE evaluation is required. We establish an equivalence between path-wise and particle-wise SMC: the Girsanov path weight admits a backward conditional expectation that recovers the previous particle-level weights, guaranteeing that both schemes produce the same unbiased terminal law. Empirically, \texttt{URGE} outperforms existing inference-time guidance baselines on synthetic tests and diffusion-model benchmarks, achieving better generation quality, while being significantly simpler to implement and fully gradient-free.

• The paper introduces URGE, a derivative‐free SMC approach that leverages Girsanov-based pathwise likelihood ratios to align diffusion model outputs without computing reward gradients.
• The methodology employs sequential resampling on particle ensembles to obtain unbiased weights, ensuring improved performance compared to traditional gradient-based correctors.
• Empirical evaluations demonstrate URGE’s robust accuracy, favorable runtime scaling, and superior results in Gaussian mixtures, inverse problems, and text-to-image generation.

Derivative-Free Pathwise Importance Resampling for Diffusion Inference via Girsanov SMC

Context and Motivation

Diffusion models formulated as SDEs parameterized by deep neural networks have established the state-of-the-art across generative modeling tasks in domains such as image synthesis, protein design, and text generation. Control and alignment at inference, particularly through task-conditioned reward functions, are central to practical downstream generation. Existing guidance-based inference-time scaling approaches typically augment the generative SDE dynamics with a drift term derived from (approximate) reward gradients. While effective, these methods—such as classifier-free guidance or FK-Correctors—incur either gradient evaluation costs, restrictive regularity assumptions, or introduce bias due to approximations, especially when reward gradients are unavailable or expensive to compute.

URGE Algorithmic Framework

This paper proposes URGE ("Unbiased Resampling via Girsanov Estimation"), a derivative-free, ensemble Sequential Monte Carlo (SMC) inference scheme for reward-conditioned sampling in diffusion models. Instead of reweighting or correcting generated samples after generation, URGE directly applies SMC via Girsanov theorem-based pathwise likelihood ratios on the generative process.

Concretely, the approach is as follows:

• Sampling proceeds by simulating an ensemble of particles under a guided SDE (with arbitrary guidance potential $G$) from an initial Gaussian base measure to a data distribution.
• At each time step, a per-path Radon-Nikodym weight, constructed via a Girsanov change of measure and the reward differential, is assigned to each trajectory.
• Particles are resampled proportionally to their weights, yielding a new ensemble that better matches the target reward-tilted posterior—in other words, the distribution proportional to $P_{\text{data}}(x) e^{r(x)}$.
• This resampling is performed at every discretized step, leveraging the reduction in particle degeneracy and variance associated with SMC, and does not rely on any reward derivatives (score, Hessian, or higher order).

Theoretical Contributions

Central theoretical results include:

• Marginalized Equivalence: The marginalized effect of URGE's pathwise resampling yields particle-level weights equivalent to previous state-of-the-art correctors, such as AFDPS and the FK-Corrector, but generalized to arbitrary reward functions without derivative computation. The effective generator augmentation in the sample space induced by the pathwise weights recovers the infinitesimal generator-based weighting structure of these methods.
• Approximation-Free Property: Due to the path measure construction and SMC resampling, URGE is provably unbiased—it does not introduce approximation error in the terminal-time posterior induced by suboptimal guidance, in contrast to local-only drift interventions.
• Implementation and Robustness: URGE enjoys straightforward implementation; it replaces reward/potential gradients with simple finite-increment computations for weights, and its accuracy and stability are maintained as time discretization is refined.

Empirical Evaluation

Synthetic Gaussian Mixture Model

On 30–60 dimensional Gaussian mixture models, URGE achieves the lowest discrepancy (MMD, SWD, mean error, covariance error) to analytic ground-truth among tested methods, including PG, AFDPS, AFDPS+VCG, and FK-Steering. Performance gains are robust to increases in dimension and number of mixture components, with the gap over baselines widening as the problem becomes more complex—demonstrating superior scaling and bias reduction.

Inverse Problems

On inverse tasks (ImageNet-256, FFHQ-256; e.g., Gaussian/motion deblurring, box inpainting, super-resolution) URGE matches or exceeds the performance (in PSNR, LPIPS) of AFDPS and surpasses reward-free baselines, while significantly reducing computational cost by obviating the need for reward function derivatives. The method is competitive or superior in runtime even in particle-rich regimes, suggesting practical deployment advantages.

Text-to-Image Generation

Applied to prompt-conditioned generation with Stable Diffusion v1.5 and SDXL:

• URGE achieves higher CLIP-Score, ImageReward, and Human Preference Score than both base samplers and FK-Steering, even without reward gradients.
• With only four particles, URGE attains or approaches the output quality of substantially larger (SDXL) models, indicating strong reward alignment and sample quality improvements via SMC ensemble effects.
• Qualitative comparisons show that URGE addresses compositional prompts (multiple object/color specification) better than FK-Steering and the base sampler: generated images exhibit closer alignment to prompt semantics and consistent object features.
• Runtime scaling with respect to ensemble size remains favorable compared to competing methods.

Implications and Future Directions

This work establishes that pathwise SMC on generative trajectories, enabled by Girsanov likelihood ratios, offers an unbiased, derivative-free alternative to gradient-based inference-time scaling in diffusion models. The implications are multifold:

• Practicality in Black-Box Reward Settings: Many real-world reward signals (e.g., neural reward models, classifier outputs) are non-differentiable/opaque. URGE enables effective reward alignment in such settings without requiring access to reward derivatives.
• Scalability: The method is compatible with high-dimensional, large-scale generation (text-to-image, protein design, etc.), since performance scales favorably with the number of SMC particles and computation remains tractable.
• Design of Importance Weights: Lifting importance weighting to path space (rather than endpoint samples) enables flexible construction, opening opportunities for non-standard reward schedules, advanced numerical integration, and potentially adaptive or learned guide policy schemes.
• Generalization Beyond Diffusion: The SMC-path viewpoint and its equivalence to state-correctors connect this domain to broader advances in nonlinear filtering, rare event simulation, and path-based Bayesian inference, suggesting avenues for method transfer and cross-pollination.

Future directions include:

• Adaptive or data-driven choice of resampling intervals and mixture ensemble sizes.
• Integration with RL-style reward learning loops.
• Deployment in domains where reward specification is fundamentally non-differentiable.
• Study of convergence rates and variance reduction techniques in high-dimensional SDE sampling.

Conclusion

URGE presents a practical, theoretically principled, and empirically validated approach for inference-time scaling in diffusion models. By leveraging pathwise SMC with Girsanov weights, it subsumes the virtues of both guidance-based correction and approximation-free resampling, while eliminating the need for reward derivatives. The result is a straightforward framework for reward-aligned generation that is broadly applicable across domains and robust to model, reward, and inference time heterogeneities.