Score-based sampling without diffusions: Guidance from a simple and modular scheme

Abstract: Sampling based on score diffusions has led to striking empirical results, and has attracted considerable attention from various research communities. It depends on availability of (approximate) Stein score functions for various levels of additive noise. We describe and analyze a modular scheme that reduces score-based sampling to solving a short sequence of ``nice'' sampling problems, for which high-accuracy samplers are known. We show how to design forward trajectories such that both (a) the terminal distribution, and (b) each of the backward conditional distribution is defined by a strongly log concave (SLC) distribution. This modular reduction allows us to exploit \emph{any} SLC sampling algorithm in order to traverse the backwards path, and we establish novel guarantees with short proofs for both uni-modal and multi-modal densities. The use of high-accuracy routines yields $\varepsilon$-accurate answers, in either KL or Wasserstein distances, with polynomial dependence on $\log(1/\varepsilon)$ and $\sqrt{d}$ dependence on the dimension.

• The paper introduces a novel modular framework that reduces complex score-based sampling to a sequence of strongly log-concave subproblems, eliminating the need for SDE discretization.
• It achieves provable complexity bounds with √d and poly-logarithmic dependence on error, offering logarithmic scaling in the condition number for unimodal targets.
• The framework is modular and flexible, enabling integration of state-of-the-art SLC samplers and promising improvements in both diffusion-based and multimodal sampling settings.

Score-Based Sampling without Diffusions: Modular Reduction to Strong Log-Concavity

Overview and Motivation

The paper "Score-based sampling without diffusions: Guidance from a simple and modular scheme" (2512.24152) presents a modular framework for efficient score-based sampling of general target distributions, entirely bypassing the discretization of stochastic differential equations (SDEs) central to diffusion models. The motivation arises from the empirical success and theoretical proliferation of score-based diffusion models in generative modeling, Bayesian inference, and Monte Carlo methods, which however rely on computationally intensive simulating of SDEs, incurring suboptimal scaling in both dimension and sampling accuracy.

Instead, the author proposes a reduction that decomposes the original sampling task into a short, fixed sequence of subproblems for which highly optimized routines exist, specifically for strongly log-concave (SLC) distributions. The theoretical contributions substantiate the reduction's claims both in the unimodal (SLC) and general multimodal setting, yielding provable guarantees and iteration complexities superior to prior diffusion-based approaches.

Modular Framework: Annealing and Backward Trajectories

The scheme emulates an annealing trajectory: starting from the original (potentially multimodal) target measure, iteratively adding Gaussian noise and thus "smoothing" the distribution, leading to a terminal measure that is strongly log-concave. This forward trajectory is parametrized by a deterministic sequence of variance parameters. Crucially, each intermediate distribution in the trajectory, and, more importantly, each backward transition (i.e., sampling from the conditional of an earlier level given the next) can be shown to be SLC with bounded condition number.

Given access to Stein scores at all levels ("annealed score functions"), sampling proceeds in reverse via a Markov chain that traverses these SLC backward conditionals, leveraging any high-accuracy SLC sampler (e.g., Langevin, MALA, Hamiltonian Monte Carlo, or accelerated mid-point schemes).

Quantitative Guarantees

Unimodal Log-Concave Setting

For a target density $p$ that is $(m, M)$-SLC ($m,M > 0$), the paper demonstrates that one can construct a trajectory of at most $K = 1 + \log_2(M/m)$ steps, such that:

• Each backward conditional to be sampled is SLC with condition number bounded by $2$.
• The terminal distribution is SLC with condition number at most $2$.
• The total oracle complexity for producing samples that are $\varepsilon$-accurate (in KL or rescaled Wasserstein-2 distance) from $p$ is:

$$T(\varepsilon) = (K+1) \cdot N_{SLC}(\varepsilon/(K+1))$$

where $N_{SLC}(\cdot)$ is the complexity of the SLC-sampler.

With state-of-the-art algorithms, this yields

$$T(\varepsilon) = O\left( \sqrt{d} \cdot \log(M/m) \cdot \log^3(1/\varepsilon) \right)$$

notably achieving logarithmic dependence on the condition number, in contrast with the polynomial dependence in $\kappa$ for classical SLC sampling methods, including ULA, MALA, and HMC.

Multimodal Setting

For general (not necessarily log-concave) targets, the methodology employs adaptive annealing steps based on conditional covariance bounds, controller by parameters $B_k$ (covariance operator norm of the original variable conditioned on the $k$-th smoothed variable). With the implemented choice of stepsizes, the following are proven:

• After $K = O(B)$ steps (where $B$ depends on initial data dispersion or domain constraints), the terminal and all backward conditionals have condition numbers at most $4$ and $2$, respectively.
• The total oracle complexity for KL-accurate sampling is

$$T(\varepsilon) = O\left( K \sqrt{d} \log^3(K/\varepsilon) \right)$$

• Under mild assumptions (bounded domain), $K=O(R^2/\delta^2)$, yielding worst-case iteration complexity with only a quadratic dependence on the spread-to-target ratio.

For Wasserstein-2, the error compounding is worse due to inverse stepsize dependence, but the piecewise SLC structure still allows poly-logarithmic dependence on $1/\varepsilon$ and mild $K$-dependence.

Key Technical Underpinnings

• Forward and Backward Hessian Control: The analysis pivots on second-order spectral bounds propagated by the forward and backward Tweedie formulas, leveraging the Brascamp–Lieb inequality for control of posterior covariances in SLC distributions.
• Error Accumulation and Backward Stability: Accumulated approximation error is rigorously bounded across backward steps using Markov kernel stability. In the SLC setting, error growth is linear; for multimodal targets, the bound incurs an additional factor reflecting the contraction constant of the backward kernel.
• Modular “Black-Box” Reduction: The framework's core strength is its modularity; any SLC sampler (and, by extension, any improved method for other structured classes, e.g., LSI or Poincaré families) can be incorporated without changing the reduction mechanism.

Contrasts with Diffusion-Based Approaches

Diffusion models (and their deterministic ODE analogues) require simulating discretized SDEs along a fine partition indexed by the target error, leading to complexity scaling at least linearly in $d$ and polynomially in $1/\varepsilon$. The proposed modular approach, by using “large steps” allowed by SLC guarantees at each backward transition, decouples the number of trajectory steps $K$ from accuracy $\varepsilon$, facilitating:

• $\sqrt{d}$-dependence (matching best known for SLC sampling, and superior to prior score-diffusion results).
• Poly-logarithmic dependence on $1/\varepsilon$ rather than polynomial.
• For SLC targets, logarithmic dependence in condition number (versus polynomial for SLC methods and diffusion models).

The method thus contradicts established lower bounds for SLC sampling that only use first-order access to the original target, by crucially leveraging access to the entire sequence of annealed Stein scores.

Open Questions and Future Directions

• Accuracy with Estimated Scores: The analysis assumes access to exact (possibly learned) annealed scores; a pressing direction is incorporation of error due to score estimation as in empirical score-based diffusion.
• Beyond SLC Reductions: Extending the modular approach to smoother, but not necessarily SLC, sub-problems (e.g., log-Sobolev or Poincaré class) could further enhance generality and reduce worst-case dependence on domain smoothness.
• Adaptivity to Low-Dimensional Structure: Recent literature on diffusion models for manifolds and low-dimensional phenomena prompts adapting the reduction to exploit effective/intrinsic rather than ambient dimension.
• Optimizing Trajectory Lengths: The worst-case scaling with quadratic dependence on bounds (e.g., $R^2/\delta^2$ in multimodal case) is likely improvable. Whether poly-logarithmic dependence on both spread and accuracy can be achieved remains open.
• Practical Implementability: Although the theoretical framework is “black-box” and modular, computational tractability with learned or approximate scores, and practical performance in high-dimensional applications, are open for detailed empirical investigation.

Conclusion

This work rigorously establishes a reduction from complex score-based sampling to a sequence of SLC sampling problems under mild conditions, improving iteration complexity with respect to both dimension and error precision. The framework transforms a core aspect of generative modeling and MCMC via a modular, theoretically robust mechanism that steps away from trajectory-discretized SDE simulation, opening a path toward more scalable and adaptable sampling algorithms in probabilistic inference and AI (2512.24152).