Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference

Abstract: Compositional score-based approaches to simulation-based inference (SBI) approximate the posterior over a shared parameter given $n$ independent observations by aggregating individually learned posterior scores: currently, there are two main propositions of such methods (Geffner et al. (2023), Linhart et al. (2026)). As the resulting composite score does not correspond to the score of any distribution along the forward diffusion path of the true multi-observation posterior, sampling from it via a reverse SDE leads to an irreducible bias. Annealed Langevin dynamics provides a principled alternative: it treats the composite score as the genuine score of a sequence of tractable bridging densities and samples from them in succession. When properly tuned, it could lead to a controllable bias. However, its hyperparameters, namely step sizes, the number of steps per level, and the number of annealing levels, have so far been chosen empirically. We derive Wasserstein bounds for annealed Langevin with approximate scores and translate them into explicit decision rules for these hyperparameters that guarantee a prescribed sampling accuracy, while highlighting different theoretical aspects of each composite score formulation. In the Gaussian setting, we obtain closed-form expressions for all relevant quantities and prove that the bridging densities of Linhart et al. (2026) consistently admit larger step sizes and require fewer total Langevin steps than those of Geffner et al. (2023). Furthermore, we show empirically that the tuning obtained in the Gaussian setting generalizes to more complex problems, thus providing a well-understood and theoretically grounded starting point for practitioners using compositional score-based approaches.

• The paper establishes a theoretical framework to set ALD hyperparameters by deriving explicit non-asymptotic Wasserstein error bounds.
• It converts error bounds into closed-form decision rules that balance score bias and discretization variance across the annealing levels.
• Empirical and analytical comparisons reveal that Linhart’s compositional scheme enables larger step sizes and fewer iterations, improving sample efficiency.

Theoretical Guidelines for Annealed Langevin Dynamics in Compositional Simulation-Based Inference

Overview and Motivation

Compositional score-based approaches to simulation-based inference (SBI) provide a practical framework for amortized Bayesian inference when the likelihood is intractable but simulation is feasible. In scenarios with $n$ i.i.d. observations sharing a common latent parameter, compositional SBI leverages individual score networks trained on single-observation data, enabling multi-observation posteriors to be constructed without additional simulation cost or retraining. This compositional paradigm, as formalized by Geffner et al. [13] and Linhart et al. [16], has notable efficiency advantages over pooling strategies.

A central technical challenge addressed in this work is the theoretically sound and efficient sampling from the composite posterior. While reverse SDE/DDPM samplers are ubiquitous in diffusion-based inference, these methods require the exact multi-observation score, which compositional approximations structurally cannot provide. Directly running the reverse SDE with a composite score induces an irreducible bias. Annealed Langevin dynamics (ALD) offers a principled alternative: it interprets the composite score as the true score of a bridge distribution sequence and samples from these intermediate densities via successive Langevin steps. The remaining practical bottleneck, and the focus of this paper, is the principled selection of ALD hyperparameters (step size, steps per level, number of levels) to guarantee sampling accuracy.

Wasserstein Error Bounds and Hyperparameter Decision Rules

The authors derive explicit non-asymptotic bounds for the Wasserstein-$2$ distance between the sample distribution output by ALD and the composite target, accounting for both the effect of inexact score matching and interpolation error between consecutive bridge densities. Building on the analysis of Dalalyan and Karagulyan, the bounds depend on per-level log-concavity and smoothness constants of the bridge densities, the square root of the score-matching error, and the Wasserstein distance between adjacent densities.

A key contribution is translating these error bounds into concrete, theoretically justified scalar decision rules for ALD hyperparameters. By explicitly balancing bias (score error, decreasing with smaller step size) and variance (finite-step discretization error, controlled by number of steps), the authors provide closed-form formulas for setting per-level step sizes and step counts to guarantee a user-specified Wasserstein error threshold $\gamma$. The parameterization is sensitive to the specific compositional scheme, as the geometry and regularity properties of the bridge densities (log-concavity, Lipschitz constants, etc.) directly determine the admissible tuning range.

This results in a theoretically backed prescription that, for a prescribed sampling accuracy, eliminates the need for heuristic or trial-and-error tuning of the ALD sampler.

Comparative Analysis: Geffner vs. Linhart Compositional Schemes

The study provides an analytic comparison of the two main compositional score constructions in the context of Gaussian SBI:

1. Geffner et al.: Aggregates individual posterior scores plus a prior score with simple weights, yielding a bridge density that is analytically tractable and well conditioned when all individual posteriors and the prior are Gaussian.
2. Linhart et al.: Incorporates a second-order Gaussian approximation in the compositional score, resulting in a more refined bridge closely tracking the true time-evolved posterior and involving explicit covariance estimation.

In the Gaussian case, all necessary quantities—score-matching error, smoothness, log-concavity, and Wasserstein distances—are explicit. The analysis shows that (except for the very first annealing levels) Linhart's bridge densities allow for larger step sizes and require fewer total ALD steps compared to Geffner's. Theoretical results (especially Proposition 3 and Corollary 1) establish that the minimum-to-maximum eigenvalue ratios (i.e., conditioning) of Linhart's intermediates are always more favorable, directly enabling more aggressive ALD step-size selection. The number of steps per level is controlled primarily by the Wasserstein distance between adjacent densities, with Linhart's construction dominating as $t \to 1$ (i.e., close to the base Gaussian).

Numerical experiments validate that these conclusions extend beyond the Gaussian regime, with empirical hyperparameter selection based on local Gaussian approximations effectively retaining Wasserstein error control in a range of non-Gaussian, multimodal, and dynamical simulation models.

Implications and Theoretical Insights

Theoretical Implications

• ALD Hyperparameter Setting: This work disambiguates the interplay between score approximation bias and ALD discretization error in compositional SBI. The derived rules allow theoretical guarantees on sample closeness (in Wasserstein) without expensive trial-and-error tuning.
• Generalization Beyond Gaussianity: The analytical techniques extend to non-Gaussian settings under mild regularity (smoothness and approximate log-concavity), showing that the bounds and selection rules remain robust whenever asymptotic normality holds (per Bernstein-von Mises).
• Efficiency Frontier for Compositional SBI: Linhart's bridge construction is established as the more sample-efficient baseline for ALD in high-$n$, high-$d$ regimes, being able to use larger steps and achieve a given accuracy with fewer score computations. This is a sharp quantitative claim supported both theoretically and empirically.

Practical Implications

• Plug-and-Play Annealed Langevin Dynamics: Practitioners can now set ALD hyperparameters for compositional inference in a provably “safe” manner, obtaining reproducible sampling accuracy across a wide range of SBI tasks and models. The guidelines can be immediately implemented where compositional score-based SBI has already been deployed.
• Tuning-Free Compositional SBI: Avoids the need for empirical tuning, which is especially valuable for high-dimensional or expensive simulators and for practitioners lacking access to ground truth posteriors.
• Design of Novel Bridge Densities: The explicit dependence of the decision rule on bridge density properties provides a clear pathway for designing improved compositional schemes that optimize ALD efficiency even further.

Directions for Future Research

The reliance on (approximate) log-concavity and smoothness, while mild, limits applicability in deeply non-Gaussian or multimodal inference regimes. A central open direction is extending these results to bridge densities satisfying only log-Sobolev inequalities or weaker regularity properties, which would further broaden theoretical guarantees. Additional research on analytical tractability and empirical estimation of bridge density properties (eigenvalues, score errors) in “difficult” SBI regimes is also warranted.

Conclusion

This work establishes the first theoretically principled framework for hyperparameter selection in annealed Langevin samplers applied to compositional score-based SBI, supporting explicit accuracy guarantees under realistic distributional assumptions. The analysis provides a comprehensive efficiency comparison between leading compositional schemes, identifying Linhart's construction as consistently superior in sample complexity for most practical regimes. The guidelines are readily implementable in both Gaussian and non-Gaussian settings, and offer a rigorous foundation for future algorithmic and theoretical advances in compositional simulation-based inference.

Reference:

"Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference" (2605.21253)