Markov Chain Monte Carlo with Diffusion Paths

Abstract: Sampling from multimodal distributions is a longstanding challenge for classical local Markov chain Monte Carlo (MCMC) methods. A popular remedy is to introduce a sequence of intermediate distributions that interpolate between the target and a simpler reference. The classical choice, tempering, raises the density to a power, but distorts the relative weights of asymmetric modes and can lead to poor mixing. We instead propose interpolating along the diffusion path, the marginals of a noising diffusion process that carries the target toward a Gaussian. This path preserves the relative weights of the modes and enjoys favorable mixing properties, which we make precise through a spectral-gap analysis of the corresponding ideal transition kernel. Sampling along the path requires its intermediate scores, which can be estimated from the unnormalized target through variational approaches, yielding only an approximate sampler. To remove the resulting bias, we introduce the Metropolis-adjusted diffusion path (MAD-Path) sampler, which corrects the diffusion-path proposal in an augmented path space and leaves the target invariant regardless of the accuracy of the learned score or the discretization error. We further quantify how these two errors affect the acceptance probability, providing guidance for practical tuning. Experiments on a range of Bayesian posteriors show that MAD-Path improves global exploration and mode-weight estimation relative to tempering-based MCMC methods and unadjusted diffusion samplers.

• The paper presents a novel diffusion path interpolation method to preserve mode weights and overcome spectral-gap limitations in high-dimensional, multimodal distributions.
• It leverages learned scores and path-space Metropolis adjustments to correct bias, ensuring optimal acceptance probabilities and reliable posterior estimation.
• Empirical results on synthetic and real Bayesian models demonstrate robust global exploration, rapid mixing, and accurate mode recovery even under scale disparities.

Markov Chain Monte Carlo with Diffusion Paths: An Expert Overview

Motivation and Background

Classical local MCMC algorithms, such as RWM, MALA, and HMC, exhibit poor mixing for multimodal targets due to rare transitions across low-probability barriers. The canonical remedy—tempering—interpolates between the target and a simpler reference via powers of the density, facilitating mode traversal but distorting relative mode weights for asymmetric mixtures and inducing torpid mixing in high dimensions. This is notably problematic when modes differ substantially in scale, resulting in exponential suppression of narrower components. Recognizing that tempering is not inherently tied to the mechanics of simulated or parallel tempering, the paper seeks a more robust interpolation scheme.

Diffusion Path Interpolation

The central proposal is to interpolate via the diffusion path: the time-marginals of the Ornstein-Uhlenbeck (OU) noising process. The forward OU moves the distribution toward a Gaussian, and its time-reversals form proposals for MCMC. Crucially, this path preserves the exact mixture weights for any intermediate state, regardless of component disparities in scale or variance.

Figure 1: Illustration of MAD-Path, showing forward diffusion mixing modes, followed by backward reconstruction using learned scores and Metropolis correction.

Spectral-gap analysis reveals that the ideal diffusion-path kernel maintains a dimension-independent gap even for highly asymmetric mixtures, in sharp contrast to tempering-based MCMC, which suffers exponentially vanishing gaps. The theoretical guarantees extend to mixture targets, with the gap decomposing into between-mode (label) and within-mode (component) contributions, bottlenecked only by mode separation and component conditioning.

Metropolis-Adjusted Diffusion Path (MAD-Path): Algorithmic Contributions

The practical hurdle is score intractability: intermediate marginals in the diffusion path require scores $\nabla \log p_t$ unavailable in most settings. The paper leverages variational path-space inference and stochastic optimal control to estimate these scores from the unnormalized target, yielding only approximate proposals. To remove bias from score and time discretization errors, the MAD-Path algorithm augments path-space Metropolis adjustment—applying the acceptance/rejection criterion to full forward/backward trajectories.

Key properties:

• Exact invariance: The target is invariant under the MAD-Path transition irrespective of score or discretization accuracy.
• Correcting learned samplers: MAD-Path fixes bias otherwise present in unadjusted diffusion-based samplers or variational flows, especially for mode-weight estimation and highly nonlinear targets.

Quantitative Analysis: Acceptance Probability and Scaling

Acceptance probability is analyzed as a function of score error and discretization.

• With exact scores: The rejection probability decays as $O(h^{1/2})$, and high-dimensional optimal-scaling analysis yields the standard acceptance rate optimality (≈0.234) for cost-normalized efficiency.
• With approximate scores: The mean log acceptance ratio is governed by the symmetrized path-space KL divergence between approximate and ideal reverse diffusions, or, equivalently, the integrated squared score error.

  Figure 2: Cost-normalized efficiency versus acceptance rate in high dimension, demonstrating the optimal scaling for MAD-Path proposals.

Empirical Results

Experiments cover challenging synthetic and real Bayesian posteriors:

• Two-Gaussian mixture with unequal variances: Only MAD-Path accurately recovers the correct mode weights; tempering-based samplers and particle methods collapse or misrepresent modes, especially under exponential scale disparity.

  Figure 3: Trace plots and empirical marginals for the two-Gaussian mixture; MAD-Path achieves rapid mode crossing and accurate mixture weights.

• Mixture of skew-normal components: MAD-Path jumps among all modes and estimates mode weights accurately; unadjusted diffusion samplers suffer from mode collapse or Gaussian shape bias within modes; tempering-based samplers remain biased.

  Figure 4: MAD-Path versus baselines for a 13-mode skew-normal mixture—accurate mode discovery and shape recovery.

• Bayesian mixture models (Old Faithful, fetal deaths): MAD-Path demonstrates superior mode exploration and correct relative weights, while tempering baselines exhibit highly variable or imbalanced success rates and mode recovery.
• Sensor network localization and SUR regression: MAD-Path enables rapid burn-in, correct initial mode coverage, and efficient mixing, even under near non-identifiability and multiple modes; tempering-based chains often require substantial warmup.

  Figure 5: Bayesian GMM mode-weight estimation—MAD-Path concentrates estimates around the true value, outperforming APT-DEO.

  

  Figure 6: Acceptance and inter-mode crossing rates for MAD-Path, confirming theoretical predictions for diffusion horizon and discretization.

Practical Considerations and Implications

MAD-Path is agnostic to the method of score estimation and is robust to inaccuracies, thanks to the path-space Metropolis adjustment. The algorithm is amortized—target gradients are only required during score learning—not at every MCMC step, which enables competitive wall-clock costs relative to traditional methods.

Implications:

• Theory: The diffusion-path approach circumvents spectral-gap bottlenecks of tempering, providing efficient global exploration in mixtures and highly separated modes.
• Practice: MAD-Path yields reliable mode discovery, accurate posterior estimation, and rapid mixing on problems where conventional samplers fail.
• Future development: Score-learning objectives more aligned with finite-horizon MAD-Path and better mode-discovery reference constructions are promising directions. Hybrid paths, non-Gaussian references, and improved control-theoretic score estimation may further accelerate convergence and acceptance.

Conclusion

Markov Chain Monte Carlo with Diffusion Paths offers a principled interpolation between complex targets and tractable references via noising diffusions, fundamentally overcoming long-standing limitations of tempering for multimodal distributions. The MAD-Path algorithm combines learned scores and path-space Metropolis adjustment to ensure correctness and practical efficiency, as validated by both theory and extensive empirical results. The approach enables robust, global mixing in Bayesian inference and opens new possibilities for generative modeling and stochastic simulation in high-dimensional, multimodal regimes.