Modular Markov chain Monte Carlo with application to multimodal sampling

Abstract: We develop a modular approach to Markov chain Monte Carlo (MCMC) sampling for unnormalized target densities. In this approach, Markov chains are constructed in parallel, each constrained to a subset of the target space. The Monte Carlo estimates from the constrained chains are then combined with appropriate weights, calculated from the transition probabilities between subsets. In addition to the computational advantages arising from its parallelized structure, this modular MCMC approach enables variance reduction for Monte Carlo estimation in settings where sampling from low-density regions is required. We develop a central limit theorem-type result for the resulting Monte Carlo estimates and propose a method for estimating their standard errors. Furthermore, by applying this modular sampling technique to simulated tempering, we propose a method for Monte Carlo estimation of expectations with respect to multimodal target distributions. This approach effectively addresses a well-known challenge of tempering-based methods: sampling efficiency can be greatly reduced when separated modes of the target distribution have different scales. We demonstrate the efficiency of the proposed methods through numerical examples, including one arising from Bayesian sparse regression with a spike-and-slab prior.

• The paper introduces a modular MCMC algorithm that partitions the sampling space and uses region-specific chains to enhance multimodal sampling efficiency.
• It employs an eigenvector-based method to compute weights, ensuring accurate estimation and variance reduction across distinct modal regions.
• The approach outperforms traditional methods in high-dimensional settings and handles heterogeneous mode scales via modular simulated tempering.

Modular Markov Chain Monte Carlo with Application to Multimodal Sampling

Introduction and Motivation

This paper introduces a modular approach to Markov chain Monte Carlo (MCMC), engineered to address core limitations in conventional MCMC methods when sampling from multimodal and/or high-dimensional target distributions. The modular framework partitions the state space into non-overlapping regions and constructs parallel chains, each restricted to one region, significantly improving computational parallelism, reducing estimator variance, and facilitating accurate exploration of disparate low-probability regions.

A central innovation is the coupling of mode-specific Markov chains via an eigenproblem: global weights for combining estimates are derived from the stationary distribution of an empirically estimated transition matrix between regions. The approach generalizes naturally to a modular simulated tempering scheme that resolves a longstanding challenge—efficient sampling from multimodal targets with modes of heterogeneous scale, where classic tempering approaches typically fail due to scale-induced torpid mixing.

Modular MCMC: Algorithmic Architecture

The modular MCMC algorithm begins by partitioning the target space $X$ into $L$ non-overlapping subsets $A_1, \dots, A_L$. For each region $A_i$, an independent, region-constrained Markov chain is simulated with the same global proposal kernel as the unconstrained chain. Chains are allowed to propose moves out of their region but remain within their region when rejecting out-of-region proposals. Detailed transition statistics between regions are maintained during simulations.

Weights for each region—necessary for recombining partitionwise expectations—are computed as components of the (left) eigenvector corresponding to eigenvalue one of the empirical transition matrix $Q$ estimated via the constrained chains. This systematically generalizes stratified sampling, and crucially, obviates the need to empirically observe rare transitions between states in the case of isolated modes.

Standard error estimates for the final modular MCMC are constructed using a combination of block-averaged variances of region-specific Markov chains and a bootstrap procedure on the estimated transition matrix. This ensures precise quantification of Monte Carlo uncertainties even in challenging multimodal settings.

Theoretical Analysis

The paper establishes a suite of asymptotic results under mild conditions, notably:

• Consistency: As the length of each constrained chain increases, both regional expectations and estimated global weights converge in probability to their true values.
• Central Limit Theorem (CLT): Given suitable geometric ergodicity and irreducibility, the modular estimator for expectations converges in distribution, at the canonical $\sqrt{n}$ rate, to a normal law centered at the true expectation with computable variance.

These results underpin the robustness of the modular approach to finite-time effects and support sound error estimation.

Modular Simulated Tempering

Multi-modal distributions present formidable difficulties due to rare mode-to-mode transitions; these issues are compounded when the scales of the modes differ substantially, as transitions between broad and narrow modes tend to stall at intermediate temperatures. The authors extend modular MCMC to a simulated tempering framework operating on an augmented space $(k, x)$, where $k$ is a temperature index and $x$ in the state space.

Each chain is constrained to a region $A_i$ and a temperature level $L$0, and modular weights are learned globally by compiling transitions among all $L$1 pairs. The method supports temperature- and mode-dependent mixture weights, facilitating efficient and flexible temperature ladder tuning and allowing atypical transitions—such as jumps between narrow and broad modes at specific temperatures—that are otherwise poorly sampled.

Algorithmic tuning strategies for temperature ladder and mixture weights are proposed, with pilot runs to iteratively adapt inverse temperature levels and weights based on empirical transition rates, ensuring numerical irreducibility and appropriate mixing.

Partitioning of the state space is handled by local optimization (gradient ascent) from diverse seeds to pinpoint modal basins; sets $L$2 are assigned by the basin of attraction under gradient flow or, in high dimensions, by proximity to initial seeds.

Numerical Experiments

Gaussian Mixtures with Heterogeneous Mode Scales

The modular simulated tempering (Modular ST) algorithm is benchmarked against adaptive parallel tempering (PT) and tempered sequential Monte Carlo (TSMC) on synthetic $L$3-dimensional Gaussian mixtures with varying scale ratios $L$4 between components ($L$5; $L$6).

Modular ST demonstrates significantly lower bias and variance in estimating modal probabilities across all $L$7 and $L$8 settings, whereas PT and TSMC show rapidly deteriorating performance as scale differences and dimensionality increase. These classical methods frequently fail to transfer mass appropriately between modes when their sizes differ greatly, leading to severe under- or over-representation of certain modes, particularly in high dimensions.

Figure 1: Distribution of estimates of $L$9—mode allocation probability—for modular ST, PT, and TSMC across dimensionality and scale ratio; modular ST maintains accurate and low-variance estimation even for large $A_1, \dots, A_L$0 and $A_1, \dots, A_L$1.

Figure 2: Estimated and empirical standard error of modular ST’s modal probability estimator, verifying the reliability of the error estimation procedure for various dimensions and scale ratios.

Bayesian Sparse Regression with Spike-and-Slab Prior

Practical utility is showcased in sparse linear regression with a spike-and-slab prior, wherein artificial collinearity produces a strongly multimodal posterior with two dominant, well-separated modes. Modular ST identifies modes, partitions the space accordingly, tunes temperature ladder and weights automatically, and recovers correct posterior probabilities assigned to the modes with standard error estimates commensurate with observed Monte Carlo variation.

Relations to Other Stratification and Multimodal Sampling Approaches

Modular MCMC shares conceptual ground with the Eigenvector Method for Umbrella Sampling (EMUS), but improves upon EMUS by working with non-overlapping partitions rather than overlapping bias functions, simplifying the design and avoiding the delicate calibration of bias potentials. In contrast to classical tempering, modular ST does not rely on observing rare transitions between modes and is robust to scale and shape heterogeneity among modes, unlike methods that depend on proximity or elliptical geometry, such as mode-jumping or Hessian-adjusted strategies.

Theoretical and Practical Implications

The modular framework recasts multimodal MCMC into an embarrassingly parallel problem, demystifying ambiguous mixing behavior seen in conventional tempering for distributions with vastly different modal scales. The most notable claim is that modular ST maintains efficient sampling and correct mode probability estimation in regimes where classic tempering approaches fail, particularly for large scale ratios and in high dimensions. This substantially broadens the practical applicability of MCMC to challenging multimodal Bayesian and statistical physics applications.

The method also offers variance reduction and improved uncertainty quantification for small-probability event estimation, an area of longstanding difficulty with standard MCMC due to poor exploration of low-density regions.

Future Directions

Potential areas for further research include adaptation of the modular strategy for even more challenging high-dimensional settings, improved automatic partitioning algorithms, and hybridization with advanced HMC kernels for further efficiency. The authors note opportunities for synergy with scalable, tempered HMC frameworks and the broader integration of modular MCMC into generic Bayesian computation pipelines.

Conclusion

This work establishes modular MCMC as a theoretically principled, computationally efficient, and practically robust approach to multimodal sampling, particularly effective for targets with widely varying mode scales and in higher dimensions. Through parallelization, eigenvector-based combination of regional expectations, and integration with simulated tempering, the modular approach systematically overcomes pathologies native to classical MCMC and tempering algorithms, opening new avenues for Monte Carlo methods in complex statistical applications.

Reference:

Joonha Park, "Modular Markov chain Monte Carlo with application to multimodal sampling" (2605.01198).