On the mixing properties of some preconditioned multiproposal Markov Chain Monte Carlo algorithms

Abstract: We study two recently discovered "dimension-free" Monte Carlo sampling algorithms, the multiproposal and multiple-try preconditioned Crank-Nicolson methods (mpCN and MTpCN). These methods were designed to address certain non-parametric (i.e. infinite-dimensional) sampling problems, defined relative to a Gaussian reference measure, by combining proposal and acceptance mechanisms that take non-trivial advantage of parallel computing architectures. We provide the first rigorous analysis of both algorithms, establishing exponential convergence to the target measure through the weak Harris framework, both for a finite number of proposals and in the infinite-proposal limit. The resulting mixing rates are independent of the dimension and uniform in the number of proposals, and apply to targets with bounded, Lipschitz log-likelihoods, without requiring convexity. At the center of the analysis are two new coupling constructions, together with analytical tools of independent interest, yielding Wasserstein contraction estimates, $L^2$ spectral gaps, and associated statistical guarantees (laws of large numbers, central limit theorems, and non-asymptotic concentration bounds) for the corresponding Monte Carlo estimators. These theoretical results are complemented by a numerical study on benchmark problems with complex posterior geometries and high-dimensional structure, comparing mpCN and MTpCN against standard pCN and independent parallel-chain implementations. The experiments indicate that the multiproposal methods can offer a shorter warm-up phase and greater robustness to the choice of tuning parameters as the number of proposals grows.

• The paper demonstrates that mpCN and MTpCN achieve exponential Wasserstein contraction, ensuring robust performance independent of dimensionality and proposal count.
• It establishes uniform spectral gaps and concentration bounds, providing strong law of large numbers and central limit theorems for advanced Bayesian computations.
• Numerical experiments confirm that these algorithms offer rapid burn-in and insensitivity to tuning, significantly enhancing the efficiency of high-dimensional MCMC sampling.

Mixing Properties of Preconditioned Multiproposal Markov Chain Monte Carlo Algorithms

Introduction and Motivation

This work provides a comprehensive theoretical and empirical investigation of two dimension-robust Monte Carlo algorithms—multiproposal preconditioned Crank–Nicolson (mpCN) and multiple-try preconditioned Crank–Nicolson (MTpCN). These methods are motivated by two converging trends in the MCMC literature: (i) the drive to develop samplers applicable to high (including infinite) dimensional spaces, particularly those with measures absolutely continuous with respect to a Gaussian reference; and (ii) the exploitation of parallelism through multiproposal strategies. Both mpCN and MTpCN are designed to maintain desirable geometric convergence rates, robust to ambient dimension and scalable with modern parallel hardware.

Algorithms and Theoretical Results

Algorithmic Structure

Both mpCN and MTpCN operate by simultaneously proposing a “cloud” of candidates at each Markov step, with the selection among proposals governed by carefully constructed acceptance probabilities that preserve the target measure. The chains are formulated with proposals of the form: $$\text{(For standard pCN)} \qquad X = \rho x_0 + \sqrt{1 - \rho^2} \xi, \quad \xi \sim \mu_0$$ For mpCN, an auxiliary “cloud center” $\bar{X}$ is generated as above, followed by $p$ independent proposals around $\bar{X}$. The selection mechanism is a Barker-type rule, ensuring reversibility and unbiasedness of the resulting kernel. MTpCN instead samples $p$ proposals directly from the current state, selects one via a reweighting, and then applies an additional accept-reject test reminiscent of standard multiple-try Metropolis methods.

Mixing Analysis

A central contribution is the application of weak Harris theory to analyze the convergence of these algorithms in Wasserstein distances suitable for infinite-dimensional state spaces, where total variation is uninformative. The main theoretical findings are:

• Exponential Wasserstein Contraction: Both mpCN and MTpCN admit exponential mixing rates in Wasserstein semimetrics induced by suitable Lyapunov functions. The contraction rate $\lambda$ and time-to-mixing $n_1$ are shown to be independent of both the underlying dimension and the number of proposals $p$, under mild assumptions (bounded, Lipschitz potential).
• $L^2_\mu$ Spectral Gap and Limit Theorems: Using the reversibility of the chains and the Harris-type contraction, the authors deduce the existence of a uniform spectral gap in $L^2_\mu$. This yields strong law of large numbers, central limit theorems, and explicit non-asymptotic concentration bounds (Hoeffding-type inequalities) for empirical time averages produced by the chains.
• Infinite-Proposal Limit: The analysis is extended to the case $\bar{X}$0, showing that the limits of the mpCN and MTpCN kernels both possess exponential Wasserstein contraction under weaker assumptions (potential merely Lipschitz, not necessarily bounded).

Analytical Techniques

Crucially, the paper introduces sophisticated coupling constructions capable of handling the multi-proposal, Barker-type selection. For the finite-proposal setting, these arguments yield uniform (in $\bar{X}$1) contraction rates, a non-trivial technical achievement given that acceptance probabilities in multiproposal chains can behave pathologically as $\bar{X}$2 increases. For the infinite-proposal setting, contraction is established via explicit couplings based on Radon–Nikodym derivatives between appropriately shifted proposal measures.

Numerical Experiments

The theoretical insights are corroborated by extensive computation on three Bayesian inverse problems:

1. Multiwell Posterior (Low-Dimensional, Multimodal): The mpCN chain demonstrates superior global exploration at moderate $\bar{X}$3, whereas for larger $\bar{X}$4 the chain exhibits mode trapping. The ESS diagnostic faithfully tracks this transition.
2. Polar-twist Posterior (Curved, Correlated Geometry): Both mpCN and MTpCN are robust to increasing $\bar{X}$5, with mpCN consistently outperforming MTpCN and standard pCN in mixing (ESS, MSJD) for fixed computational budget.
3. Solute Transport (High-Dimensional, PDE-Inspired): mpCN is shown to dramatically reduce burn-in time compared to both single and parallel pCN, evidenced by faster convergence of running averages and MSE for key functional estimators.

Figure 1: Multiwell likelihood geometry used to challenge global exploration properties of the chains.

Figure 2: ESS and MSJD behavior in the multiwell example; local mixing increases then saturates, revealing the shift from global to local exploration regimes.

Figure 3: mpCN traceplots in the multiwell setup; mode-jumping becomes rare as $\bar{X}$6 increases, confirming theoretical predictions.

Figure 4: Visualization of proposal clouds in mpCN and MTpCN for the polar-twist geometry, highlighting proposal diversity and exploratory behavior at various $\bar{X}$7.

Figure 5: ESS and MSJD comparison for mpCN, MTpCN, and pCN on the polar-twist, averaged over both coordinates.

Figure 6: Early-iteration traceplots for multiple observables in the solute transport model, demonstrating burn-in acceleration with mpCN.

Figure 7: Running MSE for several estimators over 50 replicates; mpCN yields lowest achievable error across all observables.

Figure 8: Posterior density and pairwise marginals for key $\bar{X}$8 coefficients in the transported matrix, sampled via high-$\bar{X}$9 mpCN.

Figure 9: ESS and MSJD for the solute transport problem, averaged over key coordinates; performance peaks at intermediate $p$0.

Figure 10: Fraction of $p$1 values yielding ESS within 25% of optimal as $p$2 increases; higher $p$3 enhances robustness to tuning.

Figure 11: mpCN versus embarrassingly parallel (EP) pCN chains: at stationarity, EP pCN achieves higher mixing; mpCN excels at burn-in.

Figure 12: Proposal point clouds for mpCN and MTpCN in polar twist, showing impact of $p$4 and $p$5 on exploration paths.

Figure 13: Advection matrix, state, and observations for toy PDE-inspired cases—illustrating complexity of the inverse problem geometry.

Figure 14: Marginal posteriors for select unknowns, sampled from 100 parallel pCN chains at equilibrium.

Contrasting Claims and Nuanced Conclusions

The work claims that multiproposal pCN methods, specifically mpCN and MTpCN, enjoy dimension- and proposal-count-robust mixing rates for bounded, Lipschitz targets, without the need for convexity/smoothness—a property not previously established for these algorithms. Furthermore, theoretical and empirical evidence indicates that multiproposal chains offer substantial advantages in warm-up (burn-in) phase and in robustness to tuning, particularly as $p$6 is increased.

In contrast to prior skepticism about the practical benefits of multiproposal schemes in stationary mixing [pozza_zanella_2025], these findings clarify that while stationary ESS of mpCN is upper bounded by that of $p$7 independent chains, the rapid elimination of initialization bias ("quick burn-in") and insensitivity to $p$8 selection present a strong advantage in computational practice, especially in high-dimensional, nonconvex models.

Implications, Limitations, and Future Directions

The practical implications are salient for Bayesian computation in PDE-constrained inverse problems, functional regression, and any statistical task requiring scalable, robust exploration of nonparametric posteriors on function spaces. From a theoretical perspective, the analytical techniques developed here—especially the proposal-cloud coupling and infinite-proposal limit arguments—are likely to be transferable to other state-of-the-art multiproposal and parallel MCMC schemes.

Limitations include the reliance on the (locally) bounded Lipschitz property of the potential for proposal-count independence—a restriction relaxed in the infinite-proposal case via alternative arguments. For truly unbounded or heavy-tailed targets, proposal-count uniformity may deteriorate as $p$9 increases, echoing fundamental limitations discussed in recent literature.

Future work is anticipated in several directions:

• Rigorous optimization and scaling of $\bar{X}$0 as a function of $\bar{X}$1 and target geometry, possibly through spectral gap analysis or Cheeger-type inequalities.
• Systematic numerical benchmarking across classes of multiproposal methods (including auto-tuned and surrogate-trajectory variants) and applications to more complex Bayesian PDE models.
• Extension of the coupling and contraction results to non-reversible or irreducible multiproposal chains, and further generalization to broader state space Markov models.

Conclusion

This study yields the first rigorous demonstration that preconditioned multiproposal MCMC algorithms, specifically mpCN and MTpCN, achieve dimension-robust, proposal-count-robust geometric convergence under weak conditions. The empirical evidence for rapid warm-up and tuning robustness, combined with the detailed analytical framework, substantially clarify both the power and the boundaries of these algorithms for large-scale inference tasks, and sets the stage for further methodological innovation and theoretical development in high-dimensional Monte Carlo sampling.