Solidarity of Spectral Gaps for Component-Wise Markov Chains

Abstract: Deterministic-scan and random-scan component-wise Markov chain Monte Carlo algorithms, such as Gibbs samplers and conditional Metropolis-Hastings, are popular approaches for sampling from multivariate distributions. A long-standing open question is to determine the conditions under which these algorithms have similar convergence rates. A block-wise contraction condition for the component-wise updates is used to establish a solidarity principle for the $L<sup>2$ spectral gaps of the associated Markov chains. Specifically, under this condition, the spectral gaps of the random-scan and deterministic-scan versions of the Gibbs and component-wise chains are either simultaneously positive or simultaneously zero. Moreover, the spectral gaps differ by at most polynomial factors in the number of blocks. As an application of the general results, a deterministic-scan conditional Metropolis-adjusted Langevin algorithm (MALA) for multivariate Gaussian targets is studied. The block-wise contraction condition is combined with known spectral gap bounds for the random-scan Gibbs sampler to obtain a spectral gap bound that is polynomial in dimension. The result is used to clarify how the convergence rate of the conditional MALA depends on the precision matrix of the Gaussian target and the step sizes of the block-wise MALA updates.

• The paper establishes that uniform L2 block-wise contraction guarantees solidarity between spectral gaps for deterministic and random-scan MCMC.
• It quantifies gap relationships through explicit inequalities, revealing that convergence rates suffer only polynomial inflation with increasing dimensions.
• The study applies the theory to blockwise MALA for Gaussian targets, offering practical guidance on step-size selection and asymptotic variance comparisons.

Solidarity of Spectral Gaps for Component-Wise Markov Chains

Background and Motivation

The study rigorously analyzes convergence rates for component-wise Markov chain Monte Carlo (MCMC) algorithms—specifically, deterministic-scan and random-scan update strategies. These are widely used in high-dimensional Bayesian computation, where either full-conditional (Gibbs) or inexact (e.g., conditional Metropolis–Hastings) updates are applied across blocks of variables. Understanding whether the convergence rates (and, critically, the $L^2(\Pi)$ spectral gaps that ensure geometric ergodicity) for random- and deterministic-scan versions of the same MCMC scheme are comparable has been a long-standing challenge. Prior results established such “spectral gap solidarity” only for Gibbs-type updates under restrictive settings; this paper extends these principles to the general class of reversible blockwise update chains under a quantified contractive deviation from the ideal Gibbs operator.

Main Contributions

Block-Wise Contraction and Spectral Gap Solidarity

A primary new insight is that a uniform $L^2$ block-wise contraction property—quantified by $\sup_j \lVert K_j - P_j \rVert \leq \lambda_0 < 1$ where $K_j$ is the actual update kernel and $P_j$ is the ideal Gibbs projection—guarantees the solidarity of $L^2(\Pi)$ spectral gaps for both deterministic-scan and random-scan chains. Namely, positivity of the spectral gap (and thus geometric ergodicity) for one scan scheme holds if and only if it holds for the other. The result is explicit: the gaps differ by at most a polynomial factor in the dimension (number of blocks, $d$).

Strikingly, this single blockwise contraction covers broad families of samplers—including Gibbs, Metropolis-within-Gibbs, and MALA variants—without the need for uniform ergodicity, sidestepping the intricate dependencies and scan-order pathologies that complicate non-reversible multi-block MCMC analysis. The necessity of this uniform contraction is also established: mere pointwise geometric ergodicity of conditional updates is shown insufficient via a classical two-dimensional counterexample.

Quantitative Gap Relations

Let $\eta_{\mathrm{RCW}}, \eta_{\mathrm{DCW}}$ denote the $L^2(\Pi)$ gaps for the random-scan and deterministic-scan component-wise chains. The core inequalities (with explicit dependence on $\lambda_0$ and $L^2$0) are:

• If $L^2$1, then

$L^2$2

for a universal constant $L^2$3. Conversely, under certain uniformity assumptions,

$L^2$4

• Relating to the ideal random-scan Gibbs gap $L^2$5:

$L^2$6

These represent polynomial scaling; they do not yield exponential decay in gap with dimension, so under reasonable contraction strength, gaps persist at high $L^2$7.

Asymptotic Variance Comparison and Central Limit Theorems

The authors demonstrate that not only do spectral gaps exhibit solidarity, but the associated CLT normalizing variances are also comparable. For test functions $L^2$8 with finite second moment, the difference in asymptotic variances between deterministic- and (appropriately scaled) random-scan chains is controlled by the product of their gaps:

$L^2$9

ensuring long-run statistical efficiency is not markedly lost by changing scanning schemes.

Application to Blockwise MALA for Multivariate Gaussians

To showcase practical value, the theoretical machinery is specialized to blockwise MALA targeting high-dimensional Gaussian distributions with general precision matrix $\sup_j \lVert K_j - P_j \rVert \leq \lambda_0 < 1$0. A sharp contraction bound for the MALA block kernel is obtained via conductance/isoperimetric arguments and precise Gaussian analysis, showing admissible step-size conditions and explicit rates for the deterministic-scan chain. For canonical cases—compound-symmetry and autoregressive covariance—bounding formulas clarify how between/block interact, reveal the scaling with system size, and provide step-size guidance for practitioners.

Technical Highlights

The analysis combines spectral theory of self-adjoint operators, block-contraction geometric decomposition, and probabilistic isoperimetric inequalities. Critical lemmas reduce operator analysis to contraction for $\sup_j \lVert K_j - P_j \rVert \leq \lambda_0 < 1$1, with the deterministic scan recast as a composition over the contracted orthogonal complements—facilitating telescoping estimates of the norm decay.

The necessity of the uniform contraction property is sharply illustrated: geometric ergodicity of each conditional does not guarantee any gap for the global chain without the contraction being uniformly below unity.

In the MALA example, the subtle control of the conductance for nontrivial block proposals underlines the care required in high dimensions—the blockwise spectral gap is polynomial in block size, not exponentially small under appropriate step-size selection.

Implications and Future Directions

The findings provide a robust theoretical justification for the widespread empirical observation that scan order (deterministic versus random) does not dramatically affect ergodicity for well-controlled componentwise MCMC, if local kernels are uniformly strongly mixing toward their respective conditional targets. This extension to the general blockwise setting (beyond Gibbs or uniform ergodicity) meaningfully broadens the scope of rigorous convergence analysis in MCMC theory.

Practically, this informs the MCMC design for large-scale Bayesian models: provided reversible local updates are reasonably contractive, either systematic or random scan can be chosen (even for highly blocked updates) without fundamental loss of convergence guarantees—importantly even in high dimensions, with only polynomial inflation of convergence rates.

Theoretically, these results suggest numerous potential research avenues:

• Sharp characterization of contraction constants for specific classes of block Metropolis–Hastings updates in non-Gaussian models.
• Study of the impact of more complex scan strategies (non-uniform random, adaptive order) on gap relations.
• Tradeoffs between statistical efficiency (as CLT asymptotic variance) and computational efficiency under hybrid and partially collapsed schemes.
• Extension to continuous-time limits or inhomogeneous alternate scan variants.

Conclusion

This work establishes a broad, quantitative solidarity principle for the $\sup_j \lVert K_j - P_j \rVert \leq \lambda_0 < 1$2 spectral gap between systematic and random scan variants of componentwise reversible Markov chains under a global contraction condition. The findings, supported by rigorous spectral and probabilistic analysis and concretely realized in canonical MALA settings, clarify both theoretical limitations and practical guidance for scalable MCMC design (2604.23229).