Adaptive Resampling & MCMC Kernel Selection

• Adaptive resampling and MCMC kernel selection are simulation-based methods that dynamically adjust resampling events and proposal parameters to optimize sample diversity and efficiency.
• Nested adaptation frameworks combine inner tuning of proposal scales with outer kernel selection based on metrics like ESS and ESJD to ensure fast and reliable convergence.
• Empirical benchmarks demonstrate that two-level adaptation can significantly reduce simulation time and variance, outperforming traditional fixed or auto-blocking methods in varied models.

Adaptive resampling and MCMC kernel selection refer to frameworks that dynamically optimize both resampling schedules (partitioning when to refresh particle populations) and the selection and tuning of MCMC kernels (choices of proposal types and their parameters) within modern simulation-based inference paradigms such as Markov chain Monte Carlo (MCMC), Sequential Monte Carlo (SMC), and their hybrids. These mechanisms are designed to maximize efficiency—typically via effective sample size (ESS), expected squared jump distance (ESJD), or related mixing metrics—while ensuring convergence guarantees through diminishing adaptation and containment principles.

1. Principles and Problem Setting

At the core, adaptive resampling aims to avoid particle degeneracy and excessive weight variability by triggering resampling events only when necessary, usually determined by a drop in ESS below a specified threshold. Adaptive MCMC kernel selection, in parallel, addresses the dual challenges of proposal parameter tuning and structural kernel choice across a discrete family of possible samplers or blockings. The objective is to construct a high-efficiency transition kernel, either for an MCMC or SMC setting, that achieves fast mixing per computational cost and adapts to the underlying geometry and scale of the target distribution.

This challenge is formalized in two-level adaptive schemes where, for a target distribution $\pi$, one seeks to simultaneously:

• Adapt within each kernel (tuning continuous parameters $\theta$, e.g., proposal scale or covariance),
• Adapt between kernels (selecting from a finite set of kernel structures, e.g., block versus scalar sampling, or different proposal families) (Nguyen et al., 2018).

2. Two-level Nested Adaptation Frameworks

In Nested Adaptation MCMC, also known as Auto–Adapt MCMC, adaptation occurs on two interacting levels:

1. Inner Adaptation: For a given kernel index $\iota$, proposal parameter $\theta$ is updated by a stochastic approximation scheme, typically using a step-size schedule $\gamma_k\to 0$, and a bounded adaptation map $H_\iota$ (e.g., log-scale random walk update driven by observed acceptance rates).
2. Outer Adaptation (Kernel Selection): With probability $p_n\to 0$ (ensuring $\sum_n p_n = \infty$ for persistent exploration), the kernel index $\iota$ is re-sampled, targeting the worst-mixing dimension identified through an efficiency metric $\omega_k = (N/\tau_k)/t$ (where $\theta$0 is the integrated autocorrelation time and $\theta$1 is computational time), and the corresponding kernel parameter is reset to either a default or past stored value (Nguyen et al., 2018).

This framework provably achieves ergodicity under the diminishing adaptation and containment conditions, and admits plug-and-play extension to modern kernels such as HMC, slice sampling, or auxiliary variable schemes. Empirical benchmarks demonstrate substantial outperforming of default, block, or auto-clustering MCMC via faster adaptation and optimized ESS per clock time.

Inner adaptation	Outer adaptation	Efficiency metric
Update $\theta$2	Change kernel $\theta$3	Per-dimension ESS/time ($\theta$4)
Acceptance tuning	Block/sampler swap	Worst-mixing dimension targeting

3. Adaptive Resampling Strategies

Adaptive resampling is universally guided by the (estimated) effective sample size:

$\theta$5

Resampling is performed only if $\theta$6 drops below a fraction $\theta$7 of the total particle count (commonly $\theta$8), as in ASMC (Fearnhead et al., 2010), and SMC algorithms. In settings with deterministically triggered or cost-aware resampling, this threshold can itself be adapted over time (Miasojedow et al., 2014, Laitinen et al., 28 Nov 2025). The practical effect is to concentrate computational resources when particle diversity is endangered while preserving computational efficiency.

In MCMC-maximized likelihood settings, resampling may be triggered not just within SMC but inside importance sampling augmentation routines, such as within ISReMC, where resampling can either always be performed or only under $\theta$9 for some $\iota$0 (Miasojedow et al., 2014).

4. Adaptive Kernel Selection Methodologies

Adaptive MCMC kernel selection can be framed as an online stochastic optimization problem over a joint (kernel, parameter) space. The essential procedure is:

• After each move or batch of moves, assess a kernel quality metric, such as the expected squared jumping distance (ESJD) or its Rao–Blackwellized variants,
• Accumulate empirical performance for each kernel–parameter pair $\iota$1 (e.g., via mixing diagnostics),
• Update the proposal distribution over $\iota$2 by giving higher weight to combinations achieving higher recent mixing, using an exploration–exploitation balance:

$\iota$3

where $\iota$4 is an increasing function (often linear), $\iota$5 is a jitter kernel, and $\iota$6 is the observed mixing measure (Fearnhead et al., 2010).

Asymptotically and under general conditions, the empirical distribution over $\iota$7 concentrates on the global maximizer of the mixing measure, guaranteeing identification of the best kernel and parameter in the family (Fearnhead et al., 2010).

5. Theoretical Guarantees

Adaptive schemes are governed by the requirement of diminishing adaptation (the difference between consecutive kernels vanishes in probability) and containment (the family of kernels admits a uniform drift and minorization control). Under these conditions, the adaptive chain converges in total variation to the target $\iota$8 (Roberts–Rosenthal 2007 theorem) (Nguyen et al., 2018, Fearnhead et al., 2010, Liu et al., 2024, Laitinen et al., 28 Nov 2025). Specific results from the literature include:

• Uniform ergodicity for finite adaptive kernel families and bounded adaptation maps,
• Martingale convergence theorems for adaptive MCML, showing estimates are unbiased and admit SLLN/CLT under diminishing step size and continuity (Miasojedow et al., 2014),
• Strict convexity of the asymptotic variance with respect to proposal count in i-SIR, ensuring tractable stochastic optimization of auxiliary parameter $\iota$9 (Laitinen et al., 28 Nov 2025).

6. Empirical Benchmarks and Practical Considerations

Empirical assessments across a range of models (state-space, random effects, spatial statistics, mixture models) show clear efficiency advantages:

• Nested (two-level) adaptation can reduce time to a fixed ESS by factors of 1.5–3 versus standard or auto-blocking MCMC, and by up to two orders of magnitude over all-block approaches in high-dimensional or ill-conditioned models (Nguyen et al., 2018).
• Adaptive SMC with kernel selection and tuning (ASMC and related HMC–SMC frameworks) outperforms both fixed kernel SMC and adaptive MCMC in terms of VPD, ESJD, and final acceptance rates, achieving lowest predictive variability in 5/6 mixture datasets tested (Fearnhead et al., 2010, Buchholz et al., 2018).
• Adaptive resampling and optimal L-kernel selection in SMC reduces variance by 90–99% and resampling events by 65–70% in benchmark problems; Gaussian or GMM approximations are straightforward to implement and integrate with standard SMC codebases (Green et al., 2020).
• Adaptive i-SIR with stochastic approximation of the proposal parameter $\theta$0 reliably converges to near-optimal proposal frequency, trading cost and mixing as predicted by theory (Laitinen et al., 28 Nov 2025).

7. Extensions and Advanced Adaptive Methodologies

Contemporary research has extended adaptive resampling and kernel selection to:

• Multi-kernel (mixture-of-kernels) strategies using bandit allocation and local discrepancy diagnostics, e.g., Kernel Stein Discrepancy-based bandit selection over independently running chains with cluster-based weight reweighting (Shaloudegi et al., 2018),
• Reinforcement learning-based MCMC, in which proposal design and kernel selection are cast as deterministic policy optimization in an average-reward MDP, with control over learning rates to maintain ergodicity and promote fast-mixing (Wang et al., 2024),
• Locally adaptive involutive MCMC, which adapts proposal scale per iteration based on local geometry via acceptance diagnostic-driven step-size selection, achieving irreducibility, $\theta$1 invariance, and near-optimal energy jump distance without diminishing adaptation requirements (Liu et al., 2024),
• Global-local mixture moves in ABC-MCMC, using an adaptive unit-cost ESJD criterion to determine mixture proportion and global proposal complexity, with posterior-adaptive normalizing flows for global proposals and CRN-regularized ABC-MALA for local kernels (Cao et al., 2024).

These frameworks establish adaptive resampling and kernel selection as flexible, modular approaches for simulation-based inference, grounded both in rigorous convergence theory and systematic empirical performance validation.