Block Pseudo-Marginal Metropolis–Hastings

• Block pseudo-marginal Metropolis–Hastings is an MCMC method that partially updates auxiliary variables to induce correlation among successive likelihood estimators.
• The approach reduces the variance of the log-likelihood ratio by controlling noise, thereby mitigating the sticky behavior of chains caused by independent likelihood estimates.
• Variants incorporating Crank–Nicolson updates and block partitioning provide practical tuning guidance and significant efficiency gains in high-dimensional models.

Block pseudo-marginal Metropolis–Hastings is a pseudo-marginal Metropolis–Hastings scheme in which the auxiliary variables $u$ used to construct an unbiased estimator of an intractable likelihood are updated only partially between iterations. In the standard pseudo-marginal construction, one samples on an extended space $(\theta,u)$ so that the $\theta$-marginal is the desired posterior. In block pseudo-marginal Metropolis–Hastings, the auxiliary variables are split into blocks and only a subset of blocks is refreshed at each step, while the remaining randomness is retained. This induces correlation between successive likelihood estimates, reduces the variability of the Metropolis–Hastings ratio, and is intended to mitigate the sticky behavior that arises when independent likelihood estimates are noisy (Dahlin et al., 2015).

1. Extended-state formulation and exactness

The basic pseudo-marginal setting starts from a posterior

$$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$

where the likelihood $p_\theta(y)=p(y\mid\theta)$ is analytically intractable or too costly to evaluate point-wise. The pseudo-marginal device is to construct an unbiased, non-negative likelihood estimator

$$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$

with auxiliary random numbers $u$, and then define an extended target

$$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$

where a concrete choice is

$$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$

so that marginalising out $u$ recovers $(\theta,u)$0 (Dahlin et al., 2015).

With proposal $(\theta,u)$1 and an auxiliary proposal $(\theta,u)$2, the Metropolis–Hastings acceptance probability on the extended space is

$(\theta,u)$3

What changes in block pseudo-marginal methods is not the extended target and not the exactness argument, but the proposal mechanism for the auxiliary randomness (Dahlin et al., 2015).

A closely related exact construction appears in exact active subspace Metropolis–Hastings. There, an unbiased estimator $(\theta,u)$4 of a marginal $(\theta,u)$5 is built by importance sampling over inactive variables $(\theta,u)$6, and the estimator at the current state is reused until a move is accepted. The chain’s state is effectively $(\theta,u)$7, and the old random draws are retained when a proposal is rejected. The paper identifies the original Monte Carlo within Metropolis implementation as biased, and the corrected grouped independence MH construction as asymptotically exact (Schuster et al., 2017).

2. Why independent refresh performs poorly

The main reason block pseudo-marginal methods are used is that the standard pseudo-marginal proposal refreshes all auxiliary variables independently at every iteration. Then $(\theta,u)$8 carries independent noise from step to step. When the variance of the log-likelihood estimator is high, the chain tends to become sticky: occasionally a very large likelihood overestimate is drawn, the chain jumps to that “lucky” point, and then tends to remain there because subsequent likelihood estimates are typically lower. Reducing this effect by increasing the number of particles or importance samples lowers the variance of $(\theta,u)$9, but increases computational cost (Dahlin et al., 2015).

This mechanism sits within a broader pseudo-marginal comparison theory. The asymptotic variance of the pseudo-marginal algorithm is always at least as large as that of the marginal algorithm, the pseudo-marginal chain always has a lower expected acceptance rate than the ideal Metropolis–Hastings chain, and geometric ergodicity is closely tied to properties of the weight distribution $\theta$0 in $\theta$1 (Andrieu et al., 2012). These results do not identify blocking as the unique remedy, but they make clear that likelihood-estimator noise is a structural source of inefficiency.

Large-sample asymptotics sharpen the same point. In the standard pseudo-marginal regime, analyses of the log-likelihood error $\theta$2 lead to the familiar recommendation that the variance of the log-likelihood estimator should be of order $\theta$3, but that recommendation is expensive when all randomness is refreshed independently (Schmon et al., 2018). Block pseudo-marginal methods target the same variance–cost problem by controlling correlation instead of relying only on a larger Monte Carlo budget.

3. Core block mechanism

In block pseudo-marginal Metropolis–Hastings, the auxiliary variables are partitioned as

$\theta$4

At each iteration, only a subset of blocks is updated, while the remaining blocks are kept fixed. This induces partial correlation in the likelihood estimator between iterations and can be tuned to control variance versus correlation (Dahlin et al., 2015).

The conceptual pattern is uniform across several constructions. The state of the chain includes both the parameter and a block of auxiliary randomness; a proposal updates the parameter and some or all of the random numbers used to produce the likelihood estimator; if the proposal is rejected, the current auxiliary block is retained. In exact active subspace Metropolis–Hastings, the state includes $\theta$5 and the block of importance-sampling draws $\theta$6; a new block is generated for a proposal $\theta$7, while the current block is reused if the move is rejected (Schuster et al., 2017). In the signed pseudo-marginal method for doubly intractable models, the random numbers are explicitly partitioned into $\theta$8 blocks $\theta$9, and one block is updated per iteration (Yang et al., 2022).

Construction	Block structure	Update pattern
Exact active subspace MH	$$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$0 importance-sampling draws	New block for proposed $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$1; current block retained if rejected
Signed block PMMH with block-Poisson estimator	$$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$2	Update only one block $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$3 per iteration
Augmented correlated PMMH for diffusions	$$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$4	Update local blocks $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$5

For doubly intractable problems, the block-Poisson estimator is written as

$$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$6

with one set of random variables per block. The Markov chain state is $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$7, and at each iteration only one block $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$8 is resampled. Under mild assumptions, if only one block is updated per iteration, the correlation $$\pi_\theta(\theta)=\frac{p(\theta)p_\theta(y)}{p(y)},$$9 between successive log-likelihood estimates satisfies approximately

$p_\theta(y)=p(y\mid\theta)$0

That is a literal block pseudo-marginal correlation mechanism (Yang et al., 2022).

4. Correlated pseudo-marginal updates as a global block method

A closely related construction updates all auxiliary variables at every iteration, but only partially. The Crank–Nicolson proposal

$p_\theta(y)=p(y\mid\theta)$1

or equivalently

$p_\theta(y)=p(y\mid\theta)$2

preserves $p_\theta(y)=p(y\mid\theta)$3 and induces correlation $p_\theta(y)=p(y\mid\theta)$4 between $p_\theta(y)=p(y\mid\theta)$5 and $p_\theta(y)=p(y\mid\theta)$6. The paper interprets this as a global block update: the whole vector $p_\theta(y)=p(y\mid\theta)$7 is one block, and the update is partial rather than independent (Dahlin et al., 2015).

This perspective yields a direct relation between correlated pseudo-marginal and block pseudo-marginal methods. Block methods keep many random numbers unchanged to induce correlation. The Crank–Nicolson method replaces a fraction of the random numbers through AR(1) dynamics and keeps the rest correlated with the past. Both achieve correlated likelihood estimators by only partially updating $p_\theta(y)=p(y\mid\theta)$8 (Dahlin et al., 2015).

The paper also gives explicit tuning guidance. For the recommended regime $p_\theta(y)=p(y\mid\theta)$9, corresponding to variance of the log-likelihood roughly $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$0–$$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$1, the optimal one-dimensional parameter is $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$2, with corresponding acceptance rate for the $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$3-chain around $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$4–$$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$5. In the full model, a pragmatic conclusion from the experiments is that $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$6 often works well. In the Gaussian IID example, the best integrated autocorrelation time was obtained for $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$7. In the stochastic volatility example, the integrated autocorrelation time was minimised around $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$8, and using $$\widehat{p}_\theta(y;u),\qquad \mathbb{E}[\widehat{p}_\theta(y;u)] = p_\theta(y),$$9 gave roughly a $u$0 reduction in IACT relative to the independent-$u$1 case (Dahlin et al., 2015).

The correlated pseudo-marginal large-$u$2 analysis goes further. With

$u$3

the correlated pseudo-marginal method can be tuned so that the variance of the log-likelihood ratio estimator is order $u$4 as $u$5 whenever $u$6. Numerical examples report that the efficiency of computations is increased relative to the standard pseudo-marginal algorithm by more than 20 fold for values of $u$7 of a few hundreds to more than 100 fold for values of $u$8 of around $u$9–$$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$0 (Deligiannidis et al., 2015). This suggests that block schemes that engineer comparable correlation in the underlying random numbers can target the same regime.

5. Variants and applications

Block pseudo-marginal ideas appear in several distinct application areas, often with different meanings of “block”.

For doubly intractable posteriors, the signed pseudo-marginal method introduces an auxiliary variable $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$1 so that the intractable reciprocal $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$2 is replaced by $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$3, and then uses a block-Poisson estimator to produce an unbiased estimator of $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$4 with $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$5. Because the estimator can be negative, the algorithm targets the absolute value of the estimated posterior and uses an importance sampling correction

$$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$6

where $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$7 is the sign of the estimator (Yang et al., 2022). In this setting, block pseudo-marginal updating is not only a mixing device but also a way to make a signed unbiased estimator compatible with correlated pseudo-marginal methodology.

For partially observed diffusion processes, augmented correlated pseudo-marginal Metropolis–Hastings augments the state with latent states at the observation times $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$8 and partitions the auxiliary variables by inter-observation interval: $$\bar{\pi}(\theta,u)=\frac{\exp(-\Phi_\theta(u))}{c}\,\mathcal N(u;0,I_{N_u}),$$9 The algorithm updates local blocks $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$0, avoids resampling altogether, and uses Crank–Nicolson updates within the relevant blocks. This gives a block correlated pseudo-marginal structure in which each block interacts only with local factors of the likelihood estimator (Golightly et al., 2020). The paper reports substantial increases in overall efficiency compared to competing methods, and in the examples $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$1 and $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$2 were typically optimal or close to optimal (Golightly et al., 2020).

In exact active subspace Metropolis–Hastings, the target on the active variables is

$$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$3

and the estimator

$$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$4

is reused until the chain leaves the current state. The paper does not use the term block pseudo-marginal, but the state augmentation by a block of importance-sampling draws and the rule “reuse the estimator at the current state” place it in the same exact approximate family (Schuster et al., 2017).

A further extension appears in robust particle pseudo-marginal methodology. The Frankenfilter is a partially alive particle filter with lower and upper bounds on the number of simulations and an unbiased likelihood estimator. The paper states that the Frankenfilter could also be applied within a correlated pseudo-marginal MCMC scheme and explicitly notes that its one-step algorithms could potentially be employed within the augmented correlated pseudo-marginal approach of Golightly and Sherlock (Sherlock et al., 30 Jan 2026). This suggests a direct route to block particle pseudo-marginal constructions in models with zero conditional likelihood contributions.

6. Theory, tuning, and limitations

Several theoretical strands delimit what block pseudo-marginal methods can and cannot achieve. Large-sample asymptotics show that, after rescaling, the pseudo-marginal chain converges to a limiting chain in which the target is Gaussian and the log-likelihood noise is additive Gaussian and independent of the parameter. In that limit, optimal proposal scaling and optimal noise variance can be studied directly. For $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$5, the numerically optimal $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$6 is about $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$7; for $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$8, about $$\Phi_\theta(u)= -\big(\log \widehat{p}_\theta(y;u)+\log p(\theta)\big),$$9; for $u$0, about $u$1; and as $u$2, $u$3 (Schmon et al., 2018). These values are not block-specific, but they provide target regimes for block or correlated pseudo-marginal tuning.

At the same time, pseudo-marginal comparison theory imposes a ceiling: the asymptotic variance of the pseudo-marginal algorithm is always at least as large as that of the marginal algorithm, and improved likelihood estimation makes the pseudo-marginal asymptotic variance converge to that of the ideal chain (Andrieu et al., 2012). Blocking can improve performance relative to independent-refresh pseudo-marginal implementations, but not beyond the corresponding exact-likelihood Metropolis–Hastings kernel.

The main practical limitations recur across the literature. The Crank–Nicolson analysis assumes a Gaussian approximation for the log-likelihood estimator and often a variance $u$4 that is effectively independent of $u$5; the relation between the high-dimensional tuning parameter $u$6 and the one-dimensional surrogate $u$7 is model-dependent; over-correlation makes the auxiliary chain extremely persistent; under-correlation reverts to standard pseudo-marginal Metropolis–Hastings; and exactness still requires an unbiased likelihood estimator (Dahlin et al., 2015). In signed block pseudo-marginal methods, many negative estimates make the denominator in the sign-corrected estimator small and inflate variance, so tuning must keep the probability of a positive estimate far from $u$8 (Yang et al., 2022).

Subgeometric convergence theory points to the same bottleneck in a different language: convergence rates depend on the tail of the weight distribution under the extended target, and heavy-tailed weights are a source of slow pseudo-marginal convergence (Andrieu et al., 2021). A plausible implication is that the most effective block designs are those that reduce the variability of the acceptance ratio without creating unstable tails in the estimator itself.

Taken together, these results support a unified description. Standard pseudo-marginal Metropolis–Hastings regenerates all random numbers $u$9 independently at every iteration. Block pseudo-marginal Metropolis–Hastings keeps some of that randomness fixed, or updates it only partially, so that successive likelihood estimates are correlated. Correlated pseudo-marginal methods based on Crank–Nicolson moves can be interpreted as a single global block updated partially; explicit block schemes update only selected components or selected time intervals; and augmented constructions add latent variables so that the natural factorization of the estimator becomes blockwise. The unifying objective is to control the correlation structure of the likelihood estimator across iterations while preserving the exact pseudo-marginal target.