Ensemble Rejection Sampling (ERS)

• Ensemble Rejection Sampling (ERS) is a rejection-sampling framework that replaces a single proposal with a batch of candidates to enable exact posterior simulation in nonlinear, non-Gaussian state-space models.
• ERS achieves exactness by performing acceptance correction on an extended space that includes ensemble variables and indexed trajectories, thus mitigating exponential acceptance decay.
• Empirical and theoretical results show that with ensemble sizes scaled to the sequence length, ERS maintains robust acceptance probabilities and transitions the cost from exponential to polynomial growth.

Searching arXiv for Ensemble Rejection Sampling and closely related papers. arxiv_search query: "Ensemble Rejection Sampling state-space models channel simulation distributed compression coupled rejection sampler" Ensemble Rejection Sampling (ERS) is a rejection-sampling framework in which a single proposal is replaced by an ensemble or batch, and exactness is recovered by performing accept/reject correction on an extended space. It was introduced for exact posterior simulation of latent state trajectories in nonlinear non-Gaussian state-space models, where ensembles of candidate states at each time induce $N^T$ candidate paths and the expected cost, under regularity conditions, grows cubically with sequence length rather than exponentially as in standard rejection sampling (Deligiannidis et al., 2020). Subsequent work recast the same ensemble principle for channel simulation and distributed compression, and extended it to coupled sampling schemes with asymptotically maximal coupling behavior (Phan et al., 7 Oct 2025, Corenflos et al., 2022).

1. Original formulation in state-space models

In its original form, ERS targets the posterior distribution of a latent path $X_{1:T}$ given observations $Y_{1:T}=y_{1:T}$. The target density is

$$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$

with $\mu$ the initial density, $f$ the transition density, $g$ the observation density, and $Z=p(y_{1:T})$ the marginal likelihood. ERS introduces proposal densities $q_t(x_t)$ and the associated incremental importance weights

$$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$

Rather than proposing one complete trajectory from $X_{1:T}$0, ERS draws $X_{1:T}$1 candidates at each time point, thereby defining a random grid of $X_{1:T}$2 possible trajectories, and then samples one path from a discrete approximation $X_{1:T}$3 supported on that grid (Deligiannidis et al., 2020).

The key approximation is

$X_{1:T}$4

where $X_{1:T}$5 is the path weight of the trajectory indexed by $X_{1:T}$6. The induced proposal is therefore not a single factorized path proposal, but a proposal assembled from ensembles of state samples. This is the defining structural feature of ERS: the proposal distribution is built from exponentially many candidate paths while remaining algorithmically accessible through dynamic programming.

This construction directly addresses the standard pathology of pathwise rejection sampling in hidden Markov and Feynman–Kac models. With a naively factorized proposal, the global rejection constant typically scales as a product of per-time-step bounds, and average acceptance deteriorates exponentially in $X_{1:T}$7. ERS replaces that single-path proposal with a random, ensemble-defined approximation that is much closer to the posterior path law.

2. Extended-space exactness and algorithmic mechanism

ERS is exact because the rejection step is performed on an extended space that contains the ensemble variables, the selected index path, and the selected trajectory. The extended proposal is

$X_{1:T}$8

and the extended target is

$X_{1:T}$9

The central identity is

$Y_{1:T}=y_{1:T}$0

which implies that rejection sampling on the extended space yields an accepted marginal trajectory exactly distributed according to $Y_{1:T}=y_{1:T}$1 (Deligiannidis et al., 2020).

Operationally, ERS proceeds in three stages. First, it generates the ensembles $Y_{1:T}=y_{1:T}$2. Second, it samples an index path $Y_{1:T}=y_{1:T}$3 from the discrete approximation on the random grid. Although the grid contains $Y_{1:T}=y_{1:T}$4 paths, this index path can be drawn in $Y_{1:T}=y_{1:T}$5 using forward-filtering backward-sampling on an embedded $Y_{1:T}=y_{1:T}$6-state hidden Markov model. Third, it computes a path-independent upper bound $Y_{1:T}=y_{1:T}$7 and accepts the selected trajectory with probability

$Y_{1:T}=y_{1:T}$8

The exactness result is nonasymptotic: for any $Y_{1:T}=y_{1:T}$9, the algorithm returns an exact sample from the posterior. For fixed $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$0, the average acceptance probability $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$1 satisfies $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$2 as $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$3 (Deligiannidis et al., 2020). ERS is therefore not an approximate particle smoother, nor an MCMC kernel whose exact target is reached only in the limit; it is an exact sampler with an ensemble-defined proposal.

3. Acceptance behavior, complexity, and empirical regimes

The main complexity result is derived under two-sided regularity bounds $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$4, with

$$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$5

Under these assumptions, ERS satisfies

$$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$6

If $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$7 for some $$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$8, then

$$\pi(x_{1:T})=\frac{\gamma(x_{1:T})}{Z}, \qquad \gamma(x_{1:T})=\mu(x_1)g(y_1\mid x_1)\prod_{t=2}^T f(x_t\mid x_{t-1})g(y_t\mid x_t),$$9

Because proposal generation and bound computation cost $\mu$0, choosing $\mu$1 yields expected cost $\mu$2 per exact posterior sample, rather than exponential cost in $\mu$3 for ordinary rejection sampling (Deligiannidis et al., 2020).

The original paper illustrates this scaling on several models. For a rare-event path-conditioning problem with $\mu$4, $\mu$5, and $\mu$6, estimated average acceptance probabilities remained stable across $\mu$7 when $\mu$8 was fixed: for $\mu$9, the reported acceptances were approximately $f$0 for $f$1; $f$2 for $f$3; and $f$4 for $f$5. For a nonlinear autoregressive model with $f$6, $f$7, and $f$8, one realization with $f$9 produced estimated acceptances of about $g$0 for $g$1, $g$2 for $g$3, and $g$4 for $g$5. In a stochastic volatility application to S&P 500 data with $g$6, $g$7, $g$8, and $g$9, the reported acceptance at $Z=p(y_{1:T})$0 was approximately $Z=p(y_{1:T})$1 (Deligiannidis et al., 2020).

These results delimit the practical profile of ERS. The method replaces exponential degradation by polynomial scaling, but it does not eliminate the need for good proposal design or sufficiently tight upper bounds. Its favorable complexity statement is contingent on bounded incremental weights and on ensemble sizes that typically grow with path length.

4. Finite-batch ERS for channel simulation and distributed compression

A later formulation of ERS shifts the setting from latent-state smoothing to channel simulation and distributed matching. Here each rejection step is performed over a batch of $Z=p(y_{1:T})$2 proposals. With proposal $Z=p(y_{1:T})$3, target $Z=p(y_{1:T})$4, and weights $Z=p(y_{1:T})$5, the algorithm selects a within-batch candidate via a Gumbel-max or exponential-race rule,

$Z=p(y_{1:T})$6

computes

$Z=p(y_{1:T})$7

and accepts the batch candidate if

$Z=p(y_{1:T})$8

For $Z=p(y_{1:T})$9, this reduces exactly to standard rejection sampling; for large $q_t(x_t)$0, the within-batch importance selection becomes increasingly dominant, and ERS approaches finite-batch importance-sampling or Gumbel-max behavior while preserving exactness (Phan et al., 7 Oct 2025).

This formulation makes precise the sense in which ERS interpolates between ordinary rejection sampling and finite-sample importance selection. In the channel-simulation setting it yields a coding rate

$q_t(x_t)$1

while the same paper also shows that standard rejection sampling itself can achieve

$q_t(x_t)$2

The main theoretical advance, however, is a distributed matching lemma for ERS, positioned as the rejection-sampling counterpart to the Poisson Matching Lemma of Li and Anantharam (2021) and as a generalization of the importance matching lemma of Phan et al. (2024). The paper gives matching bounds of the form

$q_t(x_t)$3

with correction terms $q_t(x_t)$4 as $q_t(x_t)$5, typically at rate $q_t(x_t)$6 under mild assumptions (Phan et al., 7 Oct 2025).

The same work applies ERS to distributed compression, including synthetic Gaussian sources and MNIST-based distributed image compression. Its practical claim is not merely that ERS is exact, but that it combines finite proposal sets, explicit termination, and matching guarantees close to Poisson-process methods, which is especially relevant when decoder-side likelihood ratios are learned rather than analytically available.

5. Coupled extensions and neighboring exact rejection frameworks

ERS has also been extended to the coupling of two target distributions. The coupled rejection sampler of Jacob, Lindsten, and Schön is modified so that each marginal rejection step is replaced by an ensemble rejection sampler, while coupling is maintained across the two ensembles. The resulting ensemble coupled rejection-sampling method samples $q_t(x_t)$7 coupled proposal pairs $q_t(x_t)$8, computes marginal importance weights $q_t(x_t)$9 and $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$0, samples a pair of indices $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$1 from a maximal coupling of the two categorical weight distributions, and then uses a shared uniform variable for the two marginal ERS acceptance tests. For any $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$2, the returned marginals are exact,

$$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$3

and if $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$4 denotes the number of ensemble attempts,

$$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$5

When $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$6 and $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$7 have no atoms, the diagonal mass satisfies

$$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$8

so a duplicated common proposal with $$w_1(x_1)=\frac{\mu(x_1)g(y_1\mid x_1)}{q_1(x_1)}, \qquad w_t(x_{t-1},x_t)=\frac{f(x_t\mid x_{t-1})g(y_t\mid x_t)}{q_t(x_t)},\quad t\ge 2.$$9 asymptotically recovers the maximal coupling probability (Corenflos et al., 2022).

This coupled construction clarifies an important point about ERS. The ensemble mechanism is not only a device for improving marginal acceptance; it can also be used to improve synchronized selection and meeting probabilities in coupled samplers. In the coupled setting, the ensemble layer attacks both low acceptance and low coordination between marginals.

A separate but adjacent line of work studies exact adaptive rejection without using ERS terminology. Constrained Adaptive Rejection Sampling (CARS) for autoregressive LLMs preserves the exact constrained distribution while adaptively ruling out invalid continuations by recording them in a trie and subtracting their probability mass from future draws. Its proposal $X_{1:T}$00 is a dynamically pruned version of the original language-model distribution, and acceptance rates improve monotonically as the cache of known-invalid prefixes grows (Parys et al., 2 Oct 2025). This is not ERS in the original sense, but it illustrates a closely related design principle: rejected samples can be amortized structurally rather than discarded.

6. Theoretical limits, terminology, and open directions

A recent theoretical analysis isolates a core ERS-like problem: how many random bits must be revealed, adaptively, to determine the accept/reject event in a rejection sampler when the threshold depends on many coupled variables. In the model considered, one samples

$X_{1:T}$01

and accepts when

$X_{1:T}$02

for an increasing $X_{1:T}$03. The quantity

$X_{1:T}$04

measures the minimax expected bit complexity of the accept/reject decision. The paper proves

$X_{1:T}$05

and shows that a naive alternating bit-revelation strategy achieves the quadratic upper bound, while an adversarial monotone target enforces the linear lower bound (Langevin et al., 29 Sep 2025).

Although that work does not study ERS by name, it is directly relevant to any ERS-style method whose rejection decision depends on many coupled coordinates. It isolates the information-theoretic cost of certifying a joint accept/reject event under lazy randomness revelation. A plausible implication is that even when the proposal mechanism is sophisticated, the acceptance test itself can retain an irreducibly dimension-dependent cost.

The literature also contains a terminological ambiguity. In “A Dyadic Simulation Approach to Efficient Range-Summability,” the acronym “ERS” denotes efficient range-summability rather than ensemble rejection sampling. That paper develops dyadic simulation trees and specialized rejection samplers for conditional split distributions, but the acronym refers to a different problem class centered on polylogarithmic-time range sums of i.i.d. random variables (Meng et al., 2021). This ambiguity is bibliographically relevant because searches for “ERS” on arXiv may retrieve conceptually unrelated material.

The present state of the subject therefore combines strong exactness results with nontrivial open questions. In the original state-space formulation, ERS delivers exact posterior draws with polynomial expected cost under strong regularity conditions (Deligiannidis et al., 2020). In distributed matching, it achieves finite-batch exactness and matching guarantees close to Poisson-process methods (Phan et al., 7 Oct 2025). In coupled sampling, it asymptotically recovers maximal coupling behavior under suitable proposals (Corenflos et al., 2022). At the same time, both proposal quality and acceptance certification remain central bottlenecks, and the gap between known lower and upper complexity bounds for multi-variable rejection decisions indicates that the optimal scaling of ensemble-based exact rejection methods is not yet fully resolved (Langevin et al., 29 Sep 2025).