Adaptive Weighted Rejection Sampling

• Adaptive weighted rejection sampling is a method that dynamically constructs proposal envelopes via adaptive partitioning and weighted mixtures to enhance sampling from complex densities.
• It uses majorization techniques and adaptive envelope construction to lower rejection rates, achieving acceptance rates of 80–99% for challenging target distributions.
• The approach is applied in Bayesian hierarchical models, constrained autoregressive models, and Monte Carlo event generation, with ongoing research on extensions to high-dimensional settings.

Adaptive weighted rejection sampling encompasses a class of rejection sampling methods that dynamically construct proposal envelopes—often mixtures or piecewise majorizers—to increase acceptance rates when sampling from target densities with intractable or unstandardized form. These techniques generalize classical rejection sampling by introducing adaptive partitions of the support, weighted proposals, and surrogate or data-driven weights, yielding substantial gains in both efficiency and generality, particularly for non-logconcave, multimodal, or constraint-defined targets.

1. Fundamentals and Problem Setting

Let the target density be defined as

$$f(x) = \frac{w(x)g(x)}{Z}, \qquad Z = \int_{\Omega} w(x)g(x)\,d\nu(x),$$

where $g(x)$ is a tractable base distribution, $w(x)\geq 0$ is a weight function, and $Z$ is typically unknown or intractable. This structure arises in Bayesian hierarchical models (with $g$ a prior, $w$ a likelihood or complex importance weight), constrained autoregressive models, and other settings where the normalization constant cannot be computed efficiently (Raim et al., 2024).

Classical rejection sampling draws $x \sim g$, then accepts $x$ with probability $w(x)/M$, requiring construction of $M\geq \sup_x w(x)$ for efficiency and correctness. Adaptive weighted rejection sampling seeks to choose or build proposal distributions $g(x)$0 adaptively, so accepted draws are not excessively rare and proposal evaluation/sample generation remains tractable even in complex or high-dimensional settings.

2. Adaptive Envelope Construction and Majorization

A central approach is to partition the domain $g(x)$1 into $g(x)$2 disjoint regions $g(x)$3 and define, on each $g(x)$4, a simple majorant (envelope) $g(x)$5—typically constant or log-linear upper bounds. This yields a composite proposal envelope: $g(x)$6 with associated proposal density $g(x)$7, $g(x)$8, $g(x)$9 (Raim et al., 2024).

Sampling proceeds by:

1. Drawing a region index $w(x)\geq 0$0, $w(x)\geq 0$1.
2. Drawing $w(x)\geq 0$2 restricted to $w(x)\geq 0$3.
3. Accepting $w(x)\geq 0$4 with probability $w(x)\geq 0$5.

This composite majorization underpins the “vertical weighted strips” (VWS) methodology, which is applicable even when $w(x)\geq 0$6 is not concave, and $w(x)\geq 0$7 is not an exponential family (Raim et al., 2024, Raim et al., 21 Sep 2025).

3. Adaptive Partitioning and Error/Acceptance Rates

Partition refinement is driven by the need to minimize the rejection rate. The VWS method adaptively bifurcates regions based on their contribution to an upper bound for the expected rejection probability. The rejection probability bound is given by: $w(x)\geq 0$8 where $w(x)\geq 0$9 and $Z$0 is a minorant (Raim et al., 2024).

Regions most responsible for poor tightness (as measured by $Z$1) are split, most often at their midpoint, with majorizers/minorizers recalculated. Splitting continues until the number of mixture components $Z$2 or the bound itself meets prescribed thresholds. With moderate $Z$3 (e.g., 10–50), the acceptance rate often reaches 80–99% for challenging target densities (Raim et al., 2024).

For persistent use within iterative samplers (e.g., Gibbs), "self-tuned" VWS maintains and incrementally adapts the mixture proposal as rejections accumulate, periodically pruning components with negligible contribution to the rejection bound (Raim et al., 21 Sep 2025).

4. Algorithmic Implementation and Pseudocode

The general adaptive weighted rejection sampler implements two central phases:

A. Offline proposal construction

$Z$9

B. Rejection sampling (online)

$g$0 (Raim et al., 2024)

Efficient implementation requires caching truncated base normalizations, efficient component selection (e.g., prefix sums), and numerical optimization for envelope construction.

5. Extensions, Theoretical Guarantees, and Empirical Performance

Adaptive weighted rejection sampling subsumes several specialized techniques:

• Direct Sampling with Adaptive Step Envelopes: Approximating the required marginalization density by a step-function envelope $Z$4 over an auxiliary variable enables exact sampling with controlled, adaptively reduced rejection rates. Knot selection is driven by the largest discrepancy between $Z$5 and $Z$6, with theoretical total-variation and rejection probability bounds in terms of step rectangles (Raim, 2022).
• Alternative Adaptive Schemes: Generalizations allow for piecewise exponential or convex-concave decompositions, as in adaptive rejection Metropolis and ratio-of-uniforms samplers—particularly for non-logconcave or multimodal targets (Martino et al., 2012, Martino et al., 2011). Each class maintains an envelope $Z$7, updates support points on rejection, and offers acceptance probabilities approaching unity as envelopes tighten.
• Weighted Proposals in Approximate Bayesian Computation (ABC) and SMC: Adaptive weighting, particularly data-driven kernel-based importance weights, dramatically raises acceptance rates in particle systems (Bonassi et al., 2015).

Acceptance rates, sample quality, and computational cost have been benchmarked on variety of distributions, from directional and von Mises–Fisher models (Raim et al., 2024) to complex discrete settings and real-world Gibbs samplers (Raim et al., 21 Sep 2025). Linear or log-linear envelope choices within each strip achieve considerable runtime/efficiency improvements, e.g., >95% acceptance in vMF, and effective sample size gains in Bayesian small area estimation scenarios.

6. Related Developments and Applications

Adaptive weighted rejection sampling methods have a broad impact across computational statistics:

• Constrained LLM Generation: Adaptive weighted rejection sampling, including prefix-trie adaptive pruning and unbiased importance weighting, outperforms local masking or naive rejection in constrained autoregressive generation, both empirically and in terms of theoretical sample complexity (Lipkin et al., 7 Apr 2025, Parys et al., 2 Oct 2025).
• Monte Carlo Event Generation with Surrogates: Two-stage adaptive rejection schemes incorporating neural network surrogates as fast minorants significantly accelerate unweighting in high-energy physics event generation while retaining unbiased sampling (Danziger et al., 2021).
• Minimax-Optimal Adaptive Schemes: Adaptive rejection sampling approaches matching lower bounds for expected rejections in $Z$8-Hölder function classes have been established (Achdou et al., 2018).

Theoretical results guarantee that, under mild assumptions, the acceptance rate is non-decreasing and proposals can be made arbitrarily tight at cost proportional to proposal refinement. These methods remain exact, i.e., all accepted samples come from the true target.

7. Limitations and High-Dimensional Extensions

Current adaptive weighted rejection sampling theory and practice are most mature for univariate or moderate-dimensional targets. High-dimensional applications (e.g., hyperrectangular VWS, Voronoi partitions) face challenges in exact optimization of strip bounds, efficient sampling from truncated baselines, and envelope construction (Raim et al., 2024, Raim et al., 21 Sep 2025). Direct integration into high-dimensional conditional Gibbs updating remains tractable, as the adaptive mixture-proposal framework is applied to each univariate full conditional separately. Direct multivariate extension of VWS and related algorithms is a subject for ongoing research.

In summary, adaptive weighted rejection sampling unifies and generalizes a body of envelope-based proposal refinement techniques, providing rigorous, efficient, and often near-optimal exact sampling for a wide class of weighted and constrained densities. These methodologies are increasingly central in modern computational statistics, Bayesian inference, and probabilistic generative modeling (Raim et al., 2024, Raim et al., 21 Sep 2025, Raim, 2022, Lipkin et al., 7 Apr 2025, Danziger et al., 2021, Achdou et al., 2018).