--- title: Rejection Sampling Protocol url: https://www.emergentmind.com/topics/rejection-sampling-protocol type: topic --- # Rejection Sampling Protocol Rejection sampling is a foundational Monte Carlo protocol for generating independent samples from a target probability distribution when direct sampling is infeasible but pointwise evaluation is tractable. The method constructs a proposal distribution and an acceptance rule governed by a domination constant, ensuring output samples are distributed exactly according to the target. Over decades, the technique has been generalized and optimized for high-dimensional inference, adaptive envelopes, regime-specific constraints, distributed settings, regenerative structures, and quantum domains. ## 1. Classical Rejection Sampling: Core Principles Let $f(x)$ denote the target density over domain $\mathcal{X}$, and $g(x)$ a proposal density such that $f(x) \leq M g(x)$ holds $\forall x \in \mathcal{X}$ for some finite $M \geq 1$. The rejection sampling protocol proceeds as follows [1509.07985]: 1. Draw $X \sim g(x)$ and $U \sim \mathrm{Uniform}[0,1]$ independently. 2. Accept $X$ if $U \leq f(X) / (M g(X))$, otherwise reject and repeat. The marginal acceptance rate is $1/M$; each accepted $X$ is exactly distributed according to $f/\int_{\mathcal{X}} f$. This structure places critical importance on procuring an efficient $g$ and tight envelope $M$. Poor choices (e.g., large $M$ or high variance in $f/g$) can render the protocol inefficient or even infeasible if $\sup_x f(x)/g(x) = \infty$. ## 2. Adaptive and Parsimonious Rejection Sampling Extensions ### 2.1 Adaptive Rejection Sampling (ARS) For univariate, log-concave targets $\pi(x) \propto \exp(V(x))$, ARS constructs a piecewise-exponential envelope $q_t(x)$ by maintaining a node set $S_t$ and taking the lower envelope of tangents to $V(x)$ at those nodes [1509.07985]. Sampling from $q_t(x)$ is efficient: - For each envelope update, the normalizing constant $c_t = \int q_t(x) dx$ decreases monotonically. - Rejections add support points, and asymptotically $q_t(x) \downarrow \pi(x)$, driving the acceptance rate to $1$. ### 2.2 Cheap Adaptive Rejection Sampling (CARS) and Parsimonious ARS CARS fixes the complexity per iteration by maintaining a node budget $M$ and allowing at most $M$ pieces in the envelope. After a rejection, a candidate point may swap with an existing node only if this swap decreases the normalizer and improves the acceptance rate [1509.07985]. Parsimonious ARS (PARS) generalizes the envelope update policy: a new node is only added if the local acceptance ratio falls below a user-specified threshold $\delta \in (0,1]$ [1710.04948]. This tolerance controls envelope complexity, yielding envelopes with lower computational cost while still achieving high acceptance rates; as $\delta\to1$, one recovers standard ARS behavior. ### 2.3 Generalizations Beyond Log-Concavity Various algorithms generalize ARS for non-log-concave or multimodal targets, including: - Reduced-potential adaptive RS, which isolates a tractable factor and builds adaptive piecewise-constant upper bounds for the remaining terms [1111.4942]. - Ratio-of-uniforms adaptive RS, which frames sampling as uniform draws over a bounded region in two dimensions, adaptively partitioned and tightened around $\sqrt{p(x)}$ [1111.4942]. ### 2.4 Minimax Near-Optimal Adaptive RS NNARS (Nearest-Neighbor Adaptive RS) constructs piecewise-constant upper envelopes on a regular grid, adapting the envelope as more $f(x)$ queries are collected. Theoretical lower and upper bounds show near-minimax optimality in rejection rate on classes of Hölder-regular target densities [1810.09390]. ## 3. Rejection Sampling under Structural and Communication Constraints ### 3.1 Distributed and Remote Rejection Sampling When the parameters of the target $p(x)$ are distributed across multiple parties, a leader orchestrates classical rejection sampling by requesting incrementally refined bitwise approximations of $p(x)$ from the custodians. This setting, formalized for general entanglement simulation, yields tight communication/complexity trade-offs: $\mathbb{E}[N]=C$ proposals, and overall expected communication in $O(m^2)$ bits where $m$ is the number of parties provided each local dimension is bounded [1807.06649]. ### 3.2 Channel Simulation: Greedy and Adaptive Greedy RS Channel simulation protocols utilize greedy rejection sampling (GRS) and adaptive variants (AGRS) to enable one party (Alice) to communicate just enough to allow another (Bob) to simulate from a target $Q$ using a shared proposal $P$. Expected code length is governed by $D_{KL}(Q\|P)$, and the optimality in run-time is achieved at $\exp(D_\infty(Q\|P))$ where $D_\infty$ is the Rényi infinity-divergence [2304.10407]. ## 4. High-Dimensional and Structured-State-Space Rejection Schemes ### 4.1 Ensemble Rejection Sampling Ensemble Rejection Sampling (ERS) addresses the curse of dimensionality in state-space models by operating on an extended space comprising sampled ensembles at each time step. The acceptance probability decays only polynomially (not exponentially) with sequence length $T$, yielding total cost $O(T^3)$ under uniform upper bounds for incremental likelihood ratios [2001.09188]. ### 4.2 Curvature-Based RS on Riemannian Manifolds CURS enables sampling from radial densities $p(x)\propto f(\mathrm{dist}(x_0,x))$ on Riemannian manifolds, leveraging Bishop’s volume-comparison principle to construct proposal densities respecting curvature bounds. The protocol provides acceptance probability $\Pi=Z/Z_\kappa$, and per-iteration cost scales as $O(d^3)$ for $d$-dimensional manifolds [2510.24537]. ## 5. Specialized Rejection Sampling Protocols ### 5.1 Regenerative Rejection Sampling (RRS) RRS frames RS in continuous time as a regenerative process: the process $X_t$ is held constant between Poisson events with intensity $f(x)$, and at each event $X$ is resampled from the stationary distribution $\pi(x)\propto f(x)$ [2603.12299]. This yields a Markov process with exponential convergence in total variation to $\pi$, and time-average ergodic estimators achieve an $O(1/t^2)$ bias rate—surpassing the $O(1/t)$ decay standard for classical MCMC. ### 5.2 Coupled Rejection Sampling Coupled RS generalizes RS for simulating couplings of distributions, with finite execution time variance even as the marginals approach each other in total variation. In the Gaussian setting, closed-form optimization gives bounds on the coupling probability, and ensemble versions asymptotically recover maximal couplings [2201.09585]. ### 5.3 Diffusion Rejection Sampling (DiffRS) In the context of reverse diffusion models, DiffRS wraps each reverse kernel with sample-wise rejection correction using local estimates of the density ratio from a discriminator. Acceptance probabilities are adaptively calibrated at each transition, provably tightening the KL sampling error bound and empirically enhancing the sample quality of diffusion models [2405.17880]. ## 6. Budgeted, Approximate, and Practical RS Protocols ### 6.1 Optimal Budgeted Rejection Sampling OBRS targets scenarios where the rejection budget is limited, optimizing the acceptance rule with respect to arbitrary $f$-divergences between the target and post-rejection distribution under a global acceptance-rate constraint. The optimal $a^*(x)$ is $\min\{1, \lambda p(x)/\widehat p(x)\}$, with $\lambda$ tuned via bisection for the specified budget [2311.00460]. ### 6.2 Empirical and Gradient-Refined Proposals ERS (Easy RS) leverages differentiable target densities: Gaussian mixture proposals are fitted and gradient-refined via softmax-maximum loss to empirically minimize the rejection constant, with the proposal envelope $g_\varphi$ and running bound $\widehat C$ updated iteratively from accepted/rejected samples. The high-probability correctness of this “empirical supremum RS” is ensured as established in [2310.00300]. ## 7. Partial Rejection Sampling and Hard Constraint Sampling Partial Rejection Sampling (PRS) provides a perfect sampling protocol from a product distribution conditional on hard constraints expressed as clauses (“bad events”) over localized variable scopes. Unlike classical RS (which resamples all variables), PRS resamples only the affected variables in any violated clause, guided by dependency graphs and resampling tables. In extremal instances, correctness (the output is exact in the conditional law) and polynomial efficiency are established via combinatorial and coupling arguments [2106.07744]. ## 8. Quantum and Infinite-Dimensional Rejection Sampling Quantum rejection sampling (QRS) extends RS to quantum amplitude encoding: a source state $\sum \sqrt{p_k}|k\rangle$ is converted coherently to $\sum \sqrt{q_k}|k\rangle$ via parameterized amplitude amplification and a semidefinite-program characterization. The optimal acceptance is given by a “water-filling” solution over amplitudes [1103.2774]. In infinite-dimensional or heavy-tailed regimes, specialized proposals—e.g., those for tempered Lévy processes—are devised using process decompositions and explicit density bounds for effective thinning [1806.00671]. --- **References** - [1509.07985] Adaptive Rejection Sampling with fixed number of nodes - [1710.04948] Parsimonious Adaptive Rejection Sampling - [1111.4942] Two adaptive rejection sampling schemes for probability density functions log-convex tails - [1810.09390] A minimax near-optimal algorithm for adaptive rejection sampling - [2001.09188] Ensemble Rejection Sampling - [2510.24537] Curvature-based rejection sampling - [2603.12299] Regenerative Rejection Sampling - [2201.09585] The Coupled Rejection Sampler - [2405.17880] Diffusion Rejection Sampling - [2311.00460] Optimal Budgeted Rejection Sampling for Generative Models - [2310.00300] An Easy Rejection Sampling Baseline via Gradient Refined Proposals - [1807.06649] Remote Sampling with Applications to General Entanglement Simulation - [2304.10407] Adaptive Greedy Rejection Sampling - [2106.07744] Fundamentals of Partial Rejection Sampling - [1103.2774] Quantum rejection sampling - [1806.00671] Rejection Sampling for Tempered Levy Processes Source: https://www.emergentmind.com/topics/rejection-sampling-protocol