---
title: Verbalized Rejection Sampling
url: https://www.emergentmind.com/topics/verbalized-rejection-sampling-vrs
type: topic
---

# Verbalized Rejection Sampling

Verbalized Rejection Sampling (VRS) is a framework that generalizes and operationalizes classical rejection sampling within new computational settings, particularly where access to the internal mechanics of the generator is unavailable or undesirable. VRS encompasses both algorithmic and natural-language instantiations, enabling correct, tractable, and often model-agnostic sampling from complex distributions in probabilistic inference, machine learning, combinatorics, and controlled natural language generation. Its core principle is to decouple proposal generation and acceptance decisions—potentially assigning these to different computational or cognitive modules, including large language models (LLMs)—while maintaining rigorous statistical guarantees.

## 1. Theoretical Foundations and Algorithmic Principles

VRS extends the classical rejection sampling paradigm, which samples from a proposal distribution \( Q \) and accepts samples with probability proportional to the target-proposal density ratio \( P(x)/MQ(x) \). In VRS, the accept/reject logic can be verbalized or modularized, with the crucial requirement that, conditioned on acceptance, the sample distribution matches the target \( P \). This decoupling enables a range of novel algorithmic strategies:

- **Natural-Language VRS:** The accept/reject criterion is embedded in prompts for LLMs, which are instructed (in plain language) to reason about and decide on acceptance, rather than relying on model-internal randomness or precise arithmetic [2506.09998].
- **Augmented Sampling:** The chain of rejected proposals prior to acceptance can be explicitly modeled and used for efficient data augmentation in MCMC, with the joint sampling process involving both accepted and rejected proposals, and resulting in tractable, normalized posteriors even for doubly-intractable models [1406.6652].
- **Trade-off Control:** VRS allows interpolation between computational cost and approximation fidelity, as in Variational Rejection Sampling, by tuning the selectivity of the acceptor (e.g., by varying an acceptance threshold) [1804.01712][2309.14612].

For a target distribution \( P(x)=\tilde{P}(x)/Z \), and proposal \( Q(x) \), classical VRS acceptance probability is

\[
A(x) = \frac{\tilde{P}(x)}{MQ(x)},
\]
where \( M \) satisfies \( MQ(x) \ge \tilde{P}(x) \ \forall\, x \).

In variational and data-driven settings, acceptance can be further generalized to

\[
a(x) = \sigma( \log \tilde{P}(x) - \log Q(x) + T ),
\]

where \( \sigma \) is the logistic/sigmoid function and \( T \) is a fidelity-control parameter.

## 2. VRS in Natural-Language and Black-Box Generative Models

Recent work has demonstrated the efficacy of VRS as a black-box augmentation for LLMs, especially in tasks where models nominally “know” target distributions but induce bias when tasked to simulate them directly. In this context [2506.09998]:

- **Prompting Framework:** The sampler presents the LLM with (i) a textual description of the target distribution, (ii) the proposal distribution, and (iii) a proposed sample, and asks for an accept (“T”) or reject (“F”) decision.
- **Empirical Findings:** Across models and prompt variants, VRS consistently halves the total variation distance between the empirical sample distribution and the target, outperforming both direct sampling and prompt engineering heuristics.
- **Theoretical Guarantees:** If LLM decisions approximate the correct acceptance probabilities up to some bounded error \( c \), then 
\[
\mathrm{TV}(\tilde{P}, P) \leq \frac{Mc}{1 - Mc},
\]
and, under mild assumptions, the total variation can be strictly less than that induced by direct simulation bias.

VRS thus provides a principled mechanism for reducing sampling bias without requiring internal access, bespoke random number generators, or heavy prompt tuning.

## 3. VRS in Variational Inference and Statistical Learning

Variational Rejection Sampling (VRS) has emerged as a tool to address the limitations of parametric approximation in variational inference. Standard variational inference matches the approximate posterior \( q_\phi(z) \) to the true posterior \( p_\theta(z|x) \) by optimizing an evidence lower bound (ELBO), but struggles for complex or multi-modal targets.

- **Flexible Variational Family:** VRS constructs a new *nonparametric* family given by
\[
r_{\theta,\phi}(z) = \frac{q_\phi(z) a_{\theta,\phi}(z)}{Z_r},
\]
where \( a_{\theta,\phi}(z) \) is a soft or hard accept/reject function [1804.01712][2309.14612].
- **Computational-Statistical Trade-off:** By varying the acceptance threshold, VRS interpolates between the original variational family and the exact posterior, balancing computational effort against approximation quality.
- **Gradient Estimation:** Recent advances employ the reparameterization trick for effective gradient estimation, yielding low-variance, pathwise gradient estimators, and enabling large-scale optimization [2309.14612].
- **Empirical Validity:** VRS improves held-out log-likelihoods and ELBOs over standard parametric or flow-based baselines, and approaches full posterior fidelity in practical scenarios (e.g., VAEs, Gaussian process classification).

## 4. Combinatorial and Constrained Generation

VRS also manifests in analytic and algorithmic sampling for combinatorial structures, and in controlling the output space of generative models (language or otherwise) under constraints:

- **Analytic Samplers:** By extending Boltzmann sampling, analytic rejection samplers allow exact or approximate-size sampling with only approximate generating function evaluations, correcting with a rejection step [1304.1881].
- **Partial Rejection Sampling:** In structured spaces (e.g., constrained graphs), partial rejection sampling resamples only the components associated with violated constraints, offering exponential speedups while preserving perfect conditional distributions [2106.07744].
- **Controlled Language Generation:** Adaptive weighted rejection sampling leverages VRS for efficient constrained decoding in LMs, reducing constraint evaluation cost and correcting local myopic biases inherent to token-masking approaches [2504.05410]. This framework supports arbitrary (even black-box) constraints, and integrates with sequential Monte Carlo methods for global distributional fidelity.

## 5. Statistical Inference with Data Augmentation and Adaptive Strategies

In scenarios where model parameters or normalizing constants are intractable, VRS enables tractable Bayesian inference and efficient sampling by data augmentation:

- **Auxiliary Variables:** Augmenting the observed data with rejected samples—modeled as explicit latent variables—eliminates intractable normalizing constants and facilitates efficient MCMC parameter updates [1406.6652].
- **Adaptive Proposals:** Recent minimax-optimal adaptive rejection sampling algorithms inform VRS strategies for proposal updating, envelope construction, and efficient sample generation under black-box or noisy feedback assumptions [1810.09390].

These techniques have been deployed in practice for rare event estimation, nonparametric density modeling with Gaussian processes, and Bayesian analysis on manifolds.

## 6. Limitations, Open Challenges, and Broader Impact

While VRS substantially expands the set of feasible, unbiased, or nearly-unbiased sampling methods in both algorithmic and black-box settings, certain limitations and research avenues remain:

- **Efficiency Constraints:** For proposal distributions highly discordant with the target, rejection rates can be high. Adaptive strategies and informed proposal selection/orchestration are key ongoing areas of research.
- **Scope of Theoretical Guarantees:** Most detailed theoretical guarantees and experimental evidence concern low-dimensional or “verbalizable” targets (e.g., Bernoulli distributions for LLMs). Scaling and extending to large, structured, or highly multimodal spaces are open questions.
- **LLM Capabilities:** In natural-language VRS, the ceiling of unbiasedness and randomness is determined by the LLM’s generative stochasticity and ability to “simulate” random mechanisms via text.

Despite these, VRS stands as a robust, flexible, and widely applicable approach for principled, model-agnostic sampling, facilitating advanced probabilistic modeling, unbiased randomized algorithms, and controlled generation across machine learning, combinatorics, and language technologies.

---

| VRS Application            | Key Innovation                                        | Notable Results                                      |
|----------------------------|------------------------------------------------------|------------------------------------------------------|
| LLM Coin Flipping [2506.09998] | Verbalized, prompt-based accept/reject step           | Halves sampling bias across LLMs versus direct sampling |
| Variational Inference [1804.01712][2309.14612] | Flexible, nonparametric posteriors via acceptance    | State-of-the-art ELBOs, improved posterior accuracy  |
| Constrainted Genetic/Language Generation [2504.05410] | Adaptive sampling, unbiased importance weighting     | Orders-of-magnitude speedup, global constraint fidelity|
| Data Augmentation for MCMC [1406.6652]        | Explicit modeling of rejected proposals              | Tractable inference for doubly-intractable models    |
| Combinatorics [1304.1881][2106.07744]         | Analytic and partial rejection for symbolic objects  | Robust exact sampling, efficient local resampling    |

---

## References

- Grover, A., et al. "Variational Rejection Sampling" [1804.01712]
- Geffner, T., et al. "Reparameterized Variational Rejection Sampling" [2309.14612]
- Bauer, M., & Mnih, A. "Resampled Priors for Variational Autoencoders" [1810.11428]
- Sharma, A., et al. "Refined α-Divergence Variational Inference via Rejection Sampling" [1909.07627]
- Rao, V., Lin, L., & Dunson, D. "Data augmentation for models based on rejection sampling" [1406.6652]
- Guo, J., Jerrum, M., & Liu, J. "Fundamentals of Partial Rejection Sampling" [2106.07744]
- Dohan, D., et al. "Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling" [2504.05410]
- Lin, X., et al. "Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling" [2506.09998]
- Duchon, P., et al. "Analytic Samplers and the Combinatorial Rejection Method" [1304.1881]
- Wei, K., et al. "A minimax near-optimal algorithm for adaptive rejection sampling" [1810.09390]

Source: https://www.emergentmind.com/topics/verbalized-rejection-sampling-vrs