Verbalized Rejection Sampling
- Verbalized Rejection Sampling (VRS) is a framework that decouples proposal generation from acceptance decisions to sample complex distributions accurately.
- It employs both natural-language and algorithmic instantiations to reduce bias and enhance sampling fidelity, even in black-box settings.
- VRS finds practical applications in machine learning, probabilistic inference, and controlled generation, balancing computational cost and accuracy.
Verbalized Rejection Sampling (VRS) is a framework that generalizes and operationalizes classical rejection sampling within new computational settings, particularly where access to the internal mechanics of the generator is unavailable or undesirable. VRS encompasses both algorithmic and natural-language instantiations, enabling correct, tractable, and often model-agnostic sampling from complex distributions in probabilistic inference, machine learning, combinatorics, and controlled natural language generation. Its core principle is to decouple proposal generation and acceptance decisions—potentially assigning these to different computational or cognitive modules, including LLMs—while maintaining rigorous statistical guarantees.
1. Theoretical Foundations and Algorithmic Principles
VRS extends the classical rejection sampling paradigm, which samples from a proposal distribution and accepts samples with probability proportional to the target-proposal density ratio . In VRS, the accept/reject logic can be verbalized or modularized, with the crucial requirement that, conditioned on acceptance, the sample distribution matches the target . This decoupling enables a range of novel algorithmic strategies:
- Natural-Language VRS: The accept/reject criterion is embedded in prompts for LLMs, which are instructed (in plain language) to reason about and decide on acceptance, rather than relying on model-internal randomness or precise arithmetic (Xiao et al., 11 Jun 2025 ).
- Augmented Sampling: The chain of rejected proposals prior to acceptance can be explicitly modeled and used for efficient data augmentation in MCMC, with the joint sampling process involving both accepted and rejected proposals, and resulting in tractable, normalized posteriors even for doubly-intractable models (Rao et al., 2014 ).
- Trade-off Control: VRS allows interpolation between computational cost and approximation fidelity, as in Variational Rejection Sampling, by tuning the selectivity of the acceptor (e.g., by varying an acceptance threshold) (Grover et al., 2018 , Jankowiak et al., 2023 ).
For a target distribution , and proposal , classical VRS acceptance probability is
where satisfies .
In variational and data-driven settings, acceptance can be further generalized to
where is the logistic/sigmoid function and is a fidelity-control parameter.
2. VRS in Natural-Language and Black-Box Generative Models
Recent work has demonstrated the efficacy of VRS as a black-box augmentation for LLMs, especially in tasks where models nominally “know” target distributions but induce bias when tasked to simulate them directly. In this context (Xiao et al., 11 Jun 2025 ):
- Prompting Framework: The sampler presents the LLM with (i) a textual description of the target distribution, (ii) the proposal distribution, and (iii) a proposed sample, and asks for an accept (“T”) or reject (“F”) decision.
- Empirical Findings: Across models and prompt variants, VRS consistently halves the total variation distance between the empirical sample distribution and the target, outperforming both direct sampling and prompt engineering heuristics.
- Theoretical Guarantees: If LLM decisions approximate the correct acceptance probabilities up to some bounded error , then
and, under mild assumptions, the total variation can be strictly less than that induced by direct simulation bias.
VRS thus provides a principled mechanism for reducing sampling bias without requiring internal access, bespoke random number generators, or heavy prompt tuning.
3. VRS in Variational Inference and Statistical Learning
Variational Rejection Sampling (VRS) has emerged as a tool to address the limitations of parametric approximation in variational inference. Standard variational inference matches the approximate posterior to the true posterior by optimizing an evidence lower bound (ELBO), but struggles for complex or multi-modal targets.
- Flexible Variational Family: VRS constructs a new nonparametric family given by
where is a soft or hard accept/reject function (Grover et al., 2018 , Jankowiak et al., 2023 ).
- Computational-Statistical Trade-off: By varying the acceptance threshold, VRS interpolates between the original variational family and the exact posterior, balancing computational effort against approximation quality.
- Gradient Estimation: Recent advances employ the reparameterization trick for effective gradient estimation, yielding low-variance, pathwise gradient estimators, and enabling large-scale optimization (Jankowiak et al., 2023 ).
- Empirical Validity: VRS improves held-out log-likelihoods and ELBOs over standard parametric or flow-based baselines, and approaches full posterior fidelity in practical scenarios (e.g., VAEs, Gaussian process classification).
4. Combinatorial and Constrained Generation
VRS also manifests in analytic and algorithmic sampling for combinatorial structures, and in controlling the output space of generative models (language or otherwise) under constraints:
- Analytic Samplers: By extending Boltzmann sampling, analytic rejection samplers allow exact or approximate-size sampling with only approximate generating function evaluations, correcting with a rejection step (Bodini et al., 2013 ).
- Partial Rejection Sampling: In structured spaces (e.g., constrained graphs), partial rejection sampling resamples only the components associated with violated constraints, offering exponential speedups while preserving perfect conditional distributions (Jerrum, 2021 ).
- Controlled Language Generation: Adaptive weighted rejection sampling leverages VRS for efficient constrained decoding in LMs, reducing constraint evaluation cost and correcting local myopic biases inherent to token-masking approaches (Lipkin et al., 7 Apr 2025 ). This framework supports arbitrary (even black-box) constraints, and integrates with sequential Monte Carlo methods for global distributional fidelity.
5. Statistical Inference with Data Augmentation and Adaptive Strategies
In scenarios where model parameters or normalizing constants are intractable, VRS enables tractable Bayesian inference and efficient sampling by data augmentation:
- Auxiliary Variables: Augmenting the observed data with rejected samples—modeled as explicit latent variables—eliminates intractable normalizing constants and facilitates efficient MCMC parameter updates (Rao et al., 2014 ).
- Adaptive Proposals: Recent minimax-optimal adaptive rejection sampling algorithms inform VRS strategies for proposal updating, envelope construction, and efficient sample generation under black-box or noisy feedback assumptions (Achdou et al., 2018 ).
These techniques have been deployed in practice for rare event estimation, nonparametric density modeling with Gaussian processes, and Bayesian analysis on manifolds.
6. Limitations, Open Challenges, and Broader Impact
While VRS substantially expands the set of feasible, unbiased, or nearly-unbiased sampling methods in both algorithmic and black-box settings, certain limitations and research avenues remain:
- Efficiency Constraints: For proposal distributions highly discordant with the target, rejection rates can be high. Adaptive strategies and informed proposal selection/orchestration are key ongoing areas of research.
- Scope of Theoretical Guarantees: Most detailed theoretical guarantees and experimental evidence concern low-dimensional or “verbalizable” targets (e.g., Bernoulli distributions for LLMs). Scaling and extending to large, structured, or highly multimodal spaces are open questions.
- LLM Capabilities: In natural-language VRS, the ceiling of unbiasedness and randomness is determined by the LLM’s generative stochasticity and ability to “simulate” random mechanisms via text.
Despite these, VRS stands as a robust, flexible, and widely applicable approach for principled, model-agnostic sampling, facilitating advanced probabilistic modeling, unbiased randomized algorithms, and controlled generation across machine learning, combinatorics, and language technologies.
| VRS Application | Key Innovation | Notable Results |
|---|---|---|
| LLM Coin Flipping (Xiao et al., 11 Jun 2025 ) | Verbalized, prompt-based accept/reject step | Halves sampling bias across LLMs versus direct sampling |
| Variational Inference (Grover et al., 2018 , Jankowiak et al., 2023 ) | Flexible, nonparametric posteriors via acceptance | State-of-the-art ELBOs, improved posterior accuracy |
| Constrainted Genetic/Language Generation (Lipkin et al., 7 Apr 2025 ) | Adaptive sampling, unbiased importance weighting | Orders-of-magnitude speedup, global constraint fidelity |
| Data Augmentation for MCMC (Rao et al., 2014 ) | Explicit modeling of rejected proposals | Tractable inference for doubly-intractable models |
| Combinatorics (Bodini et al., 2013 , Jerrum, 2021 ) | Analytic and partial rejection for symbolic objects | Robust exact sampling, efficient local resampling |
References
- Grover, A., et al. "Variational Rejection Sampling" (Grover et al., 2018 )
- Geffner, T., et al. "Reparameterized Variational Rejection Sampling" (Jankowiak et al., 2023 )
- Bauer, M., & Mnih, A. "Resampled Priors for Variational Autoencoders" (Bauer et al., 2018 )
- Sharma, A., et al. "Refined α-Divergence Variational Inference via Rejection Sampling" (Sharma et al., 2019 )
- Rao, V., Lin, L., & Dunson, D. "Data augmentation for models based on rejection sampling" (Rao et al., 2014 )
- Guo, J., Jerrum, M., & Liu, J. "Fundamentals of Partial Rejection Sampling" (Jerrum, 2021 )
- Dohan, D., et al. "Fast Controlled Generation from LLMs with Adaptive Weighted Rejection Sampling" (Lipkin et al., 7 Apr 2025 )
- Lin, X., et al. "Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling" (Xiao et al., 11 Jun 2025 )
- Duchon, P., et al. "Analytic Samplers and the Combinatorial Rejection Method" (Bodini et al., 2013 )
- Wei, K., et al. "A minimax near-optimal algorithm for adaptive rejection sampling" (Achdou et al., 2018 )