Profiling-based Rejection Sampling (PRS)

• Profiling-based Rejection Sampling (PRS) is an augmentation framework that integrates rejected proposals as auxiliary variables, making latent rejection profiles explicit for exact posterior inference.
• The method formulates an augmented joint distribution that removes intractable normalizing constants and enables the use of standard MCMC kernels, including gradient-based methods.
• PRS offers unbiased inference similar to the exchange algorithm but relies on carefully designed proposals to mitigate increased computational costs from low acceptance rates.

Searching arXiv for the primary paper and related acronym usages to ground the article. arXiv search query: (Rao et al., 2014) rejection sampling augmentation rejected proposals profiling PRS Profiling-based Rejection Sampling (PRS) is an augmentation framework for Bayesian inference in models whose data-generation mechanism is itself a rejection sampler. Instead of treating each observation only as an accepted draw from a target density $p(x \mid \theta)=f(x,\theta)/Z(\theta)$, PRS augments the state space by explicitly representing the sequence of rejected proposals that preceded that acceptance. The resulting joint law over accepted observations and rejected proposals is tractable even when the marginal likelihood involves an intractable or doubly-intractable normalizing constant $Z(\theta)$. This makes it possible to perform exact posterior inference with standard MCMC kernels, including gradient-based updates when derivatives of the augmented log-density are available (Rao et al., 2014).

1. Definition and terminological scope

In the profiling-based usage, PRS denotes the explicit modeling of the rejection sampler’s profile: for each observed datum, the rejected proposals that occurred before the first accepted proposal are introduced as auxiliary variables. The augmentation is not an approximation to rejection sampling and does not alter the underlying generative model; rather, it makes latent structure already implied by the model explicit, so that inference can proceed on an enlarged but simpler state space (Rao et al., 2014).

The acronym is overloaded in the recent literature, and this creates a recurring source of confusion. In "Fundamentals of Partial Rejection Sampling" (Jerrum, 2021), PRS stands for Partial Rejection Sampling, an algorithm for perfect sampling in finite-variable hard-constraint systems that resamples only variables in violated scopes. In "Pliable rejection sampling" (Erraqabi et al., 24 Apr 2026), PRS stands for Pliable Rejection Sampling, a kernel-estimator-based adaptive rejection sampler with high-probability guarantees on accepted sample counts. Profiling-based PRS is distinct from both: it is an auxiliary-variable construction for inference in rejection-sampling-based generative models, not a local-resampling perfect sampler and not an adaptive envelope-learning scheme.

This distinction matters conceptually. Partial Rejection Sampling targets exact sampling from a conditioned product measure under extremal or quasi-extremal structural assumptions, whereas profiling-based PRS targets posterior inference when the likelihood is induced by a rejection sampler and contains an intractable normalizer. Pliable Rejection Sampling, by contrast, modifies proposal construction so that standard accept–reject sampling becomes efficient with high probability. The shared acronym does not indicate a shared mechanism.

2. Probabilistic formulation

The generic setup assumes a target density

$p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$

on a space $X$ with base measure $\lambda(\cdot)$, where

$Z(\theta)=\int f(x,\theta)\,\lambda(dx)$

is intractable. A rejection sampler is available through a proposal $q(x \mid \theta)$ and an envelope constant $M>0$ satisfying

$q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$

for all $x$. The rejection sampler proposes $Z(\theta)$0 and accepts with probability

$Z(\theta)$1

If an accepted observation $Z(\theta)$2 is preceded by $Z(\theta)$3 rejected proposals $Z(\theta)$4, then $Z(\theta)$5 is the number of failures before the first success in a geometric trial with success probability

$Z(\theta)$6

Consequently,

$Z(\theta)$7

The marginal density of the accepted point is

$Z(\theta)$8

which is precisely the intended target law and is intractable whenever $Z(\theta)$9 is intractable. PRS replaces this difficult marginal by the augmented joint

$p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$0

with respect to the measure $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$1 on

$p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$2

Substituting the accept probability yields

$p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$3

The critical feature is that this expression contains no $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$4: only $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$5, $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$6, and $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$7 remain. Integrating out $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$8 recovers the original marginal $p(x \mid \theta)=\frac{f(x,\theta)}{Z(\theta)}$9 exactly (Rao et al., 2014).

3. Augmented inference algorithm

Profiling-based PRS operates by alternating between simulation of rejection profiles and parameter updates on the augmented space. For observations $X$0, the auxiliary variables are $X$1, where each $X$2 is the sequence of rejected proposals preceding $X$3.

A PRS iteration has two essential stages. First, one samples $X$4 given $X$5. For each observation $X$6, one runs the rejection sampler with proposal $X$7 and acceptance function $X$8 until an acceptance occurs, discards the accepted proposal, and records the preceding rejected proposals as $X$9. Second, one updates $\lambda(\cdot)$0 using the augmented posterior proportional to

$\lambda(\cdot)$1

A Metropolis–Hastings update with proposal $\lambda(\cdot)$2 uses the acceptance ratio

$\lambda(\cdot)$3

If gradients are available, Hamiltonian Monte Carlo can be applied directly to

$\lambda(\cdot)$4

The augmentation is effective because the rejected proposals satisfy the conditional independence property

$\lambda(\cdot)$5

Thus the rejection profile preceding an accepted sample is independent of the accepted location conditional on $\lambda(\cdot)$6. This permits conditional simulation of $\lambda(\cdot)$7 without any dependence on the observed $\lambda(\cdot)$8 beyond the fact that the corresponding rejection sampler terminated in acceptance. In models with partially conjugate structure, one may update the intractable components of $\lambda(\cdot)$9 using the augmented state, discard $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$0, and then update conjugate components with standard Gibbs or related steps (Rao et al., 2014).

4. Correctness and Markov-chain properties

The defining correctness statement is marginalization exactness: the augmented joint $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$1 integrates to the original rejection-sampling marginal $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$2. As a result, a data-augmentation chain that alternates between drawing

$Z(\theta)=\int f(x,\theta)\,\lambda(dx)$3

and applying an MCMC kernel invariant for

$Z(\theta)=\int f(x,\theta)\,\lambda(dx)$4

has marginal stationary distribution $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$5. The auxiliary variables therefore eliminate the intractable normalizer from computation without changing the target posterior (Rao et al., 2014).

The conditional independence relation $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$6 is not merely computationally convenient; it is the structural fact that justifies the simplicity of the augmentation. The accepted sample location is independent of the preceding rejected proposals once $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$7 is fixed, because the proposal mixture over accepted and rejected outcomes reconstructs the original proposal law $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$8. This implies that one may simulate rejection histories separately for each datum while preserving posterior correctness.

Under boundedness assumptions on $Z(\theta)=\int f(x,\theta)\,\lambda(dx)$9 and $q(x \mid \theta)$0, together with finite measure $q(x \mid \theta)$1, the PRS Markov kernel on $q(x \mid \theta)$2 satisfies a minorization condition and is uniformly ergodic. The paper states that there exists $q(x \mid \theta)$3 and a density $q(x \mid \theta)$4 such that

$q(x \mid \theta)$5

and gives the mixing-rate bound

$q(x \mid \theta)$6

where

$q(x \mid \theta)$7

Within this bound, more observations $q(x \mid \theta)$8 slow mixing, whereas better agreement between $q(x \mid \theta)$9 and $M>0$0 improves it. The augmentation is also described as unique up to permutations of rejected sequences, because the rejection profile is defined by the original generative mechanism; identifiability therefore reduces to identifiability of the underlying model rather than to any artifact introduced by the auxiliary variables.

5. Representative application domains

Three application classes organize the method’s scope in the original development. They differ substantially in state space and likelihood geometry, but all share the same rejection-sampling-induced augmentation pattern (Rao et al., 2014).

Flow-cytometry truncation. Observations are truncated to a domain $M>0$1, such as the $M>0$2-dimensional unit hypercube after normalization. The untruncated density $M>0$3 is modeled as a Dirichlet process mixture of Gaussians, and truncation corresponds to the hard acceptance rule

$M>0$4

with $M>0$5. The marginal truncated density is

$M>0$6

where $M>0$7 is intractable. The augmented joint contains only proposals outside $M>0$8 and the accepted point inside $M>0$9, so the truncation normalizer disappears. The reported implementation used blocked Gibbs sampling with a stick-breaking truncation of approximately $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$0 components, concentration $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$1, and a normal–inverse–Wishart base measure. On the two-group flow-cytometry dataset, the sampler ran in approximately $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$2 minutes for $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$3 iterations and instantiated average augmented counts of $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$4 and $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$5 points for the two groups. The resulting densities were described as realistic near the boundary and avoided ad hoc bounded-support mixtures.

Matrix Langevin distribution on the Stiefel manifold. For

$q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$6

the matrix Langevin density is

$q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$7

with $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$8 a hypergeometric function of matrix argument. Using the proposal and rejection sampler associated with Hoff’s construction, the accepted observations and rejected proposals yield an augmented joint in which the normalizer $q(x \mid \theta)\ge \frac{f(x,\theta)}{M}$9 no longer appears. The parameters $x$0 and $x$1 admit matrix-Langevin conditional updates, while the diagonal concentration parameters $x$2 can be updated by MH or HMC using derivatives involving modified Bessel functions. On the vectorcardiogram dataset, PRS with HMC achieved an order-of-magnitude higher effective samples per second than exchange MH or random-walk MH. Approximate samplers based on asymptotics for $x$3 were reported as faster but biased.

Nonparametric Gaussian process density models. In the GP-modulated logistic construction,

$x$4

with $x$5, the normalizing constant

$x$6

is intractable. Rejection sampling uses the base density $x$7 as envelope and accepts proposals $x$8 with probability $x$9. After augmenting by rejected proposals $Z(\theta)$00, the conditional law of the GP at $Z(\theta)$01 becomes that of logistic GP classification: $Z(\theta)$02 This permits HMC or elliptical slice sampling on the latent function values. On the Parkinson’s shimmer dataset, rejected proposals concentrated near the origin, the typical augmented size satisfied $Z(\theta)$03–$Z(\theta)$04, and the largest covariance dimension was approximately $Z(\theta)$05.

6. Computational behavior, comparisons, and limitations

The dominant computational quantity is the rejection-sampling acceptance probability

$Z(\theta)$06

Sampling $Z(\theta)$07 for $Z(\theta)$08 observations requires running $Z(\theta)$09 independent rejection samplers until acceptance, so the expected cost of profile generation scales as

$Z(\theta)$10

Low acceptance therefore produces long rejection profiles and high computational overhead. The effect is model-specific: in truncated mixtures, proposal generation and truncation checks are cheap; in the matrix Langevin case, HMC gradients require repeated Bessel-function evaluations; in GP density models, the dominant cost is GP inference on $Z(\theta)$11 points, with Cholesky factorization scaling as $Z(\theta)$12 (Rao et al., 2014).

Proposal design is correspondingly central. Better alignment of $Z(\theta)$13 with $Z(\theta)$14 improves $Z(\theta)$15 and reduces $Z(\theta)$16. The text emphasizes this in three distinct forms: concentrating mixture mass inside the truncation region for flow cytometry, using Hoff’s envelope for the matrix Langevin law, and choosing a base density $Z(\theta)$17 whose support roughly matches the unknown target in the GP model. Numerical stability is also model-dependent; Bessel-function evaluations in the matrix Langevin application require robust libraries, and GP implementations must monitor covariance conditioning as augmented sets grow.

The method occupies a specific niche among exact-inference strategies. Relative to the exchange algorithm, PRS uses the same basic rejection-sampling machinery but retains the rejected proposals instead of discarding them, which enables global HMC updates on $Z(\theta)$18 through the augmented gradients. Relative to pseudo-marginal MCMC, it replaces unbiased likelihood estimation by a deterministic envelope $Z(\theta)$19 and cancellation of intractable terms through augmentation. Relative to ABC, it is exact whenever a rejection sampler exists and does not rely on summaries or tolerances. These are not universal advantages: PRS depends critically on the existence of a usable rejection sampler, and severe proposal mismatch can make the auxiliary-variable state prohibitively large.

Several limitations are intrinsic rather than incidental. Heavy-tailed or poorly matched proposals lower $Z(\theta)$20 and lengthen rejection profiles. In GP density models, large $Z(\theta)$21 rapidly increases cubic linear-algebra costs unless sparse approximations or related reductions are introduced. In matrix Langevin inference, the quality of HMC depends on stable and efficient evaluation of the Bessel-based derivatives. The paper also notes, cautiously, that approximate variants which cap the profile length per iteration may reduce cost but should be treated with care. A plausible implication is that profiling-based PRS is most attractive when the rejection sampler is already exact and reasonably efficient, yet the marginal likelihood remains analytically or computationally intractable.