Verifier-Based Inference-Time Scaling

• Verifier-Based Inference-Time Scaling is an inference-time framework that uses an auxiliary verifier to filter and re-rank candidate outputs from generative models for improved accuracy and task compliance.
• Core methodologies such as Rejection Sampling, Best-of-N selection, and beam search utilize verifier scores derived from ROC curve geometry to optimize sample efficiency and scaling performance.
• Practical implementations demonstrate significant gains in domains like language and diffusion models, though increased compute costs and diminishing returns remain key considerations.

Verifier-Based Inference-Time Scaling is an inference-time algorithmic and statistical framework that improves the performance of generative models (especially LLMs, diffusion models, and related architectures) by leveraging an auxiliary verifier model to guide, re-rank, or filter candidate outputs during generation, without modifying the base model weights. This paradigm achieves parameter-free test-time performance scaling through judicious selection based on verifier scores, trading extra inference compute for higher accuracy, quality, or task compliance. The theoretical underpinnings, operational principles, and practical limitations are dictated by the properties and geometry of the verifier's receiver operating characteristic (ROC) curve and the interaction between generator output distributions, verifier mapping, and compute allocation strategies.

1. Theoretical Foundations: Geometry of the ROC Curve

Verifier-based inference-time scaling relies on the ROC curve of the verifier, defined as the mapping from false positive rate (FPR) to true positive rate (TPR) as the scoring threshold is swept. Let $g_{\rm base}$ denote the base generative model with instance-level success probability $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$, and let $f(x) \in [0,1]$ be the verifier score. The thresholded classifier $h_t(x)=\mathbf{1}\{f(x)\ge t\}$ enables computation of TPR and FPR: $$\mathrm{TPR}(t) = \Pr[h_t(x) = 1 \mid y(x) = 1], \quad \mathrm{FPR}(t) = \Pr[h_t(x) = 1 \mid y(x) = 0].$$ Varying $t$ traces the ROC curve $F \mapsto T(F)$, where $F$ is the FPR and $T(F)$ is the maximum achievable TPR at that FPR. In the context of rejection sampling (RS) and best-of-$N$ (BoN) selection, the geometry—particularly the slope $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$0 at various $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$1—precisely governs sample efficiency, scaling behavior, and upper bounds for achievable accuracy as a function of inference compute.

For rejection sampling at a fixed FPR $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$2, the expected sample cost is: $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$3 and the precision or success rate of selected samples is: $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$4 As compute scales ($\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$5, $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$6), the accuracy ceiling is set by the tangent slope at $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$7: $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$8 For BoN, the accuracy is a nonlinear functional of the entire ROC curve, depending not only on local but also global geometric properties (Dorner et al., 16 Jul 2025).

2. Core Methodologies: RS, BoN, Beam Search

Verifier-based scaling manifests via several algorithmic strategies:

• Rejection Sampling (RS): Draws i.i.d. candidates from $\pi = \Pr_{x \sim g_{\rm base}}[y(x) = 1]$9 and accepts the first passing a verifier threshold. The per-instance accuracy is locally controlled by the ROC derivative, with early scaling rate determined near $f(x) \in [0,1]$0 and asymptotic accuracy governed by $f(x) \in [0,1]$1.
• Best-of-$f(x) \in [0,1]$2 (BoN) Selection: Draws $f(x) \in [0,1]$3 candidates, scores all with the verifier, and selects the top-scoring output. The scaling curve’s gain is upper bounded by properties of the area under the ROC (AUROC) and, in the infinite-compute limit, matches the RS asymptote.
• Beam or Tree Search: For stepwise structured reasoning, at each generation step, a verifier ranks partial paths; top-scoring beams advance. When the verifier is imperfect, pruning causes diminishing returns and, eventually, inferior scaling relative to random repeated sampling—a phenomenon termed "scaling flaw" (Yu et al., 1 Feb 2025).
• Optimal Transport View: Coverage-enhancing RS or batched RS protocols can be rigorously understood via an optimal transport framework, quantifying the trade-off between coverage, verifier ROC (Youden’s $f(x) \in [0,1]$4 index), and sub-optimality through closed-form expressions (Mukherjee et al., 21 Oct 2025).

3. Verifier Model Classes and Scoring

A comprehensive taxonomy spans:

• Prompt-based ("LLM-as-judge"): Re-prompting the base model for binary or open-ended judgments, zero/few-shot, highly sensitive to prompt wording.
• Fine-tuned Discriminative Models: Binary outcome reward models (ORMs) for entire outputs and process reward models (PRMs) for stepwise labeling, trained via cross-entropy on correctness/syntactic annotations.
• Generative Verifiers: Models performing natural language critiques (supervised or RL-trained), often providing richer signals but incurring higher compute costs.
• Symbolic Verifiers: Formal execution engines for code, arithmetic, or logic, yielding hard correctness guarantees but only on domains where symbolic analysis is tractable.
• RL/Preference-trained Verifiers: Trained on pairwise or stepwise human/synthetic preferences, sometimes coupled with Q-learning or direct preference optimization (Venktesh et al., 20 Aug 2025).

Verifier assignment $f(x) \in [0,1]$5 allows for candidate selection: $f(x) \in [0,1]$6 with loss functions tailored to the supervision regime (cross-entropy for binary labels, stepwise aggregation for PRMs, log-likelihood or DPO for preference or critique generation).

4. Practical Efficiency and Numerical Examples

Efficiency of inference scaling is highly sensitive to verifier cost, type, and integration strategy:

• Empirical studies confirm that, on formal domains such as math and programming, RS and BoN enable significant pass rate improvements given modest computational increases, with diminishing returns as sample count increases due to verifier misclassification (Dorner et al., 16 Jul 2025, Yu et al., 1 Feb 2025).
• In legal reasoning, process-level tree search or outcome-level BoN delivers non-trivial gains only for