LLM-as-a-Verifier: A General-Purpose Verification Framework

Abstract: Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.

• The paper introduces a probabilistic, training-free verification framework that uses full token logit distributions to yield continuous rewards.
• It leverages scaling along score granularity, repeated evaluations, and criteria decomposition to improve discrimination between correct and incorrect outputs.
• Empirical results show enhanced verification accuracy across coding, robotics, and medical benchmarks, enabling efficient RL reward shaping.

LLM-as-a-Verifier: A General-Purpose Probabilistic Verification Framework

Introduction and Motivation

Contemporary LLM systems have achieved strong generative capabilities through pre-training, post-training, and inference-time compute scaling. However, the verification axis—systematic determination of solution correctness—has exhibited far less development. Verification is traditionally handled via LLM judges producing discrete scores or by trained reward models, both with substantial limitations: standard judges induce high tie rates and poor discrimination on complex solutions due to coarse scoring, while reward models are data-constrained and exhibit poor cross-domain generalization. This work introduces LLM-as-a-Verifier, a probabilistic, training-free framework that leverages the complete scoring-token logit distribution of LMs to deliver calibrated, fine-grained feedback at inference.

Figure 2: LLM-as-a-Verifier: a unified verification framework scalable across modalities and domains, supporting feedback for test-time scaling, progress tracking, and RL.

Methodology: Probabilistic and Scalable Verification

Probabilistic Reward Formulation

The core mechanism of LLM-as-a-Verifier replaces the conventional discrete scoring protocol with a probabilistic estimation over token logits. Specifically, given a prompt, criterion, and trajectory, the model evaluates the expected value over all possible scoring tokens, yielding a continuous reward. The formulation is:

$$R(x, \tau) = \frac{1}{CK} \sum_{c=1}^{C} \sum_{k=1}^{K}\sum_{g=1}^{G} p_\theta(v_g \mid x, c, \tau)\,\phi(v_g)$$

where $C$ is the number of evaluation criteria, $K$ is the number of repeated passes, and $G$ is granularity (the number of scoring tokens). This expectation over the scoring-token logit distribution offers multiple axes for improved discrimination between correct and incorrect trajectories.

Scaling Dimensions

LLM-as-a-Verifier defines three orthogonal axes for scaling verification quality:

• Score Granularity ($G$): Enlarging the discrete set of scoring tokens enables a finer reward continuum, improving separation between trajectories.
• Repeated Evaluation ($K$): Averaging over multiple evaluations mitigates stochasticity and prompt-induced variance.
• Criteria Decomposition ($C$): Decomposing correctness into logically distinct criteria (e.g., specification, output, error signature) reduces ambiguous or biased aggregate judgments.

Empirical results demonstrate monotonic gains along each of these axes.

Figure 1: Verification accuracy improves with increased score granularity, more evaluation repetitions, and finer criteria decomposition.

Budget-Efficient Selection via Probabilistic Pivot Tournament

Selecting the best solution from a pool of $N$ candidates is computationally burdensome with naive $\mathcal{O}(N^2)$ pairwise comparisons. The authors introduce the Probabilistic Pivot Tournament (PPT) pipeline, which uses a random Hamiltonian cycle for initial pairwise scoring, followed by pivot-based focused comparisons, achieving $\mathcal{O}(Nk^2)$ complexity for $C$0.

Figure 3: The Probabilistic Pivot Tournament (PPT) pipeline: ring pass, pivot selection, focused comparisons, and aggregation.

This combinatorial ranking algorithm maintains accuracy while dramatically lowering cost, a necessary feature for scaling Best-of-$C$1 selection.

Experimental Results Across Multiple Domains

The framework is evaluated as a trajectory reward model (TRM) in diverse domains:

• Terminal-Bench V2: Achieves 86.5% accuracy, surpassing baselines and approaching the oracle upper bound, despite task environments in which syntactically plausible trajectories frequently fail.
• SWE-Bench Verified: 78.2% accuracy in selecting correct code patches, outperforming all individual model candidates in diverse expert patching tasks.
• RoboRewardBench: 87.4% pairwise preference accuracy on manipulation trajectory progress—outperforming both specialized reward models (RoboReward-8B, Robometer-4B) and LLM-Judge protocols.
• MedAgentBench: 73.3% accuracy in EHR-based multi-step medical tasks involving information retrieval, tool use, and guideline consultation.

LLM-as-a-Verifier consistently recovers a significant proportion of the oracle headroom exposed by sampling trajectories, without any model retraining or domain-specific adaptation.

Comparative Analysis: Discrete Judges vs Probabilistic Verifier

Figure 4: Continuous LLM-as-a-Verifier consistently outperforms discrete judges both in accuracy and in reducing tie rates across repeated evaluations.

A critical comparative analysis reveals that probabilistic reward formulation (continuous reward) reduces the tie rate to zero and substantially elevates pairwise accuracy vis-à-vis standard LLM judges, even with extensive repetition. The continuous verifier reliably distinguishes between trajectories that a discrete judge cannot separate, eliminating the stagnation induced by coarse score quantization.

Fine-Grained Progress Tracking and Monitoring

The dense output of the verifier can be interpreted as a proxy for agentic task progress on long-horizon problems. For instance, on code generation and manipulation tasks, chronological Value-Order Correlation (VOC)—the Spearman rank correlation between the ordered steps in a trajectory and their corresponding verifier scores—achieves values up to 0.966 (robotics) and 0.848 (code), contrasting with weaker baselines. This allows for real-time monitoring and early detection of stalling or regressing behaviors in agentic systems.

Figure 5: Correlation between step progression and verifier score: monotonic curve for successful runs, flat or regressive for failures.

Reinforcement Learning with Verifier-Generated Dense Rewards

LLM-as-a-Verifier’s dense, fine-grained signal enables its use as a reward function for RL agents, improving sample efficiency for both off-policy (SAC) and on-policy (GRPO) methods.

Figure 6: Dense rewards from LLM-as-a-Verifier improve RL sample efficiency, reducing training steps to reach a given success rate in both robotics and mathematical reasoning.

Empirical results on LIBERO (robotics) and MATH (mathematical reasoning) indicate $C$2 and $C$3 higher sample efficiency, respectively, compared to sparse reward baselines.

Discussion and Implications

The presented approach recasts verification as an axis for scalable improvement, on par with generation and inference compute. By leveraging fine-grained, probabilistically-motivated feedback from existing LLMs, LLM-as-a-Verifier obviates the need for costly domain-specific reward model training and achieves substantial improvements across coding, robotics, and medical benchmarks. Crucially, verification scaling is orthogonal to generation scaling, with gains compounding as trajectory proposal quality increases.

Practically, these properties enable (a) robust deployment of LLM-agents in settings where ground-truth checkers are nontrivial to implement, (b) more reliable policy improvement pipelines through RL with richer reward signals, and (c) interoperable, plug-and-play evaluation modules that monitor and rank agent behavior across domains.

Theoretically, this work posits verification as a key bottleneck deserving systematic scaling, suggests the utility of full token logit distributions for reward estimation, and prompts future studies into adaptive compute allocation, learned criteria decomposition, and feedback-driven agentic improvement. Extending this framework to online learning, multi-turn RL, and dynamic environments is a promising direction.

Conclusion

LLM-as-a-Verifier constitutes a significant advance in the general-purpose evaluation of LLM-agent outputs. By converting scoring-token logit distributions into dense, continuous rewards, and scaling feedback along complementary axes, it achieves high verification accuracy, progress monitoring, and efficient reward shaping across modalities—without further training or per-domain customization. This framework establishes verification as a distinct scaling axis and highlights its centrality for robust, autonomous agent systems (2607.05391).