Outcome-based Process Verifier (OPV)

• Outcome-based Process Verifier (OPV) is a verification framework that compresses long chain-of-thought solutions into concise summaries for effective error detection.
• It integrates a CoT summarizer, process verifier, and active learning manager to reduce annotation costs while ensuring precise error localization.
• Empirical evaluations demonstrate that OPV significantly improves verification accuracy and efficiency compared to traditional outcome-based or process-based methods.

The Outcome-based Process Verifier (OPV) is a verification paradigm designed to efficiently and accurately assess the validity of long chain-of-thought (CoT) solutions produced by LLMs. Unlike conventional outcome-based or step-by-step process-based verifiers—which are limited by annotation scalability or error localization—OPV verifies the rationale process via summarization, enabling both large-scale annotation and fine-grained error detection (Wu et al., 11 Dec 2025).

1. Formal Problem Definition

OPV addresses the dual limitations of existing verification approaches. Outcome-based verification (OV) only considers the correctness of the final answer, lacking sensitivity to incorrect intermediate reasoning. Process-based verification (PV) exhaustively inspects each CoT step, incurring prohibitive annotation and computation costs—particularly in long, complex solution traces prone to redundant or tangential reasoning.

OPV formalizes verification as:

• CoT summarization: Given a problem $P$ and a raw CoT $\{c_0, c_1, \ldots, c_{m-1}\}$, the solution is summarized into a compact sequence of essential steps $\mathcal{S} = \{s_0, \dots, s_{n-1}\}$.
• Process verification over summaries: The OPV (verifier) policy $\pi$ receives $(P, \mathcal{S})$ and outputs a predicted index $\hat{\ell}\in\{-1,0,1,\ldots,n-1\}$ for the first incorrect step ($\hat{\ell}=-1$ denotes full correctness), accompanied by an explanation $\hat{\mathcal{E}}$.

This approach preserves critical reasoning elements while compressing CoT length, mitigating the costs of high-quality human annotation (Wu et al., 11 Dec 2025).

2. OPV Architecture and Pipeline

The OPV system consists of three modules:

• CoT Summarizer: Converts long, trial-and-error chains into a faithful, linear summary of 5–15 concise steps (implemented via strong LLMs such as DeepSeek-V3).
• Process Verifier: An autoregressive LLM that verifies the summarized solution, generating step-wise correctness indices and explanations. Training leverages both offline Rejection Fine-Tuning (RFT) and online Reinforcement Learning with Verifiable Rewards (RLVR).
• Active-Learning Manager: Iteratively identifies the most uncertain samples for expert annotation, limits unnecessary labeling, and incorporates new labeled data to refine verifier performance.

The architecture enables accurate error localization and explanation at substantially reduced annotation and inference costs. Summaries are typically 6–8 steps, a ~60% reduction from raw CoTs (Wu et al., 11 Dec 2025).

3. Iterative Active Learning Framework

OPV employs an uncertainty-aware active annotation loop to minimize expert effort:

1. Uncertainty Sampling: For each unlabeled summary, multiple verifier rollouts are conducted; the consistency of predictions across rollouts defines an uncertainty metric—samples with lowest consistency are targeted for expert review.
2. Expert Annotation: Human experts label the true first-error index and explanation for selected summaries.
3. Verifier Update: The annotated data is used to update the verifier via RFT and RLVR.

This cycle is repeated over multiple rounds, dynamically adjusting sampling thresholds to control annotation cost. Empirically, active learning yields a +3.6 F1 improvement over static annotation (Wu et al., 11 Dec 2025).

4. Rejection Fine-Tuning (RFT) and RLVR Training

• RFT: Offline fine-tuning treats verification as a binary classification: trajectories predicting correct indices are labeled positive; the rest are negative. The objective is binary cross-entropy:

$$L_{\rm RFT} = -\mathbb{E}_{\tau\sim\pi}[y(\tau)\log\pi(\tau) + (1-y(\tau))\log(1-\pi(\tau))]$$

• RLVR: Online RL maximizes expected reward under a tailored objective. The reward for a verifier rollout is:

$$R(\hat{\ell},\ell^*) = \begin{cases} -1, & \text{if } \operatorname{sgn}(\hat{\ell}+1)\neq\operatorname{sgn}(\ell^*+1) \ \lambda^{|\hat{\ell}-\ell^*|}, & \text{otherwise} \end{cases}$$

where $\lambda\in(0,1)$ is a decay factor penalizing distance from the true error location. Policy-gradient methods (open-source DAPO algorithm) are used for optimization (Wu et al., 11 Dec 2025).

5. Experimental Evaluation and Results

OPV is evaluated on ProcessBench and a held-out, expert-annotated OPV-Bench:

Model	Precise F1	Approx F1	Rough F1
Qwen3-Max-Preview (240 B)	67.3	70.8	76.3
DeepSeek-R1-Distill-Qwen-32B	71.1	72.9	75.5
OPV-32B	74.7	79.1	83.1

On ProcessBench with standard answers, OPV-32B achieves Rough F1 = 93.8%, matching much larger models. In collaborative reasoning for AIME 2025, integrating OPV with policy models (DeepSeek-R1-Distill-Qwen-32B) elevates accuracy from 55.2% to 73.3% at high compute budgets, with verifier-voting outperforming majority voting by +6.7 points at scale (Wu et al., 11 Dec 2025).

Step-length reduction (~60%) enables annotation time savings of 50–60%, allowing acquisition of 40 K high-quality judgments in a few person-months.

6. Comparative Analysis and Broader Applicability

OPV bridges outcome-based and process-based verification, combining the annotation efficiency of outcome labels with the error detection granularity of process supervision. Analysis across annotation stages reveals cumulative gains: active learning, RFT, and RLVR each contribute measurable F1 improvements (totaling +7.7 from naive baselines).

OPV design generalizes beyond math to domains such as formal logic and code, and is compatible with alternative summarization and uncertainty-sampling strategies (Wu et al., 11 Dec 2025). The framework is scalable (efficient at 32 B model sizes), robust (high F1 even in complex synthetic datasets), and practical for deployment in RLVR pipelines.

7. Limitations and Future Directions

OPV's summarization-based pipeline depends on the fidelity of linear summaries—any compression bottleneck may occlude subtle error propagation. Annotation cost, though sharply reduced, is still non-negligible for very large datasets. Future work may pursue:

• Application to non-mathematical CoTs (formal logic, programming).
• Automated or end-to-end learnable summarization modules.
• Advanced uncertainty metrics (e.g., Bayesian dropout approaches).
• Integration of OPV in multi-agent collaborative reasoning workflows.

These directions aim to extend OPV's utility and further scale high-quality process verification.

---

In summary, Outcome-based Process Verifier (OPV) advances scalable, high-fidelity verification of long-horizon reasoning in LLM-generated chains-of-thought. Through summarized process oversight, active learning, and RL-based fine-tuning, OPV achieves state-of-the-art annotation efficiency and detection accuracy, substantiated across multiple public and proprietary benchmarks (Wu et al., 11 Dec 2025).