Veri-R1: Online RL Framework for Claim Verification

• Veri-R1 is an online reinforcement learning framework that integrates iterative search, multi-hop reasoning, and evidence selection for claim verification.
• It employs an explicit protocol with tags (<plan>, <search>, <think>, <answer>) to enforce structured outputs and reduce generation errors.
• With improvements of up to 30% in Joint Accuracy and doubled evidence scores, Veri-R1 demonstrates significant advances over traditional offline methods.

Veri-R1 is an online reinforcement learning framework for claim verification in which a LLM interacts with a search engine, iteratively retrieves evidence, performs intermediate reasoning, and outputs both a veracity label and evidence identifiers under an explicitly structured protocol (He et al., 2 Oct 2025). It is designed for online claim verification rather than offline claim verification: in the online setting, the model receives only the claim and access to a corpus or search engine, and must actively search, reason, and decide; in the offline setting, the model is given the relevant evidence up front and only needs to reason over supplied evidence (He et al., 2 Oct 2025). The framework’s central premise is that claim verification requires an integrated treatment of planning, retrieval, multi-hop reasoning, label judgment, and evidence selection, and that these skills should be learned through feedback-driven interaction rather than prompt engineering alone (He et al., 2 Oct 2025).

1. Problem setting and conceptual scope

The paper distinguishes two task regimes. In offline claim verification, the model is supplied with a claim and relevant evidence. In online claim verification, which Veri-R1 targets, the model is given only the claim and access to a corpus or search engine, and must iteratively retrieve evidence, reason over it, and then decide the label (He et al., 2 Oct 2025). The target label space is three-way: SUPPORT, REFUTE, and NOT ENOUGH INFO (He et al., 2 Oct 2025).

The motivation is explicitly framed as a limitation of prior workflows. Prompt engineering and structured reasoning prompts can improve transparency, but they do not provide a unified learning framework for search, reasoning, and judgment. Fixed reasoning workflows are often specialized to particular datasets or domains and do not generalize well across varied verification settings. Supervised fine-tuning usually requires high-quality reasoning traces, may teach the model to imitate a reasoning format rather than learn how to search and verify, and can generalize poorly to new claim types (He et al., 2 Oct 2025).

Within this framing, Veri-R1 treats verification as an interactive control problem rather than a single-shot classification problem. A plausible implication is that the framework redefines factual verification as a sequential decision process in which evidence acquisition and judgment are jointly optimized rather than separated into independent modules.

2. Interactive verification protocol

Veri-R1 trains the LLM as an agent that alternates between search and reasoning. The trajectory is formalized as

$$T = \bigl(p,\,(s_1,i_1,t_1),\,(s_2,i_2,t_2),\dots,(s_k,i_k,t_k),\,a \bigr),$$

where $p$ is a plan, $s_i$ is the search action at step $i$, $i_i$ is the retrieved information, $t_i$ is the internal reasoning step, and $a$ is the final answer (He et al., 2 Oct 2025).

The prompting protocol uses explicit tags:

• <plan> ... </plan>
• <search> ... </search>
• > ...
• <answer> ... </answer>

The final answer must contain both a label and evidence IDs in a strict format (He et al., 2 Oct 2025). This strictness is not incidental; it is part of the training signal and is used to make parsing robust and reduce generation errors.

The paper uses up to 3 search turns during training and a maximum turn budget of 4 in the training configuration (He et al., 2 Oct 2025). Retrieval is implemented with FAISS. Sentence retrieval groups every 3 sentences into one entry, and at each turn the system retrieves the top 3 related entries (He et al., 2 Oct 2025). The effective state therefore includes the current claim, conversation history, retrieved evidence, and prior reasoning, while the action space includes generating a plan, issuing a search query, reasoning, or producing the final answer (He et al., 2 Oct 2025).

The significance of this design lies in its attempt to mirror real fact-checking conditions. Rather than assuming evidence has already been surfaced, Veri-R1 makes evidence discovery part of the learned behavior. This is the feature that most clearly separates it from offline reasoning pipelines.

3. Reinforcement learning formulation and reward shaping

The policy $\pi_\theta$ is optimized to maximize expected final reward: $$\max_\theta \; E_{T \sim \pi_\theta}\bigl[R(a)\bigr].$$ The reward is designed to shape planning, retrieval, and reasoning or judgment behavior simultaneously (He et al., 2 Oct 2025).

The paper uses Group Relative Policy Optimization (GRPO). It gives the objective as a standard GRPO or PPO-style clipped policy optimization with group-normalized rewards, with the intended meaning described as

$$L(\theta) = \mathbb{E}\Bigl[ \min\bigl(r(a)\,\hat R_{\mathrm{grp}(a)}, \; \mathrm{clip}(r(a), 1-\epsilon, 1+\epsilon)\,\hat R_{\mathrm{grp}(a)} \bigr) \Bigr],$$

where

$p$0

The paper notes that the source LaTeX is garbled, but the intended formulation is clipped policy optimization with group-normalized rewards (He et al., 2 Oct 2025).

The final reward has four components: label reward, evidence reward, format reward, and a validity weight applied to the label reward (He et al., 2 Oct 2025). The individual terms are specified as follows.

The format reward is binary: $p$1

The evidence reward is the Jaccard similarity between predicted and gold evidence sets: $p$2

The label reward is weighted toward correct veracity prediction: $p$3

The validity weight depends on the evidence hit rate

$p$4

For SUPPORT and REFUTE, full label reward is earned only when all gold evidence is retrieved; partial credit is given if more than half the gold evidence is retrieved; otherwise the label reward is suppressed. For NOT ENOUGH INFO, no validity weighting is applied (He et al., 2 Oct 2025).

The intended final combination is

$p$5

again with the caveat that the paper’s displayed LaTeX is malformed while the intended combination is stated explicitly in prose (He et al., 2 Oct 2025).

This reward design is the framework’s main mechanism for discouraging shortcut verification. The evidence reward pushes retrieval precision and coverage, the label reward pushes correct classification, the format reward enforces protocol adherence, and the validity weight prevents “correct label for wrong reasons” by withholding full credit when the evidence grounding is incomplete (He et al., 2 Oct 2025).

4. Data processing, training pipeline, and implementation

The high-level pipeline consists of collecting training data from FEVEROUS and EX-FEVER, filtering data quality with GPT-4o in an offline-rollout simulation, preparing online and offline prompts, training with GRPO, and evaluating on FEVEROUS, EX-FEVER, FEVER, HOVER, and SciFACT (He et al., 2 Oct 2025). The filtering step retains only samples where GPT-4o gets both label and evidence totally correct, and about 70% of samples are retained (He et al., 2 Oct 2025).

The training and evaluation data are curated with explicit balancing and filtering rules. For FEVEROUS, the procedure samples 3,000 instances per label and includes underrepresented challenge categories. For EX-FEVER, it uses gold labels and evidence and samples 3,000 examples per label. For FEVER, it randomly samples 900 examples total, with 300 per label. For SciFACT, it uses as many examples as possible from the smallest label count, namely 237 per label, and maps CONTRADICT to REFUTE. For HOVER, it filters ambiguous claims with GPT-4o, evaluates only 2-hop claims, and uses a balanced evaluation set of 448 per label (He et al., 2 Oct 2025).

The reported hardware and training configuration are specific. Online RL training uses 2 × NVIDIA A800-80G GPUs and takes about 15 hours for 100 steps. Key values are: train batch size 256, validation batch size 256, max prompt length 4864, max response length 512, learning rate $p$6, PPO mini-batch size 64, KL coefficient 0.001, temperature 0.8, max turns 4, and retriever top-k 3 (He et al., 2 Oct 2025). LoRA is used for the supervised fine-tuning baselines (He et al., 2 Oct 2025).

These details matter because Veri-R1 is not only a reward function layered on top of a generic LLM. It is a full training setup in which corpus construction, retrieval granularity, turn budget, strict output schema, and reward design are all coordinated around online verification behavior.

5. Evaluation protocol and empirical findings

The paper reports four evaluation metrics. Joint Accuracy requires the correct label and correct evidence. Verification Accuracy requires the correct label and all gold evidence retrieved. Label Accuracy measures the correctness of the label alone. Evidence Score is the Jaccard similarity between predicted and gold evidence (He et al., 2 Oct 2025).

The headline findings are up to 30% absolute gain in Joint Accuracy, 23% improvement in Verification Accuracy, 22% improvement in Label Accuracy, and evidence score improvement by up to 150% (He et al., 2 Oct 2025). The abstract further summarizes the evidence-side gain as doubling evidence score (He et al., 2 Oct 2025).

Representative results are given for Qwen2.5-3B-Instruct-OnlineRL. On FEVEROUS it reaches Joint 28.91, Veri 36.28, and Label 61.22; on EX-FEVER, Joint 31.69, Veri 32.45, and Label 61.09; on FEVER, Joint 49.11, Veri 55.89, and Label 69.56; on SciFACT, Joint 38.82, Veri 43.18, and Label 63.43; and on HOVER, Joint 53.70, Veri 55.10, and Label 63.70 (He et al., 2 Oct 2025). For Llama3.2-3B-Instruct-OnlineRL, the corresponding results are FEVEROUS: Joint 19.27, Veri 26.30, Label 53.17; EX-FEVER: Joint 28.52, Veri 30.29, Label 59.32; FEVER: Joint 40.11, Veri 53.11, Label 68.44; SciFACT: Joint 31.65, Veri 44.30, Label 66.53; HOVER: Joint 52.60, Veri 54.90, Label 65.40 (He et al., 2 Oct 2025).

Evidence Score results for Qwen2.5-3B-OnlineRL are also reported: FEVEROUS 0.4769, EX-FEVER 0.4635, FEVER 0.4630, SciFACT 0.2713, and HOVER 0.6562 (He et al., 2 Oct 2025). The comparisons in the paper support several broad conclusions: online RL usually beats supervised fine-tuning, online RL usually beats offline RL, online RL often matches or surpasses larger models, and GPT-4o remains strong but far from solving online claim verification robustly (He et al., 2 Oct 2025).

These findings are significant because the improvements are not limited to label prediction. Veri-R1 is explicitly evaluated on faithfulness-sensitive metrics that require evidence retrieval quality, and its strongest gains are tied to those metrics rather than to answer-only accuracy.

6. Ablations, interpretation, and relation to neighboring verification frameworks

The ablation studies identify two reward components as especially important. Adding the evidence reward stabilizes evidence retrieval: without it, evidence score may rise briefly and then degrade, whereas with it, evidence score improves consistently (He et al., 2 Oct 2025). The validity weight is also important for higher Verification Accuracy, higher evidence cover rate, and prevention of shortcut solutions; without the validity weight, the model can get the label right with incomplete evidence and the evidence cover rate drops after some training steps (He et al., 2 Oct 2025).

The paper also analyzes the relationship between output logits and label accuracy. Logits mostly cluster near 1, suggesting generally high confidence. For SUPPORT and REFUTE, higher confidence correlates with higher accuracy. For NOT ENOUGH INFO, the relationship is weaker or reversed. Larger models tend to be less willing to predict NOT ENOUGH INFO, implying overconfidence or bias toward definite labels (He et al., 2 Oct 2025). This suggests that confidence calibration is label-dependent in online claim verification and that uncertainty handling remains incomplete.

The qualitative comparison between online RL and offline RL further clarifies the framework’s behavior. Offline RL tends to use coarse, word-level decomposition, often issues a generic query once, retrieves redundant or unfocused evidence, and may predict NOT ENOUGH INFO with irrelevant evidence. Online RL performs fine-grained subclaim decomposition, issues distinct queries for each subclaim, integrates retrieved evidence step-by-step into reasoning, and reaches more faithful final predictions (He et al., 2 Oct 2025).

Several misconceptions are addressed by the broader literature. Veri-R1 is not an offline reasoning benchmark and not merely a prompt template; it is an online RL framework centered on interactive retrieval and reward shaping (He et al., 2 Oct 2025). It is also distinct from VERIRAG, which is a verification pipeline for healthcare claim verification in retrieval-augmented generation that adds a scientific-audit layer through the Veritable, the Hard-to-Vary Score, and a Dynamic Acceptance Threshold rather than training an online search-and-reason agent (Mohole et al., 23 Jul 2025). Relative to R1-Onevision, which uses rule-based RL with answer checking and format enforcement for multimodal reasoning but does not include a dedicated verifier model or explicit step-by-step proof checking, Veri-R1 places the verification problem in an online retrieval environment (Yang et al., 13 Mar 2025). Relative to Table-R1, which studies table reasoning with reinforcement learning from verifiable rewards and GRPO, Veri-R1 applies a comparable verifiable-reward logic to claim verification with iterative search (Yang et al., 29 May 2025). Relative to Skill-R1, which uses verifiable rewards to optimize a lightweight skill generator while keeping the task model frozen, Veri-R1 directly trains the claim-verification policy in an interactive setting (Vishe et al., 10 May 2026).

The paper also states clear limitations. Training and evaluation use a fixed local corpus and retriever, while real-world claim verification operates over much larger, dynamic corpora (He et al., 2 Oct 2025). Gold evidence is available for training reward construction, claims are mapped into SUPPORT, REFUTE, or NOT ENOUGH INFO, the retrieval corpus is static during training and evaluation, and the system assumes that a strict output schema with tags can be enforced (He et al., 2 Oct 2025). These constraints delimit the current scope of the framework. A plausible implication is that Veri-R1 should be understood as a foundation for online evidence-grounded verification rather than as a complete solution to open-web fact checking.

7. Place within the emerging “verifiable-reward” paradigm

Veri-R1 belongs to a broader movement toward training reasoning systems with explicit, automatically checkable reward signals rather than relying only on supervised imitation. In Table-R1, the relevant analogue is RLVR with task-specific verifiable rewards for table question answering, fact verification, and free-form QA, coupled with GRPO and strict reasoning or answer formatting (Yang et al., 29 May 2025). In R1-Onevision, the analogue is rule-based RL that checks final answers and enforces <think> tags, but without deeper semantic verification of each reasoning step (Yang et al., 13 Mar 2025). In Skill-R1, verifiable rewards supervise recurrent skill evolution rather than direct task-model adaptation (Vishe et al., 10 May 2026). In VERIRAG, the emphasis shifts from online RL to methodological auditing of retrieved scientific evidence (Mohole et al., 23 Jul 2025).

Against this backdrop, Veri-R1’s distinguishing contribution is to combine online search interaction with reward shaping over both evidence use and final judgment (He et al., 2 Oct 2025). Its label reward alone would resemble conventional answer-level supervision; its evidence reward, validity weighting, and strict output protocol make the framework explicitly faithfulness-oriented. This suggests that Veri-R1 is best viewed not simply as another claim classifier, but as a training paradigm for evidence-grounded verification trajectories in which planning, retrieval, reasoning, and decision are optimized together.