Claim-Evidence Gap Overview

• Claim-evidence gap is the mismatch between a claim’s surface form and the required supporting evidence in verification systems.
• It spans various applications including fact verification, multimodal reasoning, scholarly QA, and evidence-grounded summarization.
• Diagnostic methods and claim-evidence interfaces help identify routing failures, shortcut verifications, and warrant mismatches in complex setups.

The claim-evidence gap is the mismatch between a claim’s surface form and the evidence that is actually required, retrieved, represented, routed, cited, or used to justify it. Across recent work, the gap appears in several closely related forms: a scientific claim may be internally encoded by a model but not used at prediction time; a database-grounded claim may require a latent executable program rather than any readable snippet; a verifier may accept a claim after checking only its most salient constraint; a system may cite passages that are topically relevant but do not entail its answer; or a summarizer may generate clinically plausible statements that are unsupported by the underlying record (Kumar et al., 1 Jun 2026, Theologitis et al., 21 Jan 2026, Liu et al., 13 Apr 2026, Dey et al., 23 Jun 2026, Liu et al., 11 Mar 2026). The concept therefore spans fact verification, multimodal reasoning, scholarly QA, peer review, financial QA, and evidence-grounded summarization.

1. Conceptual scope

In the literature, the claim-evidence gap is not a single pathology. Some work treats it as a gap between a natural-language claim and the evidence needed to verify it; other work treats it as a gap between evidence already present inside a model and evidence actually used to produce an output. “Encoded but Not Routed” makes this distinction explicit by separating perception/encoding failure from routing/usage failure in multimodal scientific verification, concluding that chart evidence is often represented in hidden states but fails to reach the final prediction position (Kumar et al., 1 Jun 2026). By contrast, ClaimDB defines the gap in structured-data terms: the claim is linguistic, but the relevant evidence is often the result of querying, joining, filtering, aggregating, or sorting across databases averaging 11.6 tables and 4.6M records, so verification becomes executable reasoning rather than reading (Theologitis et al., 21 Jan 2026).

A second line of work shows that the gap can persist even when evidence is available and retrieval is nominally successful. “When Verification Fails” argues that many systems do not verify the full claim under the Closed-World Assumption, but instead perform salient-constraint checking, accepting a claim when its most salient constraint is supported even if a non-salient asserted constraint is contradicted or unsupported (Liu et al., 13 Apr 2026). “The Warrant Gap” sharpens this further: a system may output the correct Supports label while citing evidence that does not license the claim, so verdict accuracy and evidential admissibility come apart (Dey et al., 23 Jun 2026).

A third strand relocates the gap earlier or later in the pipeline. ContextClaim argues that verifiable claim detection itself suffers from a claim-evidence mismatch because verifiability is not purely a linguistic property of the claim text; it depends on whether entities can be grounded and whether relevant evidence plausibly exists (Li et al., 31 Mar 2026). PaperTrail and EvidenceLens move in the opposite direction, showing that after generation the key problem is often not retrieval alone but claim-level provenance: coarse citations do not reveal which answer claims are supported, unsupported, or omitted, especially when evidence spans text, tables, and charts (Martin-Boyle et al., 24 Feb 2026, Gu et al., 19 Jun 2026).

2. Formalizations of claim-to-evidence relations

One formalization models a claim as a set of asserted constraints. In “When Verification Fails,” if a claim $c$ asserts constraints $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$, then under the Closed-World Assumption a claim is accepted only if every asserted constraint is positively supported by some evidence subset. The paper contrasts this with an Open-World rule and with salient-constraint checking, which verifies only the most salient constraint $c^*$ (Liu et al., 13 Apr 2026). This formal separation matters because many benchmark negatives are constructed so that the salient violated constraint is also the decisive one, making shortcut reasoning observationally indistinguishable from full verification.

Another formalization treats insufficient evidence as an evidence-conditioned label rather than a claim property. NEI-CAP represents an instance as $x=(c,E,y,z,g)$, where $c$ is the claim, $E$ the evidence set, $$y \in \{\mathrm{Support}, \mathrm{Refute}, \mathrm{NEI}\}$$, $z$ the construction family that produced the evidence condition, and $g$ a group identifier (Qiu et al., 26 May 2026). This representation encodes the paper’s central thesis that evidence absence is not evidence insufficiency: empty evidence, off-topic evidence, same-document non-rationales, and high-overlap near-misses may all receive the same NEI label while testing very different competencies.

A third formalization centers on warrant admissibility. “The Warrant Gap” defines the Warranted Supports Proportion as

$$\mathrm{WSP}=\frac{\big|\{i \in S : \mathrm{NLI}(\hat{w}_i,c_i)=Entail\}\big|}{\big|S\big|},$$

where $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$0 is the set of examples for which the system predicted Supports, $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$1 is the cited warrant, and $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$2 is the claim (Dey et al., 23 Jun 2026). WSP quantifies a specific claim-evidence gap: the fraction of positive verdicts whose cited evidence actually entails the claim.

Evidence sufficiency can also be formalized as a coverage problem. “Minimal Evidence Group Identification for Claim Verification” defines a Minimal Evidence Group (MEG) as an evidence group that fully supports the claim, is non-redundant, and is minimal in cardinality; in the paper’s idealized formulation, MEG identification reduces to Set Cover over atomic claim units (Li et al., 2024). This reframes the gap as the difference between merely relevant evidence and a complete, minimally sufficient support set.

3. Evidence regimes and benchmark constructions

The empirical literature shows that the claim-evidence gap changes character with the evidence regime. In multimodal scientific verification, SciTabAlign+ controls for content while varying representation: 162 unique scientific claims are paired with one table and four chart variants derived from the same data, yielding 372 table-based instances and 648 chart-based instances (Kumar et al., 1 Jun 2026). Because content is matched, performance differences isolate format-conditioned failures.

In structured-data verification, ClaimDB widens the gap intentionally. It contains 53,368 claims over 80 real-life databases spanning 11 high-level domains and 36 subdomains, with labels ENTAILED, CONTRADICTED, and NOT ENOUGH INFO; the evidence for a claim is usually an implicit database computation rather than a localized snippet (Theologitis et al., 21 Jan 2026). Its NEI taxonomy—Out-of-Schema, Counterfactual, and Subjective—also shows that claim-evidence mismatch may be ontological, hypothetical, or epistemic rather than merely retrieval-related.

Open-web verification introduces still another regime. AVeriTeC contains 4,568 real-world claims from 50 fact-checking organizations, each annotated with question-answer pairs, source-backed evidence, justifications, and one of four verdicts: supported, refuted, not enough evidence, or conflicting evidence/cherry-picking (Schlichtkrull et al., 2023). Here the gap is the distance from a messy real-world claim to the intermediate questions and heterogeneous web evidence needed to justify a verdict.

Scientific long-context verification pushes the evidence boundary outward. SciClaimHunt pairs claims with entire research papers rather than abstracts, with 87,109 train, 10,884 dev, and 10,900 test claim-paper pairs, plus a numerical subset, SciClaimHunt_Num, for numeral-bearing claims (Kumar et al., 14 Feb 2025). FactReview extends the same logic beyond manuscripts, combining claim extraction, literature positioning, and bounded code execution so that review judgments can be grounded in manuscript text, nearby literature, and execution traces rather than manuscript rhetoric alone (Xu et al., 5 Apr 2026).

4. Mechanisms that produce the gap

One major mechanism is format-sensitive routing failure. In “Encoded but Not Routed,” models perform far better on tables than on charts even when both express the same data. For Qwen2.5-VL-32B on the basic chart, the best mean-pool probe reaches 88.9\% AUROC, while the last-token probe reaches only 71.9\%, indicating that chart-relevant information exists in internal representations but does not reach the prediction site (Kumar et al., 1 Jun 2026). Attention analysis shows two distinct failure modes: in the Qwen family, final-token attention to chart/image tokens falls to only 4–11\% of the proportional baseline; in InternVL3-8B, image attention remains near proportional, but integration into the final label still fails.

A second mechanism is shortcut verification. Models can reject obvious negatives yet still over-accept compositionally infeasible claims. Across NLI4CT, SCITAB, and SciVer, “When Verification Fails” shows that models with high standard benchmark performance systematically lose rejection accuracy on adversarial negatives constructed so that the salient constraint is supported but a non-salient composed constraint is not (Liu et al., 13 Apr 2026). This demonstrates that evidence may be present and even correctly retrieved, while the verifier still fails to compose it into full-claim judgment.

A third mechanism is construction-sensitive NEI behavior. NEI-CAP shows that models trained on shortcut-prone NEI constructions do not transfer reliably to semantically related hard NEI. Placeholder-trained models achieve 1.000 NEI-F1 on placeholder NEI but 0.000 on BM25 near-miss and cited non-rationale hard NEI, while hard-construction training transfers substantially better (Qiu et al., 26 May 2026). Aggregate NEI scores therefore hide whether the model learned evidence absence, topic mismatch, or genuine insufficiency.

A fourth mechanism is warrant failure under correct labels. Direct prompting can yield acceptable verdict accuracy while citing non-entailing evidence. On DP, Llama3.1-8B reaches 67.4 accuracy under direct prompting but only 29.7 WSP, showing that many nominally correct Supports decisions are unsupported at the warrant level (Dey et al., 23 Jun 2026). The same paper also shows that naive 5W1H decomposition can lose up to 27.6 accuracy points, not because structure is inherently harmful, but because rigid decomposition strips away full-claim context.

Finally, there is a human-interface manifestation of the gap. PaperTrail decomposes answers and sources into claims and evidence, surfacing included and omitted paper claims, yet in a within-subjects study with 26 researchers it significantly lowered trust—4.22 ± 1.22 to 3.68 ± 1.24, $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$3—without significantly changing reliance (Martin-Boyle et al., 24 Feb 2026). This suggests that making the gap visible is not identical to making it behaviorally actionable.

5. Diagnostic and corrective methodologies

Recent work increasingly treats the claim-evidence gap as a diagnosable systems property rather than a single end-task score. Layer-wise probing and attention analysis diagnose whether evidence is absent from representations or merely fails to be routed (Kumar et al., 1 Jun 2026). Construction-aware evaluation, as in NEI-CAP, attaches provenance to each insufficient-evidence example and stress-tests transfer across evidence conditions (Qiu et al., 26 May 2026). Warrant-oriented evaluation, via WSP, checks whether cited Supports evidence entails the claim and calibrates that automatic check against human gold evidence at AUC 0.92 and precision 0.98 (Dey et al., 23 Jun 2026).

A second family of methods introduces explicit claim-evidence interfaces. CACDD frames long-form answer verification as atomic claim decomposition $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$4, arguing that evidence units align to proposition-level claims rather than to whole answers or whole sentences (Zhang et al., 2024). PaperTrail operationalizes this in scholarly QA with answer-claim and paper-claim matching (Martin-Boyle et al., 24 Feb 2026). EvidenceLens extends the same idea to financial QA through a multimodal claim-evidence matrix over claims $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$5 and evidence items $\mathcal{C}(c)=\{c_1,\ldots,c_n\}$6, making support, contradiction, support gaps, and modality imbalance explicit (Gu et al., 19 Jun 2026).

A third family targets intervention. StepGap labels reasoning-step failures as Contradicted Claim, Irrelevant Evidence, or Missing Bridge, showing that step-level F1, not question-level Q-F1, is the appropriate diagnostic for multi-hop evidence gaps (Ji et al., 23 May 2026). ToE turns verification into a dynamically expanding argument tree with RL-based multi-source retrieval and node-level veracity and reliability scores, so that retrieval continues or decomposition occurs when evidence remains weak or conflicting (Wang et al., 26 Jun 2026). In clinical summarization, VERI-DPO uses a retrieval-augmented verifier to mine contradiction-anchored, length-controlled preference pairs and then distills them into the summarizer with DPO, reducing Not Supported claim rates from 10.7\% to 1.9\% under a local verifier judge and from 11.6\% to 6.4\% under GPT-4o while improving validity from 76.7\% to 82.5\% (Liu et al., 11 Mar 2026).

Meta-evaluation itself can also exhibit a claim-evidence gap. Acceptance Cards argues that a held-out installed-gap reduction does not, by itself, warrant a safe fine-tuning defense claim; it requires conjunctive evidence for statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer (Konrad et al., 11 May 2026). This extends the concept beyond instance-level verification to claims made about evaluation results themselves.

6. Implications, misconceptions, and open directions

A recurring misconception is that poor evidence-grounded performance necessarily indicates failed perception or failed retrieval. The chart-verification results show that a model may “know” enough about the chart internally while still answering incorrectly because the evidence is not routed to the decision variable (Kumar et al., 1 Jun 2026). The NEI literature shows the parallel misconception on the negative side: predicting NEI on empty or obviously irrelevant evidence is not the same as recognizing semantically related insufficient evidence (Qiu et al., 26 May 2026). The warrant literature adds a third correction: correct verdicts do not imply warranted verdicts (Dey et al., 23 Jun 2026).

Another misconception is that citation suffices as provenance. PaperTrail and EvidenceLens both show that document- or passage-level citation is too coarse when the real issue is which exact claims are supported, contradicted, partially supported, or omitted (Martin-Boyle et al., 24 Feb 2026, Gu et al., 19 Jun 2026). In scientific writing, RIGOURATE pushes the same logic from verification to proportionality, assigning continuous overstatement scores to claims from abstracts and introductions relative to evidence from the paper body, thereby treating scientific exaggeration as a graded claim-evidence mismatch rather than a binary factual error (James et al., 7 Jan 2026).

The literature also converges on several limitations. Some diagnostics are correlational rather than causal, as in probing and attention studies (Kumar et al., 1 Jun 2026). Some benchmarks are tightly controlled but small, such as SciTabAlign+ with 162 unique claims (Kumar et al., 1 Jun 2026). Others are large but depend on automatically generated claims or LLM judges, as in ClaimDB, SciClaimHunt, and RIGOURATE (Theologitis et al., 21 Jan 2026, Kumar et al., 14 Feb 2025, James et al., 7 Jan 2026). Human disagreement remains substantial when the task concerns nuance rather than contradiction, as ClaimDiff’s Krippendorff’s alpha of 0.46 for ClaimDiff-S and 0.47 for ClaimDiff-W indicates (Ko et al., 2022). These limitations do not collapse the concept; they indicate that the claim-evidence gap is itself heterogeneous.

Taken together, these works suggest that future progress will depend less on a single monolithic verifier and more on explicit interfaces between claims, evidence units, evidence provenance, and decision rules. The most common direction is toward controlled paired datasets, claim decomposition, executable reasoning, construction-aware evaluation, architecture-specific routing interventions, and claim-level provenance displays (Kumar et al., 1 Jun 2026, Theologitis et al., 21 Jan 2026, Dey et al., 23 Jun 2026, Qiu et al., 26 May 2026). In that synthesis, the claim-evidence gap is best understood not as a narrow benchmark artifact but as a general systems problem: evidence must not only exist, but be retrievable, representable, compositionally sufficient, propositionally aligned, and decision-effective.