Scientific Claim Source Retrieval

• Scientific Claim Source Retrieval is the task of identifying documents that provide evidence for a claim, combining retrieval, rationale selection, and stance prediction.
• It employs both traditional lexical methods and modern joint modeling approaches to improve recall and precision in evidence identification.
• Recent advances extend the task to include multimodal and structured evidence, enhancing verification and source credibility assessment.

Searching arXiv for the cited scientific claim retrieval and verification papers to ground the article. arXiv search queries:

1. (Zhang et al., 2021)
2. (Wadden et al., 2020)
3. (Pradeep et al., 2020)
4. (Vladika et al., 2024)
5. (Deng et al., 25 Jun 2025)
6. (Ortega et al., 24 Mar 2025)
7. (Mohole et al., 23 Jul 2025)
8. (Bakagianni et al., 22 May 2026)
9. (Ghorbanpour et al., 19 Apr 2026) 10. (James et al., 7 Jan 2026)
10. (Wang et al., 11 Mar 2025)
11. (Biswas et al., 21 Feb 2026)
12. (Ding et al., 29 May 2026)
13. (Kargupta et al., 12 Jun 2025)
14. (Besrour et al., 17 Oct 2025)
15. (Košprdić et al., 2024)
16. (Rana et al., 2022)

Scientific claim source retrieval is the task of identifying scientific documents that provide evidence for a claim and, in many formulations, also locating the specific evidence spans and determining whether the source supports, refutes, or leaves the claim unresolved. In the SciFact formulation introduced by Wadden et al., the system receives a scientific claim and a corpus of abstracts, then predicts document-level labels and sentence-level rationales; in later work by Zhang et al., the same problem is cast as a jointly trained retrieval–rationale–stance model rather than a strictly sequential pipeline (Wadden et al., 2020, Zhang et al., 2021). In multilingual and social-media settings, the task is also described as known-item retrieval, where a short, informal claim must be matched against a large collection so that the gold paper appears in the top-$k$ results (Bakagianni et al., 22 May 2026).

1. Formal task definition and problem scope

A common formalization begins with a claim $q$ and a large corpus $\mathcal{A}$ of scientific abstracts. Each abstract $a \in \mathcal{A}$ is represented as a title $t$ and a list of sentences $S=\{s_1,\dots,s_l\}$. In the SciFact-style setting, the system predicts a binary relevance label $y^b \in \{0,1\}$ for abstract retrieval, sentence-level rationale labels $y^r_i \in \{0,1\}$, and a stance label $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$ (Zhang et al., 2021). Wadden et al. describe the same core outputs at the abstract level as $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$ together with a rationale set $q$0 (Wadden et al., 2020).

This core formulation has been generalized along several axes. In social-media source retrieval, the claim is often a short post in English, German, or French, the collection is an English-language set of approximately $q$1 publications, and the objective is to rank the collection so that the single annotated gold paper $q$2 appears in the top-$q$3 (Bakagianni et al., 22 May 2026). In peer-review verification, the input is not an already isolated claim but raw review text, which must first be segmented into atomic, check-worthy spans before retrieval can begin (Ghorbanpour et al., 19 Apr 2026). In full-paper settings, retrieval units are no longer limited to abstracts or sentences, but can include sections, passages, tables, figures, equations, and citation-linked structural units (Deng et al., 25 Jun 2025).

The task is therefore broader than document retrieval alone. The data consistently treats source retrieval as coupled to evidence identification and claim verification. This suggests that the boundary between information retrieval, natural language inference, and evidence aggregation is intrinsic rather than incidental.

2. Benchmark foundations and canonical pipeline structure

The benchmark that most strongly shaped the area is SciFact. It contains $q$4 claims split into train $q$5, dev $q$6, and test $q$7, with evidence abstracts drawn from a $q$8-abstract corpus. The label counts are $q$9 Supports, $\mathcal{A}$0 Refutes, and $\mathcal{A}$1 NoInfo; annotators also mark minimal sentence-level rationales, typically no more than three sentences (Wadden et al., 2020). The dataset was designed to require both document retrieval and rationale-backed verdict prediction rather than label assignment from preselected evidence.

The baseline SciFact pipeline, called VeriSci, is explicitly modular. First, abstracts are retrieved with TF–IDF over unigram and bigram features; in the reported SciFact setup, $\mathcal{A}$2 was chosen on the development set. Second, a rationale selector scores each sentence $\mathcal{A}$3 conditioned on the claim. Third, a label predictor classifies the claim–abstract pair as Support, Refute, or NoInfo from the predicted rationales, with the convention that an empty rationale set implies NoInfo (Wadden et al., 2020). On SciFact test with open retrieval, VeriSci reaches abstract F1 $\mathcal{A}$4 and sentence F1 $\mathcal{A}$5, whereas oracle abstracts and oracle rationales yield abstract F1 $\mathcal{A}$6 and sentence F1 $\mathcal{A}$7 (Wadden et al., 2020).

These numbers make the retrieval bottleneck explicit. Retrieval errors not only reduce recall of relevant papers but also deprive downstream modules of any chance to recover. The early literature therefore treats scientific claim source retrieval as an end-to-end pipeline whose first stage controls the attainable ceiling of final verification performance.

3. From pipelines to joint and unified models

Subsequent systems primarily differ in how tightly they couple abstract retrieval, rationale selection, and stance prediction. VERT5ERINI adapts T5 to all three subtasks in a text-to-text format: $\mathcal{A}$8 performs abstract retrieval through BM25 followed by a monoT5 reranker, $\mathcal{A}$9 performs sentence selection with the same reranking architecture, and $a \in \mathcal{A}$0 predicts one of the tokens “true,” “weak,” or “false,” mapped to Supports, NoInfo, and Refutes (Pradeep et al., 2020). On SciFact, VERT5ERINI reports abstract retrieval Recall@3 of $a \in \mathcal{A}$1 for monoT5 versus $a \in \mathcal{A}$2 for BM25, sentence selection F1 of $a \in \mathcal{A}$3, label prediction macro-F1 of $a \in \mathcal{A}$4, and hidden-test abstract Label+Rationale F1 of $a \in \mathcal{A}$5 (Pradeep et al., 2020).

RerrFact retains a modular design but reduces the abstract representation before neural classification. Instead of feeding the full abstract, it constructs a four-span representation $a \in \mathcal{A}$6, uses TF–IDF cosine similarity to seed the top-$a \in \mathcal{A}$7 abstracts, and then applies stage-wise binary classifiers for abstract relevance, rationale selection, and a two-step stance decision. On SciFact test, RerrFact reports Selection+Label F1 $a \in \mathcal{A}$8, with the two-step stance classifier improving dev F1 to $a \in \mathcal{A}$9 relative to a single three-way alternative (Rana et al., 2022).

ARSJoint is the clearest move away from a loose pipeline. Zhang et al. formulate abstract retrieval, rationale selection, and stance prediction as a joint machine-reading-comprehension framework over a sequence of the form

$t$0

encoded by BioBERT-large or RoBERTa-large (Zhang et al., 2021). The model uses a hierarchical attention network, feeds claim information into all modules, and trains with a single objective

$t$1

where $t$2 is a rationale-regularization term aligning abstract-retrieval sentence attention with rationale predictions (Zhang et al., 2021). On the SciFact development set, ARSJoint with BioBERT reports sentence-level Selection-Only $t$3, $t$4, $t$5; sentence-level Selection+Label $t$6, $t$7, $t$8; abstract-level Label-Only $t$9, $S=\{s_1,\dots,s_l\}$0, $S=\{s_1,\dots,s_l\}$1; and abstract-level Label+Rationale $S=\{s_1,\dots,s_l\}$2, $S=\{s_1,\dots,s_l\}$3, $S=\{s_1,\dots,s_l\}$4, outperforming the listed prior models (Zhang et al., 2021).

The main methodological dispute in this phase of the literature is therefore pipeline versus joint training. The pipeline view emphasizes modularity and replaceable components; the joint view emphasizes error propagation, parameter sharing, and explicit inter-task consistency.

4. Retrieval models, knowledge sources, and open-domain settings

Once the task moves beyond a fixed abstract corpus, the choice of retrieval model and knowledge source becomes a first-order design decision. Vladika and Matthes hold evidence selection and verdict prediction constant while varying document retrieval across PubMed, Wikipedia, and Google, using BM25 and semantic search. With $S=\{s_1,\dots,s_l\}$5, they report that PubMed works better with specialized biomedical claims, Wikipedia is more suited for everyday health concerns, BM25 excels in retrieval precision, and semantic search improves recall of relevant evidence (Vladika et al., 2024). The reported verdict-level results reflect the trade-off: on PubMed for SCIFACT, BM25 yields $S=\{s_1,\dots,s_l\}$6 precision/recall/F$S=\{s_1,\dots,s_l\}$7, while semantic retrieval yields $S=\{s_1,\dots,s_l\}$8; on Wikipedia for COVERT, BM25 yields $S=\{s_1,\dots,s_l\}$9, while semantic retrieval yields $y^b \in \{0,1\}$0 (Vladika et al., 2024).

Short, informal, and multilingual claims create additional mismatch between query language and scientific prose. In CheckThat! 2026 Task 1, the retrieval target is a single gold paper in a $y^b \in \{0,1\}$1-document English collection, and the proposed system uses a three-stage architecture: a dual-encoder dense retriever based on BAAI/bge-large-en-v1.5, a multilingual cross-encoder reranker based on jinaai/jina-reranker-v2-base-multilingual, and an LLM-based disagreement resolver invoked only when the first two stages disagree (Bakagianni et al., 22 May 2026). The paper’s main contribution is cluster-aware hard-negative mining. For dense retrieval, macro Recall@20 rises from $y^b \in \{0,1\}$2 with in-batch negatives to $y^b \in \{0,1\}$3 with cluster-non-gold negatives; for reranking, macro MRR@5 rises from $y^b \in \{0,1\}$4 to $y^b \in \{0,1\}$5. The final system ranks $y^b \in \{0,1\}$6th among $y^b \in \{0,1\}$7 submissions (Bakagianni et al., 22 May 2026).

A related CheckThat! 2025 system from DS@GT keeps retrieval and reranking simpler: BM25-PyTorch followed by one of several rerankers over the top $y^b \in \{0,1\}$8 documents per tweet. Its official submission uses a T5 reranker and achieves MRR@5 $y^b \in \{0,1\}$9 on the blind test set, compared with the organizers’ BM25 baseline of $y^r_i \in \{0,1\}$0 (Schofield et al., 9 Jul 2025). The same study reports that replacing the tweet entirely with an LLM rewrite harms performance, while concatenating the original tweet with a formal rewrite gives consistent small gains (Schofield et al., 9 Jul 2025).

Open-domain scientific claim source retrieval is thus not tied to a single corpus or a single retrieval paradigm. The data supports a recurring pattern: lexical retrieval remains a strong precision-oriented baseline, dense retrieval improves semantic coverage, and multi-stage systems increasingly separate candidate generation from fine-grained reranking.

5. Evidence granularity, verification, and source quality

A second major axis of variation concerns what counts as retrievable evidence. Abstract-only benchmarks simplify the task, but several systems explicitly move to passage-level or full-document evidence. Peerispect processes peer reviews by extracting factual spans, segmenting the manuscript into overlapping passages of fixed length, retrieving with BM25, dense retrieval, or a hybrid union of top-$y^r_i \in \{0,1\}$1 sparse and top-$y^r_i \in \{0,1\}$2 dense results, reranking the top-$y^r_i \in \{0,1\}$3 candidates with a BERT-large cross-encoder, and then performing four-way NLI with labels {Supported, Partially Supported, Contradicted, Undetermined} (Ghorbanpour et al., 19 Apr 2026). On its Controlled Manuscript Claims and Real-World Review Claims benchmarks, the paper reports that hybrid retrieval plus reranking gives CMC accuracy approximately $y^r_i \in \{0,1\}$4 and RRC accuracy approximately $y^r_i \in \{0,1\}$5, the best result on real reviews (Ghorbanpour et al., 19 Apr 2026).

The perspective paper on “advanced evidence retrieval from complex structured academic papers” argues that real-world scientific fact-checking requires structured document parsing, time-aware retrieval, multimodal handling of tables, figures, and equations, and explicit credibility assessment (Deng et al., 25 Jun 2025). In the sketched architecture, the retrieval score combines lexical-semantic relevance $y^r_i \in \{0,1\}$6, verifier feedback $y^r_i \in \{0,1\}$7, and credibility $y^r_i \in \{0,1\}$8 into

$y^r_i \in \{0,1\}$9

with additional time weighting through $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$0 (Deng et al., 25 Jun 2025). Preliminary experiments in the same paper report that verifier feedback improves Recall@5 on SciFact-Open from $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$1 to $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$2, dynamic cutoff reduces verifier calls by $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$3–$$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$4 with at most a $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$5 F1 drop, section-aware retrieval recovers $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$6 more relevant Results paragraphs, and multimodal fusion improves chart-based claim verification by $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$7–$$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$8 F1 over text-only (Deng et al., 25 Jun 2025).

Quality-aware retrieval goes further in VERIRAG, which adds a structured audit of methodological rigor. Each paper is assigned a stance $$y^e \in \{0\;(\text{“SUPPORTS”}),1\;(\text{“REFUTES”}),2\;(\text{“NOINFO”})\}$$9 and an audit vector $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$0 over an 11-point checklist covering Data Quality and Inferential Validity, then aggregated into a Hard-to-Vary score and evaluated against a Dynamic Acceptance Threshold (Mohole et al., 23 Jul 2025). Across four temporal scenarios, VERIRAG reports macro-F1 scores $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$1, $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$2, $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$3, and $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$4, outperforming the next-best baseline by $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$5–$$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$6 points in each scenario (Mohole et al., 23 Jul 2025). A plausible implication is that source retrieval in scientific settings is increasingly being asked not only to find relevant papers, but also to distinguish scientifically credible from merely semantically similar evidence.

6. Multi-source aggregation, contradiction, and emerging retrieval representations

Several recent systems treat disagreement itself as a retrieval target. CIBER retrieves the top-$$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$7 candidate documents with semantic embeddings and FAISS, then interrogates each document with original, agreement, and conflict probes, collecting Support, Refute, and Neutral labels over repeated LLM calls before fusing them with Weighted Proportions, Weighted Information Gain, or Dempster–Shafer-style belief updates (Wang et al., 11 Mar 2025). On the combined evaluation, GPT-3.5 with CIBER reaches accuracy $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$8 and macro-F1 $$y(c,a)\in\{\text{Supports},\text{Refutes},\text{NoInfo}\}$$9, compared with $q$00 and $q$01 for a standard RAG baseline (Wang et al., 11 Mar 2025).

A related open-domain system explicitly retrieves evidence for both the original claim $q$02 and its negation $q$03 across Wikipedia, PubMed, and Google, filters sentences with SPICED embeddings, removes exact overlaps through symmetric difference, and aggregates the remaining evidence for LLM verification (Biswas et al., 21 Feb 2026). The stated purpose is to guard against confirmation bias and capture both supporting and contradicting information from diverse sources (Biswas et al., 21 Feb 2026). This suggests a shift from single-pool evidence retrieval toward retrieval strategies that model contradiction, counter-evidence, and source-level disagreement as first-class objects.

Other work changes the retrieval substrate itself. “Reading Between the Citations” proposes a typed claim network in which each cross-document reference is reified as a claim node carrying source, target, claim text, and stance labels such as Critique, Adoption, Benchmark, and Neutral (Ding et al., 29 May 2026). Retrieval then fuses the original question stream, claim-derived queries, and linked-paper boosts with Reciprocal Rank Fusion rather than relying on flat chunk search alone (Ding et al., 29 May 2026). ClaimSpect builds an aspect tree over a claim and ranks corpus segments by a discriminativeness score that rewards relevance to a target aspect and penalizes relevance to sibling aspects, enabling hierarchical retrieval for nuanced claims (Kargupta et al., 12 Jun 2025). SQuAI, operating over approximately $q$04 million arXiv full-text papers, decomposes complex questions into sub-questions, retrieves evidence with hybrid BM25 and E5 scoring, adaptively filters candidate evidence, and generates answers with in-line arXiv citations and supporting sentences; on three benchmarks it improves the average of answer relevance, contextual relevance, and faithfulness by $q$05 to $q$06 over a strong RAG baseline (Besrour et al., 17 Oct 2025).

Across these systems, scientific claim source retrieval is no longer defined solely as “find the matching abstract.” It increasingly includes typed inter-paper claims, hierarchical aspect discovery, explicit contradiction retrieval, and citation-grounded generation. The literature therefore points toward a broader conception in which retrieval is responsible for coverage, provenance, disagreement, and evidential structure simultaneously.