Previously Fact-Checked Claim Detection

• PFCD is a process that determines if a new claim matches an existing verified claim by ranking archived fact-checks based on relevance.
• It employs a two-stage pipeline combining high-recall lexical retrieval and semantic reranking to efficiently surface relevant fact-checks.
• Recent methods integrate multilingual retrieval and LLM-based filtering to improve cross-lingual performance and mitigate redundancy.

Previously Fact-Checked Claim Detection (PFCD), also called claim matching or previously fact-checked claim retrieval, is the task of determining whether a new claim, post, or sentence corresponds to a claim that has already been checked by professional fact-checkers, and of returning the most relevant prior fact-checks. In the literature, PFCD is positioned as an early-stage fact-checking function: before retrieving open-web evidence or predicting veracity from scratch, a system queries an archive of verified claims in order to avoid duplicated work, accelerate verification, and expose a reusable human-written explanation through the associated fact-check article (Shaar et al., 2020, Nakov et al., 2021, Vykopal et al., 4 Mar 2025).

1. Conceptual scope and relation to adjacent tasks

PFCD was formalized as a ranking problem in which, given a check-worthy input claim and a set of verified claims, the system ranks the verified claims so that those that can help verify the input claim, or a sub-claim in it, are placed above those that are not useful (Shaar et al., 2020). This formulation immediately separates PFCD from several nearby tasks. It is not merely paraphrase detection, because a useful fact-check may be lexically distant from the input claim. It is not standard NLI or RTE, because the operational question is not entailment between two isolated sentences but whether an existing verified claim is useful for verifying a new claim. It is also distinct from end-to-end fact verification, whose goal is to assign a truth label rather than to retrieve a reusable prior fact-check (Shaar et al., 2020).

Survey work on automated fact-checking places PFCD under the “similarity” dimension of claim detection, alongside verified claim retrieval and claim matching, and distinguishes it from “verifiability” and “priority” detection (Panchendrarajan et al., 2024). In that taxonomy, PFCD addresses repeated or semantically related claims that should not trigger redundant human verification. A central motivation, stated repeatedly across the literature, is that misinformation recurs across time, platforms, and languages, while human fact-checking capacity remains limited (Panchendrarajan et al., 2024, Vykopal et al., 4 Mar 2025).

PFCD also has a broader document-level interpretation. In debate and transcript settings, the system may be asked to rank all sentences in a document so that those containing claims verifiable by a previously fact-checked claim appear as high as possible, together with the corresponding evidence (Shaar et al., 2021). This formulation expands PFCD from query-to-archive matching into a prioritization problem over long inputs.

2. Task formulations and pipeline role

The dominant historical formulation treats PFCD as retrieval or ranking. CLEF CheckThat! Task 2 defined it as follows: given a claim in a tweet, debate, or speech, and a set of previously fact-checked claims, rank the verified claims in order of their usefulness for fact-checking the new claim (Nakov et al., 2021). In this setting, the output is a top-$n$ ranked list rather than a binary label.

More recent work also uses a pairwise classification formulation. In multilingual PFCD, a post and a candidate fact-checked claim are presented as a pair, and the system predicts whether the claim is relevant or irrelevant. Relevance is defined pragmatically: a fact-checked claim is relevant when it is implied by the statements in the post and could help a fact-checker verify the post’s veracity (Vykopal et al., 4 Mar 2025). This formulation is typically used as the second stage of a pipeline in which an initial retriever supplies candidate claims and a stronger classifier removes false positives (Vykopal et al., 4 Mar 2025).

A more elaborate human-centered formulation extends PFCD beyond matching alone. One deployed multilingual system treats PFCD as a pipeline that retrieves candidate fact-checks, filters irrelevant ones with an LLM, summarizes relevant fact-checks in English, and predicts claim veracity from the retrieved evidence (Vykopal et al., 29 Apr 2025). In this architecture, PFCD is not full verification; it is the retrieval-relevance component that determines whether existing fact-checks can be reused.

These formulations are complementary rather than contradictory. Ranking is natural when the archive is large and relevance is graded operationally. Binary classification is natural once retrieval has reduced the search space to candidate pairs. Document-level sentence ranking is natural when the input is a long transcript and only some sentences are claim-bearing (Nakov et al., 2021, Vykopal et al., 4 Mar 2025, Shaar et al., 2021).

3. Datasets, benchmarks, and annotation regimes

PFCD research has been shaped by a progression from small event-specific datasets to multilingual corpora and weakly supervised collections. Representative resources include CT–VCR–21 for CheckThat!, MultiClaim for multilingual retrieval, AMC-16K for multilingual relevance classification, CrowdChecked for large noisy supervision, and MultiClaimNet for claim clustering (Nakov et al., 2021, Pikuliak et al., 2023, Vykopal et al., 4 Mar 2025, Hardalov et al., 2022, Panchendrarajan et al., 28 Mar 2025).

Resource	Coverage	PFCD role
CT–VCR–21	2A Arabic: 858 input tweets, 1,039 Input–VerClaim pairs, 30,329 verified claims; 2A English: 1,401 tweets, 1,401 pairs, 13,835 verified claims; 2B English: 669 input claims, 804 pairs, 19,250 verified claims	CLEF-2021 ranking benchmark
MultiClaim	205,751 fact-checks, 28,092 social media posts, 31,305 post–fact-check connections; 39 fact-check languages, 27 post languages	large multilingual retrieval corpus
AMC-16K	16,000 annotated post–claim pairs; 8,000 monolingual and 8,000 cross-lingual; 20 languages	multilingual pair-classification evaluation
CrowdChecked	332,660 unique tweet–article pairs, 316,564 unique tweets, 10,340 Snopes fact-checking articles	weak supervision for PFCD ranking
MultiClaimNet	54.4K claim pairs, 30.9K clusters, 85.3K claims, 78 languages	claim clustering for scalable PFCD

The earlier PFCD literature also introduced smaller English datasets built from PolitiFact and Snopes. One foundational dataset contains 768 PolitiFact input–verified-claim pairs derived from 78 public events between 2012 and 2019, with 16,636 verified claims in the archive; the companion Snopes dataset contains 1,000 input claims paired with 10,396 verified claims (Shaar et al., 2020). These resources established the retrieval-and-reranking protocol later adopted in evaluation campaigns.

Manual annotation remains central in multilingual PFCD. AMC-16K was created as a manually annotated subset of MultiClaim with six annotators and labels Yes, No, and Cannot tell, later resolved to binary decisions; agreement was reported as Fleiss’ kappa 0.60 pre-annotation and 0.62 post-annotation (Vykopal et al., 4 Mar 2025). Document-level PFCD annotation is more involved still: one corpus built from seven political debates begins with 5,054 transcript sentences, filters them to 700 check-worthy sentences, and then manually annotates candidate links to a PolitiFact archive, resulting in 125 input sentences verifiable by the database and 198 verifying pairs (Shaar et al., 2021).

The largest resources also expose a central evaluation difficulty: gold links are often incomplete. MultiClaim explicitly warns that professional fact-checker links do not exhaust all valid post–fact-check matches, and later multilingual pair-classification work notes that recall cannot be measured exactly because annotations are not exhaustive (Pikuliak et al., 2023, Vykopal et al., 4 Mar 2025).

4. Retrieval, reranking, and pre-LLM modeling strategies

Early PFCD systems converged on a two-stage design: a high-recall lexical retriever followed by a semantic reranker. The initial PFCD study used BM25 over fields such as verified claim, title, and body, then reranked retrieved candidates with RankSVM using BM25 scores, sentence-BERT similarities, and reciprocal ranks; on PolitiFact, the best reranker reached MRR = 0.608, improving over BM25 Body = 0.565, while on Snopes the best reranker reached MRR = 0.788 over a best BM25 baseline of 0.664 (Shaar et al., 2020). CheckThat! system submissions followed the same pattern: bigIR used BM25 retrieval plus an AraBERT reranker for Arabic, while Aschern combined TF-IDF, fine-tuned Sentence-BERT, and LambdaMART for English (Nakov et al., 2021).

Context modeling became important once PFCD moved beyond isolated tweets. In political debates, claims are often fragmented, referential, or embedded in surrounding discourse. A dedicated study categorized examples as clean, clean-hard, part-of, and context-dep, and showed that source-side local context and co-reference resolution matter most. On the realistic Debate-Level Chrono split, the baseline MAP dropped from 0.602 under the earlier sentence-level random split to 0.429, while src-coref + FC(3,1) reached 0.532, a gain of about 10.3 MAP points absolute (Shaar et al., 2021). This result established PFCD as a contextual matching problem rather than a pure sentence similarity problem.

Article-aware reranking has also become specialized. MTM, a reranker for fact-checking articles, uses a ROUGE-guided Transformer to capture lexical and semantic event information and a Pattern Memory Bank to capture recurring debunking templates. It selects key sentences from each article and uses them for final relevance prediction and explanation. MTM reports MRR = 0.931 on Twitter and MRR = 0.902 on Weibo, with human evaluation showing that it identifies at least one key sentence in 83.0% of sampled long Weibo articles (Sheng et al., 2021).

Two other lines of work target the supervision bottleneck and the query formulation problem. CrowdChecked mines social-media replies containing Snopes links and builds a noisy dataset of 332,660 tweet–article pairs; with modified self-adaptive training, trainable temperature, similarity-group shuffling, and reranking, it reaches MAP@5 90.3 on the CLEF’21 test set, about 2 MAP@5 points above the prior state of the art (Hardalov et al., 2022). A separate “summarize-then-retrieve” workflow rewrites noisy social-media claims into concise abstractive queries; it improves from 9.52% Recall@5 and 0.09 MRR under raw verbatim querying to 34.91 Recall@5 and 0.30 MRR with fine-tuned Distilled PEGASUS (Bhatnagar et al., 2022).

5. Multilingual and LLM-based PFCD

Multilingual PFCD emerged when retrieval moved from English and Arabic benchmarks to global archives. MultiClaim expanded the setting to 205,751 fact-checks in 39 languages, 28,092 posts in 27 languages, and 31,305 post–fact-check connections, and showed that translation into English often beats direct multilingual retrieval. In that benchmark, the best English-version methods outperformed the best original-language methods by about 14% S@10 for monolingual retrieval and about 35% S@10 for crosslingual retrieval (Pikuliak et al., 2023).

LLM-based PFCD introduces a different second-stage decision layer. A comprehensive multilingual study evaluated 7 open-source LLMs across 20 languages in monolingual and cross-lingual settings and tested five prompting strategies: zero-shot, zero-shot plus task description, few-shot plus task description, chain-of-thought, and cross-lingual-thought, where the latter translates the post and claim into English before deciding relevance (Vykopal et al., 4 Mar 2025). The main findings were that the best models reached Macro F1 above 80% in some settings, 70B+ models consistently outperform 10B- models, performance is substantially stronger for high-resource than for low-resource languages, and cross-lingual PFCD is harder by about 4.5 Macro F1 points on average. Translation into English usually helps low-resource languages and non-Latin scripts, but not universally; few-shot settings can be harmed by translation for some models (Vykopal et al., 4 Mar 2025).

A related multilingual system integrates dense retrieval, LLM filtering, summarization, and veracity prediction. It retrieves the top 50 fact-checks with Multilingual E5 Large, then uses an LLM to keep only directly relevant fact-checks and explain their relevance. For direct retrieval over 20 languages, Multilingual E5 Large achieved S@10 = 0.84, compared with 0.83 for GTR-T5-Large, 0.82 for BGE-M3, and 0.62 for BM25. In the filtration stage, Mistral Large gave the best trade-off with Macro F1 = 59.82, TNR = 90.23, and FNR = 15.38 (Vykopal et al., 29 Apr 2025). The same system reports that retrieved fact-checks dramatically improve veracity prediction, with Mistral Large rising from a baseline Macro F1 = 26.53 to 63.05 when retrieved evidence is provided (Vykopal et al., 29 Apr 2025).

SemEval-2025 systems further illustrate the multilingual design space. One submission fine-tuned multilingual and English bi-encoders, used machine translation for crosslingual alignment, and combined models by majority voting; its final system reached 85% crosslingual and 92% monolingual Success@10 (Azadi et al., 12 Mar 2025). Another used a two-stage pipeline with LLM-based translation into English, a fine-tuned Stella 400M dense retriever, Qwen2.5-72B-Instruct reranking, and Reciprocal Rank Fusion, reporting 0.938 on the monolingual test set and 0.81025 on the crosslingual test set (Devadiga et al., 23 Apr 2025). Across these systems, a recurring engineering conclusion is that translation to a pivot language can be simpler and stronger than direct multilingual retrieval when high-quality English retrievers are available (Pikuliak et al., 2023, Devadiga et al., 23 Apr 2025).

6. Evaluation, biases, and scaling directions

PFCD evaluation depends on task formulation. Ranking benchmarks use MAP, MRR, Precision@$k$, Recall@$k$, HIT@$k$, and Success@$K$, while pair-classification benchmarks use Macro F1, TNR, and FNR (Nakov et al., 2021, Vykopal et al., 4 Mar 2025). In multilingual retrieval, Success-at-$K$ is often the main measure:

$\mathrm{S@K} = \frac{1}{N}\sum_{i=1}^{N} \mathbb{I}\left[\mathrm{rank}(d_i^\* \mid q_i) \le K\right]$

where $d_i^\*$ is the desired fact-check for query $q_i$ (Pikuliak et al., 2023). Document-level PFCD introduced stricter variants such as $AP_H$, which give credit only when a sentence is ranked highly and at least one relevant verified claim is also retrieved (Shaar et al., 2021).

Two major limitations recur across the literature. The first is label incompleteness. MultiClaim notes that many correct retrieved fact-checks may be counted as false positives because the dataset contains only the links surfaced by the collection pipeline, not all valid matches (Pikuliak et al., 2023). AMC-16K makes the same point from the annotation side: because labeling is not exhaustive, recall cannot be measured exactly and evaluation is effectively precision-based through sampled annotations (Vykopal et al., 4 Mar 2025). The second limitation is semantic ambiguity. Common PFCD failure modes include topic similarity mistaken for actual relevance, missing context, contradictions in reasoning, wrong output language, repeating sequences, and refusals (Vykopal et al., 4 Mar 2025).

Bias analysis has added a further layer of scrutiny. A 2025 study on language and retrieval bias shows that multilingual LLMs usually perform best in English and worse on low-resource or non-Latin-script languages, although richer prompts and larger models reduce that gap (Vykopal et al., 29 Sep 2025). The same work defines retrieval bias as the tendency of retrievers to over-surface certain popular or generic claims. Using dense retrieval on the MultiClaim test split of 1,239 post-claim pairs, it reports Success@1 38.80% and Success@20 88.50% for Multilingual E5, and shows that some claims are retrieved disproportionately often, including over-retrieved COVID-related and Ukraine/Russia claims as well as noisy artifacts such as truncated JavaScript and raw social-media URLs (Vykopal et al., 29 Sep 2025). A plausible implication is that aggregate retrieval metrics can overstate robustness when head claims dominate the retrieved set.

Scaling PFCD beyond pairwise matching has therefore become a prominent research direction. The survey literature explicitly noted that there was no annotated dataset for claim clusters, even in monolingual settings (Panchendrarajan et al., 2024). MultiClaimNet addresses this gap with three multilingual claim-cluster datasets spanning 86 unique languages overall, including a large resource with 54.4K claim pairs, 30.9K clusters, and 85.3K claims in 78 languages (Panchendrarajan et al., 28 Mar 2025). In this formulation, clusters are formed by linking similar claims and treating connected components as reusable fact-level units. This suggests a shift from retrieving one prior fact-check to retrieving a cluster of paraphrases, translations, and recurrent narratives.

PFCD also increasingly interfaces with broader evidence-centric verification systems. FAKTA combines retrieval, stance detection, sentence-level rationale extraction, and linguistic analysis in an end-to-end fact-checking framework, while FactDetect extracts short facts from evidence and uses them as intermediate reasoning objects in claim verification (Nadeem et al., 2019, Jafari et al., 2024). These systems are not dedicated PFCD benchmarks, but they indicate that retrieval, explanation, and downstream verdict reasoning are converging. For operational fact-checking, the resulting picture is consistent: PFCD is most effective when treated not as an isolated similarity score, but as a retrieval-relevance layer embedded in a human-centered pipeline that must handle archive scale, contextual ambiguity, multilingual transfer, and explanation quality.