Fact Verification: Evaluating Textual Claims

• Fact Verification (FV) is an evaluative paradigm that systematically assesses the factuality of textual claims using external evidence and algorithmic comparisons.
• It employs a two-stage process featuring evidence retrieval with methods like BM25 and neural inference models such as fine-tuned NLI classifiers.
• Hybrid approaches integrating FV with Hallucination Detection enhance evaluation metrics like AUC, accuracy, and F1, supporting robust LLM deployments.

Fact Verification (FV) is an evaluative paradigm designed to systematically assess the factuality of textual claims, most often in the context of outputs generated by LLMs. FV leverages both external evidence sources and algorithmic methods—ranging from retrieval-based comparison to sophisticated neural inference—to estimate the probability that a given output is consistent with established knowledge. In current research, FV is especially critical for tasks demanding high reliability, such as scientific QA, medical document analysis, or real-world deployment of LLM systems. Increasingly, FV is studied both in isolation and in relation to Hallucination Detection (HD), with new research (e.g., the UniFact framework (Su et al., 2 Dec 2025)) explicitly connecting and integrating these paradigms to drive progress on robust model factuality.

1. Formal Definition and Task Formulation

Fact Verification (FV) is formulated as origin-agnostic, text-centric binary classification. Given:

• a textual claim $y$
• a set of retrieved “trusted” evidence passages $\mathcal{E} = \{e_k\}_{k=1}^K$

FV estimates the probability that $y$ is non-factual given $\mathcal{E}$: $$p_{\mathrm{FV}}(y, \mathcal{E}) = D_{\mathrm{FV}}(y, \mathcal{E}) \in [0,1],$$ where $D_{\mathrm{FV}}$ is a discriminative model, typically a natural language inference (NLI) or a pretrained LLM verifier, and higher scores indicate conflict with the evidence (i.e., likely non-factual).

Key distinction: FV operates on arbitrary input claims $y$ whether or not they were generated by LLMs. It is not model-specific and treats $y$ as a static assertion independent of provenance. This origin-agnostic stance is in contrast with Hallucination Detection (HD), which focuses on LLM-specific generative artifacts and may exploit model-internal signals (Su et al., 2 Dec 2025).

2. Methodological Frameworks and Algorithms

Standard FV pipelines consist of two main stages:

1. Evidence Retrieval: For each claim $y$ (and optionally its generating prompt $x$), a retrieval system such as BM25, DPR, or ColBERT selects top-$K$ evidence passages from a high-quality corpus (e.g., Wikipedia).
2. Claim-Evidence Comparison: The claim $y$ and retrieved passages $\mathcal{E}$ are fed to a discriminative model $D_{\mathrm{FV}}$—commonly either a fine-tuned BERT-based NLI model or a prompting-based LLM verifier. The model outputs a verdict (Supported, Contradicted, Not Enough Information) or a continuous factuality score.

Variants in the literature include:

• LLM-Q: LLM verifier using question $q$ for retrieval.
• LLM-QA: LLM verifier using ($q$, $y$) for retrieval.
• BERT-Q: BERT-NLI classifier using $q$ for retrieval.
• BERT-QA: BERT-NLI classifier using ($q$, $y$) for retrieval.

Decision thresholds or continuous scores are used to produce binary or calibrated factuality judgements.

3. Relationship to Hallucination Detection and Unified Evaluation

Historically, FV evolved from information retrieval and fake-news detection, targeting static, origin-agnostic benchmarks such as FEVER. By contrast, HD focuses on LLM-generative outputs, seeking model-internal uncertainty signals such as logit entropy, token-level activation dynamics, or cross-sample consistency. The two paradigms have typically employed non-overlapping datasets, metrics, and evaluation protocols (Su et al., 2 Dec 2025).

The UniFact framework unifies FV and HD in a single instance-level comparison protocol. It dynamically generates LLM outputs, labels their factuality via a separate “judge” model with held-out gold evidence, and supplies all necessary input to both FV and HD methods for fair benchmarking.

Notable findings:

• No paradigm is universally superior; FV and HD achieve comparable state-of-the-art performance depending on dataset, model, and metric.
• The methods exhibit complementary strengths: FV excels when retrieval systems surface relevant, veridical evidence, while HD captures generative inconsistencies even outside the corpus scope.
• Hybrid approaches (score fusion, evidence-aware pipelines) that integrate both FV and HD consistently set new AUC benchmarks (Su et al., 2 Dec 2025).

4. Benchmarks and Quantitative Performance

FV has been evaluated on both classic datasets (FEVER, MultiNLI) and newly dynamically-generated QA corpora through UniFact:

• TriviaQA
• NQ-Open
• PopQA
• 2WikiMultihopQA (Bridge, Comp)
• HotpotQA-Comp

On these tasks, BERT-QA and LLM-QA variants have achieved AUC scores up to 0.8256 (PopQA), often outperforming model-centric HD on certain test splits. However, in other domains and models, HD methods such as LNPE can be competitive or superior.

Metrics used include AUC, accuracy, F1, precision, recall, and calibrated ensemble synergy scores. FV methods often rely on high-quality retrieval, and their performance varies with the ability to find relevant evidence.

5. Limitations, Synergies, and Future Directions

Limitations:

• FV is bounded by the coverage and retrieval quality of the evidence corpus. Out-of-knowledge claims challenge its effectiveness.
• Static datasets (FEVER, etc.) do not reflect evolving error modes in real LLM generations.
• “NEI” (Not Enough Information) outcomes yield indeterminate predictions, requiring fallback or ensemble methods (Su et al., 2 Dec 2025).

Synergies:

• Combining FV with HD leads to superior coverage, error correction rates, and robustness, capturing more diverse factual errors.
• Score-level fusion and evidence-aware pipelines demonstrated best-in-class performance, outperforming standalone methods by up to 10 points in AUC and F1.

Future Directions:

• Joint optimization of retrieval and generative uncertainty models.
• Extension to multimodal verification (e.g., incorporating vision-language evidence).
• Advanced calibration strategies for aligning differing signal strengths from FV and HD.
• Perpetually dynamic benchmarking as implemented in UniFact to maintain up-to-date evaluation across emerging LLMs.

6. Taxonomy and Research Schism

FV, along with HD, forms a comprehensive taxonomy for factuality evaluation in generative AI:

• FV: Text-centric, evidence-based, origin-agnostic.
• HD: Model-centric, uncertainty-based, generation-specific.
• Unified factuality: Hybrid methods combining complementary signals for robust detection.

The research schism between FV and HD is a consequence of distinct operational assumptions, benchmarks, venues, and evaluation methodologies. UniFact (Su et al., 2 Dec 2025) demonstrates the essential need for unified frameworks, arguing for an integrated research agenda and providing open-source infrastructure to support direct, up-to-date comparative studies.

7. Practical Implications for LLM Deployment

Fact Verification remains indispensable for real-world LLM usage in settings where external grounding and accountability are required. State-of-the-art FV systems deliver competitive accuracy, scalable evaluation, and interpretability for both claim-level and instance-level assessment. However, their practical deployment should consider corpus relevance, retrieval latency, and ensemble integration with model-centric HD signals to maximize reliability and robustness in demanding applications (Su et al., 2 Dec 2025).