When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection (2511.04643v1)

Abstract: The proliferation of misinformation necessitates robust yet computationally efficient fact verification systems. While current state-of-the-art approaches leverage LLMs for generating explanatory rationales, these methods face significant computational barriers and hallucination risks in real-world deployments. We present DeReC (Dense Retrieval Classification), a lightweight framework that demonstrates how general-purpose text embeddings can effectively replace autoregressive LLM-based approaches in fact verification tasks. By combining dense retrieval with specialized classification, our system achieves better accuracy while being significantly more efficient. DeReC outperforms explanation-generating LLMs in efficiency, reducing runtime by 95% on RAWFC (23 minutes 36 seconds compared to 454 minutes 12 seconds) and by 92% on LIAR-RAW (134 minutes 14 seconds compared to 1692 minutes 23 seconds), showcasing its effectiveness across varying dataset sizes. On the RAWFC dataset, DeReC achieves an F1 score of 65.58%, surpassing the state-of-the-art method L-Defense (61.20%). Our results demonstrate that carefully engineered retrieval-based systems can match or exceed LLM performance in specialized tasks while being significantly more practical for real-world deployment.

• The paper demonstrates that replacing LLM generation with dense retrieval-classification significantly improves fact verification accuracy and efficiency.
• It employs dense text embeddings and FAISS for rapid evidence extraction, achieving a 95% reduction in runtime compared to LLM-based systems.
• Results show superior performance on RAWFC and LIAR-RAW benchmarks, underscoring its potential for real-time fake news detection.

Dense Evidence Retrieval for Scalable Fake News Detection: An Analysis of DeReC

Introduction

The paper "When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection" (2511.04643) introduces DeReC, a dense retrieval-classification framework for automated fact verification. The work addresses the computational and reliability limitations of LLM-based fact-checking systems, particularly those that generate natural language rationales. DeReC leverages dense text embeddings and efficient similarity search to ground veracity predictions in retrieved evidence, bypassing the need for generative LLMs. The system demonstrates superior accuracy and efficiency on standard benchmarks, challenging the prevailing assumption that LLM-based generation is necessary for high-performance fact verification.

Motivation and Background

Automated fact verification has become increasingly critical due to the proliferation of misinformation. State-of-the-art systems often employ LLMs to generate explanations for their decisions, but these approaches are hindered by three main issues:

1. Computational inefficiency: LLM inference, especially for explanation generation, is resource-intensive and slow, making real-time deployment impractical.
2. Hallucination risk: LLM-generated rationales can contain factual errors or inconsistencies, undermining trust in the system.
3. Lack of evidence grounding: Generated explanations may not be directly linked to verifiable sources, reducing transparency.

Retrieval-Augmented Generation (RAG) frameworks have attempted to mitigate some of these issues by incorporating external evidence into LLM prompts. However, the generative step remains a bottleneck. DeReC proposes a paradigm shift: replace the generative component with a targeted classifier that directly consumes retrieved evidence, thus improving both efficiency and reliability.

System Architecture

DeReC is structured as a three-stage pipeline:

1. Evidence Extraction: All sentences from a corpus of raw media reports are embedded using a dense text embedding model. The paper evaluates two models: Alibaba-NLP/gte-Qwen2-1.5B-instruct (1.5B parameters) and nomic-ai/nomic-embed-text-v1.5 (137M parameters). Embeddings are generated via contrastive learning objectives to ensure semantic similarity is preserved in the vector space.
2. Evidence Retrieval: For each claim, the same embedding model encodes the claim text. FAISS (Facebook AI Similarity Search) is used to perform efficient cosine similarity search over the evidence embeddings, retrieving the top-$k$ (empirically $k=10$) most relevant sentences. The FAISS index is constructed with normalized vectors, enabling sub-linear search complexity.
3. Veracity Prediction: The claim and retrieved evidence sentences are concatenated and fed into a DeBERTa-v3-large classifier, fine-tuned for multi-class veracity prediction. The input format is:

   [CLS] claim [SEP] evidence_1 [SEP] ... [SEP] evidence_k [SEP]

   The [CLS] token's contextual representation is used for classification via a softmax layer. Training minimizes cross-entropy loss.

This architecture eliminates the need for autoregressive generation, resulting in a computational complexity of $O(l + \log s)$, where $l$ is sequence length and $s$ is corpus size, compared to $O(n \times l^2)$ for LLM-based generation.

Experimental Evaluation

Datasets

DeReC is evaluated on two benchmarks:

• RAWFC: Three-class classification (false, half-true, true) with claims and associated raw reports.
• LIAR-RAW: Six-class classification (pants-fire, false, barely-true, half-true, mostly-true, true) with a larger and more diverse set of claims and evidence.

Baselines

Comparisons include traditional neural models (dEFEND, SentHAN, SBERT-FC, CofCED, GenFE), as well as LLM-based systems (FactLLaMA, L-Defense with ChatGPT and LLaMA2 backends).

Results

DeReC achieves the following:

• RAWFC: F1 score of 65.58% (DeReC-qwen), outperforming L-Defense (61.20%) and all other baselines.
• LIAR-RAW: F1 score of 33.13% (DeReC-qwen), surpassing L-Defense and traditional baselines.

The DeReC-nomic variant (137M parameters) achieves comparable results on RAWFC (F1 64.61%) and competitive results on LIAR-RAW, demonstrating the effectiveness of smaller embedding models.

Efficiency

DeReC demonstrates dramatic runtime reductions:

• On RAWFC, DeReC-nomic completes the pipeline in 23m36s, a 95% reduction compared to L-Defense (454m12s).
• On LIAR-RAW, DeReC-nomic completes in 134m14s, a 92% reduction compared to L-Defense (1692m23s).

The primary efficiency gain is from eliminating LLM-based explanation generation, which dominates the runtime in L-Defense.

Resource Requirements

• DeReC's embedding models require 0.5GB (nomic) to 6GB (Qwen2) in FP32, compared to 7B+ parameter LLMs.
• The FAISS index scales linearly with the number of evidence sentences, which may pose memory challenges for extremely large corpora.

Implications and Discussion

Practical Implications

• Deployment Feasibility: DeReC's efficiency enables real-time or near-real-time fact verification on commodity hardware, making it suitable for large-scale or resource-constrained environments.
• Scalability: The modular design allows for parallelization and straightforward scaling to larger evidence corpora.
• Interpretability: While DeReC does not generate natural language explanations, its predictions are directly grounded in retrieved evidence, enhancing transparency.

Theoretical Implications

• Rationale Generation: The results challenge the necessity of LLM-based rationale generation for high-accuracy fact verification. Dense retrieval and targeted classification can match or exceed LLM performance.
• Model Size: Effective fact verification does not require massive LLMs; smaller, specialized models can suffice when combined with efficient retrieval.
• Future Directions: The architecture is amenable to improvements in embedding models, dynamic evidence updates, multilingual support, and lightweight explanation generation.

Limitations

• Evidence Corpus Quality: Performance is contingent on the completeness and quality of the evidence corpus.
• Memory for Retrieval: FAISS index size may become a bottleneck for very large corpora.
• Lack of Explanations: Absence of generated rationales may limit utility in settings requiring human-interpretable justifications.

Conclusion

DeReC demonstrates that dense retrieval and classification can outperform LLM-based generation for fact verification, both in accuracy and efficiency. The system achieves state-of-the-art results on standard benchmarks with a 95% reduction in runtime, using models with orders of magnitude fewer parameters. These findings suggest that, for specialized tasks like fact verification, targeted retrieval-classification pipelines are preferable to computationally intensive generative LLMs. The work opens avenues for further research into efficient, scalable, and interpretable fact-checking systems, with immediate applicability to real-world misinformation mitigation.