Multilingual Fact-Checking at Scale: Fine-Tuned Compact Models vs LLMs

Abstract: We present a multilingual fact-checking system deployed at Factiverse, designed for high-throughput and low-latency operation across diverse languages. The system follows a modular pipeline with three stages: claim detection, evidence retrieval and re-ranking, and veracity prediction. We fine-tune XLM-RoBERTa-Large for claim detection, mmBERT-base for three-label stance classification (Supports/Refutes/Mixed), and a SetFit-based multilingual re-ranker for claim--evidence matching. We compare these components against strong LLM baselines, including GPT-5.2, Claude Opus~4.6, and Qwen3-8b. Experiments on production data spanning 114 languages for claim detection and 28 languages for veracity prediction show that task-specific fine-tuning provides strong and stable multilingual performance, while the fine-tuned retrieval model remains competitive with modern proprietary embeddings. Same-hardware latency measurements further show large efficiency gains for encoder-based components, supporting their use in production deployments with tight cost and privacy constraints. Overall, compact fine-tuned, self-hosted models remain a practical and effective foundation for multilingual fact-checking at scale. Code and data used for this study are available at https://github.com/factiverse/factcheck-editor.

• The paper demonstrates that fine-tuned transformers deliver competitive claim detection and veracity prediction compared to LLMs, especially in low-resource languages.
• The proposed modular pipeline employs claim detection, evidence retrieval, and veracity prediction, achieving impressive latency and cost efficiencies for production-scale fact-checking.
• Empirical results show that compact models maintain stability, robustness, and privacy with significantly lower inference costs compared to LLM-based approaches.

Comprehensive Evaluation of Multilingual Fact-Checking: Fine-Tuned Compact Models versus LLMs

Introduction

The proliferation of online misinformation has outpaced the capacity of manual fact-checking processes, necessitating robust, automated, and multilingual fact-checking pipelines. This paper presents a scalable, production-level fact-checking system implemented at Factiverse, dissecting the end-to-end pipeline into three core tasks—claim detection, evidence retrieval (with re-ranking), and veracity prediction—comparing fine-tuned compact Transformer-based models with LLMs, including GPT-5.2, Claude Opus~4.6, and Qwen3-8b. The study offers a detailed empirical assessment over 114 languages for claim detection and 28 languages for veracity prediction, focusing on accuracy, efficiency, and privacy-preserving deployment.

System Architecture and Methodology

The Factiverse architecture consists of a modular, three-stage pipeline that leverages dedicated, fine-tuned encoder models for all core decision-making tasks:

1. Claim Detection: Utilizes XLM-RoBERTa-Large, fine-tuned to identify check-worthy factual statements using supervised datasets (ClaimBuster, CheckThat!, and proprietary Factiverse data). The model employs a class-weighted loss to counter the inherent label imbalance toward non-check-worthy sentences and leverages XLM-R's pre-trained multilingual representations for robust transfer to low-resource languages.
2. Evidence Retrieval and Re-Ranking: Each claim is transformed into targeted queries via an LLM (Gemma4-E4b-it), retrieving candidate evidence snippets from diverse sources (search engines, Wikipedia, Semantic Scholar, Factiverse archive). A SetFit classifier—fine-tuned on human-labeled claim-evidence pairs—is used for bi-encoder-based passage ranking, applying contrastive learning augmented with multilingual back-translation and domain-specific data.
3. Veracity Prediction: mmBERT-base, fine-tuned on multilingual stance datasets, performs three-way classification (Supports, Refutes, Mixed) of claim-evidence pairs. To prevent models from overfitting to explicit verdict phrases, evidence is purged of such cues prior to training. This component also employs class rebalancing to avoid majority-class collapse. Generative LLMs are reserved for auxiliary tasks (justification summary, correction generation).

Latency, throughput, and cost are measured on standardized hardware (NVIDIA H100 NVL 95GB GPUs), ensuring fair efficiency comparisons. Evaluation datasets are translation-augmented to maximize multilingual coverage, with language-level and error-mode analyses provided.

Experimental Results

Claim Detection Performance

Fine-tuned XLM-RoBERTa-Large substantially outperforms LLM baselines for claim detection, particularly in low-resource and morphologically complex languages. Although Claude Opus~4.6 shows a marginally higher Macro-F1 (0.7489 vs. 0.7277), the fine-tuned model achieves higher Micro-F1 (0.7931) and is dominant in English (Macro-F1 0.9045).

(Figure 1)

Figure 1: Evaluation of claim detection for 114 languages, showing consistent Factiverse (fine-tuned XLM-RoBERTa-Large) performance across both high- and low-resource languages, outperforming LLM baselines in stability and coverage.

The aggregate results highlight that additional supervised data for lower-resource languages would further increase the advantage of fine-tuning. LLMs exhibit quality drops and instability in long-tail languages, while the encoder's performance remains comparatively stable—crucial for global deployments.

Evidence Retrieval and Re-Ranking

Factiverse’s domain-adapted, fine-tuned XLM-RoBERTa-Large embedding model achieves the highest F1 (0.8216), balancing recall (0.9429) and precision (0.7279), and surpasses both public and proprietary bi-encoder baselines on practical evidence matching. Compact models remain not only competitive but preferable for large-scale deployments where throughput and privacy constraints are critical, owing to their lower inference cost and deterministic outputs.

Veracity Prediction

In veracity prediction, GPT-5.2 attains the highest aggregate Macro-F1 (0.6457), but the fine-tuned mmBERT-base model (0.6205) is highly competitive and achieves the best class-balanced accuracy on English (EN Macro-F1 0.8036). Fine-tuned encoders degrade more gracefully than LLMs in lower-resource settings.

Figure 2: Evaluation of veracity prediction for 28 languages, comparing mmBERT-base (Factiverse) with GPT-5.2, Claude Opus~4.6, and Qwen3-8b, revealing sharp LLM performance drops in low-resource languages and relative encoder stability.

The language-level error analysis reveals significant LLM failure modes: inconsistent format adherence, instruction-following breakdowns in low-resource languages, overreliance on surface verdict cues, and class collapse on ambiguous (Mixed) classifications. Compact encoders, benefiting from tailored supervision and explicit preprocessing, avoid these pitfalls, resulting in more robust outputs.

Error Analysis and Systemic Implications

The study’s systematic error analysis documents four high-frequency LLM failure modes: refusal responses with instruction non-compliance, verdict phrase leakage, class collapse (especially for Mixed), and pronounced performance degradation outside major languages. These are mitigated by fine-tuned encoder pipelines through discrete label heads, verdict-phrase scrubbing, and aggressive data augmentation.

Practically, the findings advocate for industrial pipelines that deploy compact, well-tuned encoders for discrete-label tasks—reserving LLMs only for generative subcomponents (decontextualization, justification, correction) where their flexibility is indispensable.

Inference Efficiency and Cost

Latency and throughput benchmarking highlight dramatic efficiency deltas:

• Fine-tuned encoders process claims in ~1.3ms (claim detection) and ~3.7ms (stance), with per-1k decision costs <$0.01, outperforming LLMs by factors exceeding 100x.
• LLM-based per-item inference is orders of magnitude slower and costlier, particularly over API endpoints, and introduces privacy and data-governance concerns incompatible with newsroom or enterprise standards.

This efficiency renders compact models the perspicuous choice for production-scale, privacy-sensitive, and high-throughput settings.

Theoretical and Practical Implications

From a theoretical lens, the results support the continued relevance of modular fact-checking pipelines rooted in compact encoder-based architectures, even as generalist LLMs improve. While LLM generalization offers potential, stability, controllability, and deterministic label mapping remain critical for industrial fact verification. Empirical results confirm that targeted fine-tuning, adaptation for data imbalance, and multilingual augmentation collectively close the performance gap with substantially larger models, especially for well-defined, discrete-label tasks.

Practically, these findings reinforce best practices for industrial AI deployment—emphasizing end-to-end latency, deterministic outputs, privacy control, and cost efficiency. The research invites future exploration of hybrid architectures—triggering LLM escalation only for low-confidence or high-ambiguity cases—and multi-agent orchestration embodying specialized reasoning components within broader agentic workflows.

Conclusion

This paper demonstrates that fine-tuned, compact, self-hosted transformer models remain robust, efficient, and effective foundations for scalable, multilingual fact-checking, rivaling and often surpassing monolithic LLMs for discrete-label tasks under production constraints. The case for modular systems built on targeted, domain-adapted encoders is reinforced by both quantitative accuracy and qualitative error analysis. Future developments will likely focus on extending these pipelines through larger native-language datasets, dynamic hybrid escalation, and more nuanced agentic orchestration—ensuring industrial fact-checking remains globally reliable, scalable, and trustworthy across an ever-diversifying information landscape.