Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generative Large Language Models in Automated Fact-Checking: A Survey (2407.02351v2)

Published 2 Jul 2024 in cs.CL

Abstract: The dissemination of false information on online platforms presents a serious societal challenge. While manual fact-checking remains crucial, LLMs offer promising opportunities to support fact-checkers with their vast knowledge and advanced reasoning capabilities. This survey explores the application of generative LLMs in fact-checking, highlighting various approaches and techniques for prompting or fine-tuning these models. By providing an overview of existing methods and their limitations, the survey aims to enhance the understanding of how LLMs can be used in fact-checking and to facilitate further progress in their integration into the fact-checking process.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (128)
  1. PoliMi-FlatEarthers at CheckThat! 2022: GPT-3 applied to claim detection. Working Notes of CLEF.
  2. Fake news, disinformation and misinformation in social media: a review. Social Network Analysis and Mining, 13(1):30.
  3. Overview of the clef-2023 checkthat! lab task 1 on check-worthiness in multimodal and multigenre content. Working Notes of CLEF.
  4. Generative ai for explainable automated fact checking on the factex: A new benchmark dataset. In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 1–13. Springer.
  5. Ta’keed: The first generative fact-checking system for arabic claims. arXiv preprint arXiv:2401.14067.
  6. The fact extraction and VERification over unstructured and structured information (FEVEROUS) shared task. In Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER), pages 1–13, Dominican Republic. Association for Computational Linguistics.
  7. A benchmark dataset of check-worthy factual claims. arXiv preprint arXiv:2004.14425.
  8. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
  9. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023.
  10. Checkthat! at clef 2020: Enabling the automatic identification and verification of claims in social media. In Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Proceedings, Part II 42, pages 499–507. Springer.
  11. Automatic claim review for climate science via explanation generation. arXiv preprint arXiv:2107.14740.
  12. Mars Gokturk Buchholz. 2023. Assessing the Effectiveness of GPT-3 in Detecting False Political Statements: A Case Study on the LIAR Dataset. arXiv preprint arXiv:2306.08190.
  13. Are large language models good fact checkers: A preliminary study. arXiv preprint arXiv:2311.17355.
  14. Recep Firat Cekinel and Pinar Karagoz. 2024. Explaining veracity predictions with evidence summarization: A multi-task model approach. arXiv preprint arXiv:2402.06443.
  15. An empirical study of using chatgpt for fact verification task. arXiv preprint arXiv:2311.06592.
  16. Canyu Chen and Kai Shu. 2023a. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788.
  17. Canyu Chen and Kai Shu. 2023b. Combating misinformation in the age of llms: Opportunities and challenges. arXiv preprint arXiv:2311.05656.
  18. Generating literal and implied subquestions to fact-check complex claims. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3495–3516, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  19. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint arXiv:1909.02164.
  20. Eun Cheol Choi and Emilio Ferrara. 2023. Automated claim matching with large language models: Empowering fact-checkers in the fight against misinformation. arXiv preprint arXiv:2310.09223.
  21. Eun Cheol Choi and Emilio Ferrara. 2024. Fact-gpt: Fact-checking augmentation via claim matching with llms. arXiv preprint arXiv:2402.05904.
  22. Overview of the trec 2020 health misinformation track. In TREC.
  23. Limeng Cui and Dongwon Lee. 2020. Coaid: Covid-19 healthcare misinformation dataset. arXiv preprint arXiv:2006.00885.
  24. Climate-fever: A dataset for verification of real-world climate claims. arXiv preprint arXiv:2012.00614.
  25. Nus-ids at checkthat! 2022: identifying check-worthiness of tweets using checkthat5. Working Notes of CLEF.
  26. Fool me twice: Entailment from Wikipedia gamification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 352–365, Online. Association for Computational Linguistics.
  27. Msvec: A multidomain testing dataset for scientific claim verification. In Proceedings of the Twenty-Fourth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, MobiHoc ’23, page 504–509, New York, NY, USA. Association for Computing Machinery.
  28. A multistage retrieval system for health-related misinformation detection. Engineering Applications of Artificial Intelligence, 115:105211.
  29. NewsClaims: A new benchmark for claim detection from news with attribute knowledge. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6002–6018, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  30. Ambifc: Fact-checking ambiguous claims with evidence. arXiv preprint arXiv:2104.00640.
  31. Language models hallucinate, but may excel at fact verification. arXiv preprint arXiv:2310.14564.
  32. A Survey on Automated Fact-Checking. Transactions of the Association for Computational Linguistics, 10:178–206.
  33. Ashim Gupta and Vivek Srikumar. 2021. X-fact: A new benchmark dataset for multilingual fact checking. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 675–682, Online. Association for Computational Linguistics.
  34. LESA: Linguistic encapsulation and semantic amalgamation based generalised claim detection from online content. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3178–3188, Online. Association for Computational Linguistics.
  35. Exclaim: Explainable neural claim verification using rationalization. In 2022 IEEE 29th Annual Software Technology Conference (STC). IEEE.
  36. ArCOV19-rumors: Arabic COVID-19 Twitter dataset for misinformation detection. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 72–81, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  37. Leveraging chat-gpt for efficient fact-checking.(2023). Available on: https://doi. org/10.31234/osf. io/qnjkf and.
  38. Using chatgpt to fight misinformation: Chatgpt nails 72% of 12,000 verified claims.
  39. Do large language models know about facts? arXiv preprint arXiv:2310.05177.
  40. CHEF: A pilot Chinese dataset for evidence-based fact-checking. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3362–3376, Seattle, United States. Association for Computational Linguistics.
  41. Yue Huang and Lichao Sun. 2023. Harnessing the power of chatgpt in fake news: An in-depth exploration in generation, detection and explanation. arXiv preprint arXiv:2310.05046.
  42. Is it indeed bigger better? the comprehensive study of claim detection lms applied for disinformation tackling. arXiv preprint arXiv:2311.06121.
  43. Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research, 24(251):1–43.
  44. Disinformation Detection: An Evolving Challenge in the Age of LLMs. arXiv preprint arXiv:2309.15847.
  45. Exploring listwise evidence reasoning with t5 for fact verification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 402–410, Online. Association for Computational Linguistics.
  46. HoVer: A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3441–3460, Online. Association for Computational Linguistics.
  47. WiCE: Real-world entailment for claims in Wikipedia. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7561–7583, Singapore. Association for Computational Linguistics.
  48. Wice: Real-world entailment for claims in wikipedia. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online. Association for Computational Linguistics.
  49. Wei-Yu Kao and An-Zi Yen. 2024. How we refute claims: Automatic fact-checking through flaw identification and explanation. arXiv preprint arXiv:2401.15312.
  50. WatClaimCheck: A new dataset for claim entailment and inference. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1293–1304, Dublin, Ireland. Association for Computational Linguistics.
  51. Can llms produce faithful explanations for fact-checking? towards faithful explainable fact-checking via multi-agent debate. arXiv preprint arXiv:2402.07401.
  52. Neema Kotonya and Francesca Toni. 2020. Explainable automated fact-checking for public health claims. arXiv preprint arXiv:2010.09926.
  53. Misinformation has high perplexity. arXiv preprint arXiv:2006.04666.
  54. Automated fact-checking of climate change claims with large language models. arXiv preprint arXiv:2401.12566.
  55. Retrieval-augmented generation for knowledge-intensive nlp tasks. arXiv preprint arXiv:2005.11401.
  56. Self-checker: Plug-and-play modules for fact-checking with large language models. arXiv preprint arXiv:2305.14623.
  57. Yifan Li and ChengXiang Zhai. 2023. An exploration of large language models for verification of news headlines. In 2023 IEEE International Conference on Data Mining Workshops (ICDMW), pages 197–206.
  58. A revisit of fake news dataset with augmented fact-checking by chatgpt. arXiv preprint arXiv:2312.11870.
  59. Teller: A trustworthy framework for explainable, generalizable and controllable fake news detection. arXiv preprint arXiv:2402.07776.
  60. Can large language models detect rumors on social media? arXiv preprint arXiv:2402.03916.
  61. Real-time rumor debunking on twitter. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15, page 1867–1870, New York, NY, USA. Association for Computing Machinery.
  62. Ex-fever: A dataset for multi-hop explainable fact verification. arXiv preprint arXiv:2310.09754.
  63. Detecting rumors from microblogs with recurrent neural networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, page 3818–3824. AAAI Press.
  64. Christopher Malon. 2021. Team papelo at FEVEROUS: Multi-hop evidence pursuit. In Proceedings of the Fourth Workshop on Fact Extraction and VERification (FEVER), pages 40–49, Dominican Republic. Association for Computational Linguistics.
  65. Michael McCloskey and Neal J. Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. volume 24 of Psychology of Learning and Motivation, pages 109–165. Academic Press.
  66. Large language models: A survey. arXiv preprint arXiv:2402.06196.
  67. Global-liar: Factuality of llms over time and geographic regions. arXiv preprint arXiv:2401.17839.
  68. Overview of the clef-2022 checkthat! lab task 1 on identifying relevant claims in tweets. In 2022 Conference and Labs of the Evaluation Forum, CLEF 2022, pages 368–392. CEUR Workshop Proceedings (CEUR-WS. org).
  69. Automated fact-checking for assisting human fact-checkers. ArXiv, abs/2103.07769.
  70. Overview of the clef–2021 checkthat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. arXiv preprint arXiv:2109.12987.
  71. Mdfend: Multi-domain fake news detection. In Proceedings of the 30th ACM International Conference on Information; Knowledge Management, CIKM ’21. ACM.
  72. Deep learning-based claim matching with multiple negatives training. In Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), pages 134–139, Online. Association for Computational Linguistics.
  73. Multi-hop fact checking of political claims. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 3892–3898. International Joint Conferences on Artificial Intelligence Organization. Main Track.
  74. Qacheck: A demonstration system for question-guided multi-hop fact-checking. arXiv preprint arXiv:2310.07609.
  75. Fact-checking complex claims with program-guided reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6981–7004, Toronto, Canada. Association for Computational Linguistics.
  76. Fighting an Infodemic: COVID-19 Fake News Dataset, page 21–29. Springer International Publishing.
  77. Towards reliable misinformation mitigation: Generalization, uncertainty, and gpt-4. arXiv preprint arXiv:2305.14928.
  78. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523–2544, Online. Association for Computational Linguistics.
  79. Multilingual previously fact-checked claim retrieval. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16477–16500, Singapore. Association for Computational Linguistics.
  80. Multilingual previously fact-checked claim retrieval. arXiv preprint arXiv:2305.07991.
  81. Cross-lingual learning for text processing: A survey. Expert Systems with Applications, 165:113765.
  82. Scientific Claim Verification with VERT5ERINI. arXiv preprint arXiv:2010.11930.
  83. Vera: Prediction techniques for reducing harmful misinformation in consumer health search. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, page 2066–2070, New York, NY, USA. Association for Computing Machinery.
  84. Automated query generation for evidence collection from web search engines. arXiv preprint arXiv:2303.08652.
  85. Dorian Quelle and Alexandre Bovet. 2023. The perils & promises of fact-checking with large language models. arXiv preprint arXiv:2310.13549.
  86. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2931–2937, Copenhagen, Denmark. Association for Computational Linguistics.
  87. Overview of the GermEval 2021 shared task on the identification of toxic, engaging, and fact-claiming comments. In Proceedings of the GermEval 2021 Shared Task on the Identification of Toxic, Engaging, and Fact-Claiming Comments, pages 1–12, Duesseldorf, Germany. Association for Computational Linguistics.
  88. Searching for scientific evidence in a pandemic: An overview of trec-covid. arXiv preprint arXiv:2104.09632.
  89. Benchmarking the generation of fact checking explanations. Transactions of the Association for Computational Linguistics, 11:1250–1264.
  90. COVID-fact: Fact extraction and verification of real-world claims on COVID-19 pandemic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2116–2129, Online. Association for Computational Linguistics.
  91. Minds versus machines: Rethinking entailment verification with language models. arXiv preprint arXiv:2402.03686.
  92. Evidence-based fact-checking of health-related claims. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3499–3512, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  93. Openfact at checkthat! 2023: Head-to-head gpt vs. bert-a comparative study of transformers language models for the detection of check-worthy claims. In CEUR Workshop Proceedings, volume 3497.
  94. Averitec: A dataset for real-world claim verification with evidence from the web. arXiv preprint arXiv:2305.13117.
  95. Get your vitamin c! robust fact verification with contrastive evidence. arXiv preprint arXiv:2103.08541.
  96. Towards debiasing fact verification models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3419–3425, Hong Kong, China. Association for Computational Linguistics.
  97. Vinay Setty. 2024. Surprising efficacy of fine-tuned transformers for fact-checking over larger language models. arXiv preprint arXiv:2402.12147.
  98. That is a known lie: Detecting previously fact-checked claims. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3607–3618, Online. Association for Computational Linguistics.
  99. SD-H Michael Shliselberg and Shiri Dori-Hacohen. 2022. Riet lab at checkthat! 2022: improving decoder based re-ranking for claim matching. Working Notes of CLEF, pages 05–08.
  100. Fakenewsnet: A data repository with news content, social context and spatialtemporal information for studying fake news on social media. arXiv preprint arXiv:1809.01286.
  101. Utdrm: unsupervised method for training debunked-narrative retrieval models. EPJ Data Science, 12(1):59.
  102. Monant medical misinformation dataset: Mapping articles to fact-checked claims. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’22, page 2949–2959, New York, NY, USA. Association for Computing Machinery.
  103. Dominik Stammbach and Elliott Ash. 2020. e-fever: Explanations and summaries for automated fact checking. Proceedings of the 2020 Truth and Trust Online (TTO 2020), pages 32–43.
  104. Environmental claim detection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1051–1066, Toronto, Canada. Association for Computational Linguistics.
  105. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561.
  106. From chaos to clarity: Claim normalization to empower fact-checking. arXiv preprint arXiv:2310.14338.
  107. Evidence-based interpretable open-domain fact-checking with large language models. arXiv preprint arXiv:2312.05834.
  108. James Thorne and Andreas Vlachos. 2018. Automated fact checking: Task formulations, methods and future directions. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3346–3359, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  109. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, New Orleans, Louisiana. Association for Computational Linguistics.
  110. Hoai Nam Tran and Udo Kruschwitz. 2022. ur-iw-hnt at checkthat! 2022: cross-lingual text summarization for fake news detection. Working Notes of CLEF.
  111. Comparing gpt-4 and open-source language models in misinformation mitigation. arXiv preprint arXiv:2401.06920.
  112. Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pages 18–22, Baltimore, MD, USA. Association for Computational Linguistics.
  113. Disinformation capabilities of large language models. arXiv preprint arXiv:2311.08838.
  114. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association for Computational Linguistics.
  115. Check-COVID: Fact-checking COVID-19 news claims with scientific evidence. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14114–14127, Toronto, Canada. Association for Computational Linguistics.
  116. Haoran Wang and Kai Shu. 2023. Explainable claim verification via knowledge-grounded reasoning with large language models. arXiv preprint arXiv:2310.05253.
  117. William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426, Vancouver, Canada. Association for Computational Linguistics.
  118. Factcheck-bench: Fine-grained evaluation benchmark for automatic fact-checkers. arXiv preprint arXiv:2311.09000.
  119. Jiaying Wu and Bryan Hooi. 2023. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. arXiv preprint arXiv:2310.10830.
  120. A coarse-to-fine cascaded evidence-distillation neural network for explainable fake news detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2608–2621, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  121. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
  122. Fengzhu Zeng and Wei Gao. 2023. Prompt to be consistent is better than self-consistent? few-shot and zero-shot fact verification with pre-trained language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4555–4569, Toronto, Canada. Association for Computational Linguistics.
  123. Fengzhu Zeng and Wei Gao. 2024. Justilm: Few-shot justification generation for explainable fact-checking of real-world claims. arXiv preprint arXiv:2401.08026.
  124. Automated fact-checking: A survey. Language and Linguistics Compass, 15(10):e12438.
  125. Xia Zeng and Arkaitz Zubiaga. 2024. Maple: Micro analysis of pairwise language evolution for few-shot claim verification. arXiv preprint arXiv:2401.16282.
  126. Do we need language-specific fact-checking models? the case of chinese. arXiv preprint arXiv:2401.15498.
  127. Are large language models table-based fact-checkers? arXiv preprint arXiv:2402.02549.
  128. Xuan Zhang and Wei Gao. 2023. Towards llm-based fact verification on news claims with a hierarchical step-by-step prompting method. arXiv preprint arXiv:2310.00305.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Ivan Vykopal (8 papers)
  2. Matúš Pikuliak (12 papers)
  3. Simon Ostermann (26 papers)
  4. Marián Šimko (10 papers)
Citations (1)
View on Semantic Scholar