Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Automated Fact-Checking of Climate Change Claims with Large Language Models (2401.12566v1)

Published 23 Jan 2024 in cs.CL

Abstract: This paper presents Climinator, a novel AI-based tool designed to automate the fact-checking of climate change claims. Utilizing an array of LLMs informed by authoritative sources like the IPCC reports and peer-reviewed scientific literature, Climinator employs an innovative Mediator-Advocate framework. This design allows Climinator to effectively synthesize varying scientific perspectives, leading to robust, evidence-based evaluations. Our model demonstrates remarkable accuracy when testing claims collected from Climate Feedback and Skeptical Science. Notably, when integrating an advocate with a climate science denial perspective in our framework, Climinator's iterative debate process reliably converges towards scientific consensus, underscoring its adeptness at reconciling diverse viewpoints into science-based, factual conclusions. While our research is subject to certain limitations and necessitates careful interpretation, our approach holds significant potential. We hope to stimulate further research and encourage exploring its applicability in other contexts, including political fact-checking and legal domains.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Generating Fact Checking Explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7352–7364. Online: Association for Computational Linguistics.
  2. LLM Augmented LLMs: Expanding Capabilities through Composition. ArXiv:2401.02412 [cs].
  3. Improving Language Models by Retrieving from Trillions of Tokens. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvari, C.; Niu, G.; and Sabato, S., eds., Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, 2206–2240. PMLR.
  4. Assessing Large Language Models on Climate Information. arXiv preprint arXiv:2310.02932.
  5. Complex Claim Verification with Evidence Retrieved in the Wild. ArXiv:2305.11859 [cs].
  6. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models. ArXiv:2401.01335 [cs, stat].
  7. Computational Journalism: A Call to Arms to Database Researchers. In CIDR.
  8. Climate-fever: A dataset for verification of real-world climate claims. arXiv preprint arXiv:2012.00614.
  9. Improving Factuality and Reasoning in Language Models through Multiagent Debate. arXiv preprint arXiv:2305.14325.
  10. Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction. ArXiv:2207.14116 [cs].
  11. Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv:2312.10997 [cs].
  12. Graves, L. 2018. Understanding the Promise and Limits of Automated Fact-Checking. Technical report, Reuters Institute, University of Oxford.
  13. A Survey on Automated Fact-Checking. Transactions of the Association for Computational Linguistics, 10: 178–206.
  14. ClaimBuster: The First-Ever End-to-End Fact-Checking System. Proc. VLDB Endow., 10(12): 1945–1948.
  15. Climate Change Reconsidered II: Physical Science (Chicago, IL: The Heartland Institute, 2013). Heartland Institute.
  16. Climate Change Reconsidered II: Fossil Fuels (Chicago, IL: The Heartland Institute, 2013). Heartland Institute.
  17. HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification. ArXiv:2011.03088 [cs].
  18. DeliData: A dataset for deliberation in multi-party problem solving. arXiv:2108.05271.
  19. Generalization through Memorization: Nearest Neighbor Language Models. In International Conference on Learning Representations (ICLR).
  20. Explainable Automated Fact-Checking: A Survey. In Proceedings of the 28th International Conference on Computational Linguistics, 5430–5443. Barcelona, Spain (Online): International Committee on Computational Linguistics.
  21. Why Do Humans Reason? Arguments for an Argumentative Theory. Behavioral and Brain Sciences, 34(2): 57–74.
  22. Minsky, M. 1986. The society of mind.
  23. Consensus revisited: quantifying scientific agreement on climate change and climate expertise among Earth scientists 10 years later. Environmental Research Letters, 16.
  24. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774.
  25. AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web. arXiv:2305.13117.
  26. Hysteresis of tropical forests in the 21st century. Nature communications, 11(1): 4978.
  27. e-FEVER: Explanations and Summaries for Automated Fact Checking. In De Cristofaro, E.; and Nakov, P., eds., Proceedings of the 2020 Truth and Trust Online (TTO 2020), 32 – 43. Arlington, VA: Hacks Hackers. ISBN 978-1-7359904-0-8. Conference for Truth and Trust Online (TTO 2020) (virtual); Conference Location: online; Conference Date: October 16-17, 2020; Due to the Coronavirus (COVID-19) the conference was conducted virtually.
  28. FEVER: a large-scale dataset for fact extraction and VERification. arXiv preprint arXiv:1803.05355.
  29. Twitter. 2021. Community Notes on Twitter.
  30. ChatClimate: Grounding conversational AI in climate science. Communications Earth & Environment, 4(1): 480.
  31. Vlachos, A. 2021. Fact-checking as a conversation, by Andreas Vlachos.
  32. Fact Checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, 18–22. Association for Computational Linguistics.
  33. Fact Checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, 18–22. Baltimore, MD, USA: Association for Computational Linguistics.
  34. The spread of true and false news online. Science, 359: 1146–1151.
  35. SciFact-Open: Towards open-domain scientific claim verification. ArXiv:2210.13777 [cs].
  36. Mindstorms in Natural Language-Based Societies of Mind. arXiv preprint arXiv:2305.17066.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (13)
  1. Markus Leippold (24 papers)
  2. Saeid Ashraf Vaghefi (5 papers)
  3. Dominik Stammbach (16 papers)
  4. Veruska Muccione (3 papers)
  5. Julia Bingler (5 papers)
  6. Jingwei Ni (15 papers)
  7. Chiara Colesanti-Senni (4 papers)
  8. Tobias Wekhof (3 papers)
  9. Tobias Schimanski (10 papers)
  10. Glen Gostlow (4 papers)
  11. Tingyu Yu (4 papers)
  12. Juerg Luterbacher (1 paper)
  13. Christian Huggel (1 paper)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now