Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generative Debunking of Climate Misinformation (2407.05599v1)

Published 8 Jul 2024 in cs.CL and cs.CY

Abstract: Misinformation about climate change causes numerous negative impacts, necessitating corrective responses. Psychological research has offered various strategies for reducing the influence of climate misinformation, such as the fact-myth-fallacy-fact-structure. However, practically implementing corrective interventions at scale represents a challenge. Automatic detection and correction of misinformation offers a solution to the misinformation problem. This study documents the development of LLMs that accept as input a climate myth and produce a debunking that adheres to the fact-myth-fallacy-fact (``truth sandwich'') structure, by incorporating contrarian claim classification and fallacy detection into an LLM prompting framework. We combine open (Mixtral, Palm2) and proprietary (GPT-4) LLMs with prompting strategies of varying complexity. Experiments reveal promising performance of GPT-4 and Mixtral if combined with structured prompts. We identify specific challenges of debunking generation and human evaluation, and map out avenues for future work. We release a dataset of high-quality truth-sandwich debunkings, source code and a demo of the debunking system.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Multitask instruction-based prompting for fallacy recognition. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8172–8187, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  2. Palm 2 technical report. arXiv preprint arXiv:2305.10403.
  3. Computer-assisted classification of contrarian claims about climate change. Scientific Reports, 11(1):22320.
  4. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46.
  5. John Cook. 2020. Deconstructing climate science denial. Research handbook on communicating climate change, pages 62–78.
  6. John Cook. 2024. The 4d framework. The Companion to Development Studies, page 10.
  7. Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their influence. PloS one, 12(5):e0175799.
  8. Climate-fever: A dataset for verification of real-world climate claims. Preprint, arXiv:2012.00614.
  9. Reminders and repetition of misinformation: Helping or hindering its retraction? Journal of applied research in memory and cognition, 6(2):185–192.
  10. Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & cognition, 38:1087–1100.
  11. Knowledge does not protect against illusory truth. Journal of experimental psychology: general, 144(5):993.
  12. Lisa K Fazio and Carrie L Sherry. 2020. The effect of repetition on truth judgments across development. Psychological Science, 31(9):1150–1160.
  13. A survey on automated fact-checking. Transactions of the Association for Computational Linguistics, 10:178–206.
  14. Kilem Gwet. 2001. Handbook of inter-rater reliability: How to estimate the level of agreement between two or multiple raters. Gaithersburg, MD: STATAXIS Publishing Company.
  15. The quest to automate fact-checking. In Proceedings of the 2015 computation+ journalism symposium. Citeseer.
  16. Is explanation the cure? misinformation mitigation in the short term and long term. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1313–1323, Singapore. Association for Computational Linguistics.
  17. John C Jahnke. 1965. Primacy and recency effects in serial-position curves of immediate recall. Journal of experimental psychology, 70(1):130.
  18. Mixtral of experts. arXiv preprint arXiv:2401.04088.
  19. Logical fallacy detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 7180–7198, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  20. Laura M König. 2023. Debunking nutrition myths: An experimental test of the ‘truth sandwich’text format. British Journal of Health Psychology, 28(4):1000–1010.
  21. Neema Kotonya and Francesca Toni. 2020. Explainable automated fact-checking: A survey. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5430–5443, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  22. The debunking handbook 2020.
  23. Letting the gorilla emerge from the mist: Getting past post-truth. Journal of Applied Research in Memory and Cognition, 6(4):418–424.
  24. Brendan Nyhan and Jason Reifler. 2010. When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2):303–330.
  25. OpenAI. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  26. S Ichtiaque Rasool and Stephen H Schneider. 1971. Atmospheric carbon dioxide and aerosols: Effects of large increases on global climate. Science, 173(3992):138–141.
  27. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  28. Augmented cards: A machine learning approach to identifying triggers of climate change misinformation on twitter. Preprint, arXiv:2404.15673.
  29. In-context impersonation reveals large language models’ strengths and biases. Preprint, arXiv:2305.14930.
  30. Philipp Schmid and Cornelia Betsch. 2019. Effective strategies for rebutting science denialism in public discussions. Nature Human Behaviour, 3(9):931–939.
  31. Colleen M Seifert. 2002. The continued influence of misinformation in memory: What makes a correction effective? In Psychology of learning and motivation, volume 41, pages 265–292. Elsevier.
  32. Margaret Sullivan. 2018. Instead of trump’s propaganda, how about a nice ‘truth sandwich’. The Washington Post, 17.
  33. Monika Taddicken and Laura Wolff. 2023. Climate change-related counter-attitudinal fake news exposure and its effects on search and selection behavior. Environmental Communication, 17(7):720–739.
  34. Inoculating the public against misinformation about climate change. Global challenges, 1(2):1600008.
  35. The spread of true and false news online. science, 359(6380):1146–1151.
  36. Testing the effectiveness of correction placement and type on instagram. The International Journal of Press/Politics, 25(4):632–652.
  37. Thomas Wood and Ethan Porter. 2019. The elusive backfire effect: Mass attitudes’ steadfast factual adherence. Political Behavior, 41:135–163.
  38. React: Synergizing reasoning and acting in language models. In NeurIPS 2022 Foundation Models for Decision Making Workshop.
  39. Detecting fallacies in climate misinformation: A technocognitive approach to identifying misleading argumentation. Preprint, arXiv:2405.08254.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets