Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation (2408.00024v1)

Published 31 Jul 2024 in cs.AI and cs.CY

Abstract: Advanced AI systems, specifically LLMs, have the capability to generate not just misinformation, but also deceptive explanations that can justify and propagate false information and erode trust in the truth. We examined the impact of deceptive AI generated explanations on individuals' beliefs in a pre-registered online experiment with 23,840 observations from 1,192 participants. We found that in addition to being more persuasive than accurate and honest explanations, AI-generated deceptive explanations can significantly amplify belief in false news headlines and undermine true ones as compared to AI systems that simply classify the headline incorrectly as being true/false. Moreover, our results show that personal factors such as cognitive reflection and trust in AI do not necessarily protect individuals from these effects caused by deceptive AI generated explanations. Instead, our results show that the logical validity of AI generated deceptive explanations, that is whether the explanation has a causal effect on the truthfulness of the AI's classification, plays a critical role in countering their persuasiveness - with logically invalid explanations being deemed less credible. This underscores the importance of teaching logical reasoning and critical thinking skills to identify logically invalid arguments, fostering greater resilience against advanced AI-driven misinformation.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Katarina Kertysova. Artificial intelligence and disinformation: How ai changes the way disinformation is produced, disseminated, and can be countered. Security and Human Rights, 29(1-4):55–81, 2018.
  2. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246, 2023.
  3. How persuasive is ai-generated propaganda? PNAS nexus, 3(2):pgae034, 2024.
  4. Deepfake detection by human crowds, machines, and machine-informed crowds. Proceedings of the National Academy of Sciences, 119(1):e2110013119, 2022.
  5. Human detection of political speech deepfakes across transcripts, audio, and video. arXiv preprint arXiv:2202.12883, 2023.
  6. Digital literacy is associated with more discerning accuracy judgments but not sharing intentions. Harvard Kennedy School, Misinformation Review, 2021.
  7. Ai-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences, 119(8):e2120481119, 2022.
  8. " how do i fool you?" manipulating user trust via misleading black box explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 79–85, 2020.
  9. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  10. Art and the science of generative ai. Science, 380(6650):1110–1111, 2023.
  11. Interacting with opinionated language models changes users’ views. Arxiv Open Access, 2022.
  12. Artificial intelligence can persuade humans on political issues. OSF Preprints, 2023.
  13. How ai can distort human beliefs. Science, 380(6651):1222–1223, 2023.
  14. Working with ai to persuade: Examining a large language model’s ability to generate pro-vaccination messages. Stanford Preprint, 2023.
  15. Quantifying the potential persuasive returns to political microtargeting. Proceedings of the National Academy of Sciences, 120(25):e2216261120, 2023.
  16. Explainable ai: A brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II 8, pages 563–574. Springer, 2019.
  17. Xai—explainable artificial intelligence. Science robotics, 4(37):eaay7120, 2019.
  18. Towards explainability for ai fairness. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers, pages 375–386. Springer, 2020.
  19. Fairness in decision-making—the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  20. Where is your evidence: improving fact-checking by justification modeling. In Proceedings of the first workshop on fact extraction and verification (FEVER), pages 85–90, 2018.
  21. Don’t just tell me, ask me: Ai systems that intelligently frame explanations as questions improve human logical discernment accuracy over causal ai explanations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2023.
  22. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency, pages 29–38, 2019.
  23. Explanations can reduce overreliance on ai systems during decision-making. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1–38, 2023.
  24. The mindlessness of ostensibly thoughtful action: The role of" placebic" information in interpersonal interaction. Journal of personality and social psychology, 36(6):635, 1978.
  25. Valerie S Folkes. Mindlessness or mindfulness: A partial replication and extension of langer, blank, and chanowitz. Journal of Personality and Social Psychology, 1985.
  26. The impact of placebic explanations on trust in intelligent systems. In Extended abstracts of the 2019 CHI conference on human factors in computing systems, pages 1–6, 2019.
  27. Do people engage cognitively with ai? impact of ai assistance on incidental learning. In 27th international conference on intelligent user interfaces, pages 794–806, 2022.
  28. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860, 2023.
  29. Lucas Graves. Anatomy of a fact check: Objective practice and the contested epistemology of fact checking. Communication, culture & critique, 10(3):518–537, 2017.
  30. FactCheck.org. Factcheck - our process, Dec 2016.
  31. Alec Fisher. The logic of real arguments. Cambridge University Press, 2004.
  32. 110 best true or false quiz questions for an easier take on game night. https://www.cosmopolitan.com/uk/worklife/a32612392/best-true-false-quiz-questions/, May 2020. Accessed: 2022-01-01.
  33. Shane Frederick. Cognitive reflection and decision making. Journal of Economic perspectives, 19(4):25–42, 2005.
  34. Assessing miserly information processing: An expansion of the cognitive reflection test. Thinking & Reasoning, 20(2):147–168, 2014.
  35. Do explanations increase the effectiveness of ai-crowd generated fake news warnings? In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 183–193, 2022.
  36. An integrative model of organizational trust. Academy of management review, 20(3):709–734, 1995.
  37. The spread of true and false news online. science, 359(6380):1146–1151, 2018.
  38. Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188:39–50, 2019.
  39. Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151:90–103, 2019.
  40. Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–20, 2023.
  41. Who needs explanation and when? juggling explainable ai and user epistemic uncertainty. International Journal of Human-Computer Studies, 165:102839, 2022.
  42. “fake news” or real science? critical thinking to assess information on covid-19. In Frontiers in Education, volume 6, page 646909. Frontiers Media SA, 2021.
  43. The evolution of overconfidence. Nature, 477(7364):317–320, 2011.
  44. The domain specificity and generality of overconfidence: Individual differences in performance estimation bias. Psychonomic Bulletin & Review, 4(3):387–392, 1997.
  45. Deconstructing climate misinformation to identify reasoning errors. Environmental Research Letters, 13(2):024018, 2018.
  46. Wearable reasoner: towards enhanced human rationality through a wearable device with an explainable ai assistant. In Proceedings of the Augmented Humans International Conference, pages 1–12, 2020.
  47. Valdemar M Danry. AI Enhanced Reasoning: Augmenting Human Critical Thinking with AI Systems. PhD thesis, Massachusetts Institute of Technology, 2023.
  48. Influencing human–ai interaction by priming beliefs about ai can increase perceived trustworthiness, empathy and effectiveness. Nature Machine Intelligence, 5(10):1076–1086, 2023.
  49. What label should be applied to content produced by generative ai? 2023.
  50. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486–1496, 2023.
  51. Framing the origins of covid-19. Science communication, 42(5):562–585, 2020.
  52. Toward a model of text comprehension and production. Psychological review, 85(5):363, 1978.
  53. William H DuBay. The principles of readability. Online Submission, 2004.
  54. Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Valdemar Danry (6 papers)
  2. Pat Pataranutaporn (17 papers)
  3. Matthew Groh (20 papers)
  4. Ziv Epstein (16 papers)
  5. Pattie Maes (46 papers)

HackerNews

  1. Deceptive AI is most convincing (2 points, 0 comments)