Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Reopening of Pandora's Box: Analyzing the Role of LLMs in the Evolving Battle Against AI-Generated Fake News (2410.19250v1)

Published 25 Oct 2024 in cs.CL

Abstract: With the rise of AI-generated content spewed at scale from LLMs, genuine concerns about the spread of fake news have intensified. The perceived ability of LLMs to produce convincing fake news at scale poses new challenges for both human and automated fake news detection systems. To address this gap, this work presents the findings from a university-level competition which aimed to explore how LLMs can be used by humans to create fake news, and to assess the ability of human annotators and AI models to detect it. A total of 110 participants used LLMs to create 252 unique fake news stories, and 84 annotators participated in the detection tasks. Our findings indicate that LLMs are ~68% more effective at detecting real news than humans. However, for fake news detection, the performance of LLMs and humans remains comparable (~60% accuracy). Additionally, we examine the impact of visual elements (e.g., pictures) in news on the accuracy of detecting fake news stories. Finally, we also examine various strategies used by fake news creators to enhance the credibility of their AI-generated content. This work highlights the increasing complexity of detecting AI-generated fake news, particularly in collaborative human-AI settings.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Demystifying neural fake news via linguistic feature-based interpretation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6586–6599.
  2. Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2):211–236.
  3. Evaluating the fake news problem at the scale of the information ecosystem. Science advances, 6(14):eaay3539.
  4. Media literacy tips promoting reliable news improve discernment and enhance trust in traditional media. Communications Psychology, 2(1):74.
  5. Canyu Chen and Kai Shu. 2023a. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788.
  6. Canyu Chen and Kai Shu. 2023b. Combating misinformation in the age of llms: Opportunities and challenges. AI Magazine.
  7. Exploring online news credibility: The relative influence of traditional and technological factors. Journal of computer-mediated communication, 17(2):171–186.
  8. Automatic deception detection: Methods for finding fake news. Proceedings of the association for information science and technology, 52(1):1–4.
  9. Llm detectors still fall short of real world: Case of llm-generated short news-like posts. arXiv e-prints, pages arXiv–2409.
  10. Bag of what? simple noun phrase extraction for text analysis. In Proceedings of the first workshop on NLP and computational social science, pages 114–124.
  11. Disinformation detection: An evolving challenge in the age of llms. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pages 427–435. SIAM.
  12. Catching chameleons: Detecting evolving disinformation generated using large language models. arXiv preprint arXiv:2406.17992.
  13. Hamid Keshavarz. 2014. How credible is information on the web: Reflections on misinformation and disinformation. Infopreneurship Journal, 1(2):1–17.
  14. Covid-19 vaccine misinformation in middle income countries. Association for Computational Linguistics.
  15. All the news that’s fit to fabricate: Ai-generated text as a tool of media misinformation. Journal of experimental political science, 9(1):104–117.
  16. Fighting fire with fire: The dual role of llms in crafting and detecting elusive disinformation. arXiv preprint arXiv:2310.15515.
  17. Linguistic-based detection of fake news in social media. Forthcoming, International Journal of English Linguistics, 11(1).
  18. Sohad Murrar and Markus Brauer. 2018. Mixed model analysis of variance. The SAGE encyclopedia of educational research, measurement, and evaluation, 1:1075–1078.
  19. On the risk of misinformation pollution with large language models. arXiv preprint arXiv:2305.13661.
  20. Sergey Sanovich. 2017. Computational propaganda in russia: The origins of digital misinformation.
  21. Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter, 19(1):22–36.
  22. Adapting fake news detection to the era of large language models. arXiv preprint arXiv:2311.04917.
  23. Med-mmhl: A multi-modal dataset for detecting human-and llm-generated misinformation in the medical domain. arXiv preprint arXiv:2306.08871.
  24. Monolingual and multilingual misinformation detection for low-resource languages: A comprehensive survey. arXiv preprint arXiv:2410.18390.
  25. Implementing bert and fine-tuned roberta to detect ai generated news by chatgpt. arXiv preprint arXiv:2306.07401.
  26. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3367–3378.
  27. Jiaying Wu and Bryan Hooi. 2023. Decor: Degree-corrected social graph refinement for fake news detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2582–2593.
  28. Combating misinformation in the era of generative ai models. In Proceedings of the 31st ACM International Conference on Multimedia, pages 9291–9298.
  29. A structured response to misinformation: Defining and annotating credibility indicators in news articles. In Companion Proceedings of the The Web Conference 2018, pages 603–612.
  30. More human than human: Llm-generated narratives outperform human-llm interleaved narratives. In Proceedings of the 15th Conference on Creativity and Cognition, pages 368–370.
  31. Xinyi Zhou and Reza Zafarani. 2020. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53(5):1–40.
  32. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Xinyu Wang (186 papers)
  2. Wenbo Zhang (49 papers)
  3. Sai Koneru (15 papers)
  4. Hangzhi Guo (6 papers)
  5. Bonam Mingole (3 papers)
  6. S. Shyam Sundar (5 papers)
  7. Sarah Rajtmajer (31 papers)
  8. Amulya Yadav (17 papers)