Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Blessing or curse? A survey on the Impact of Generative AI on Fake News (2404.03021v1)

Published 3 Apr 2024 in cs.CL and cs.AI

Abstract: Fake news significantly influence our society. They impact consumers, voters, and many other societal groups. While Fake News exist for a centuries, Generative AI brings fake news on a new level. It is now possible to automate the creation of masses of high-quality individually targeted Fake News. On the other end, Generative AI can also help detecting Fake News. Both fields are young but developing fast. This survey provides a comprehensive examination of the research and practical use of Generative AI for Fake News detection and creation in 2024. Following the Structured Literature Survey approach, the paper synthesizes current results in the following topic clusters 1) enabling technologies, 2) creation of Fake News, 3) case study social media as most relevant distribution channel, 4) detection of Fake News, and 5) deepfakes as upcoming technology. The article also identifies current challenges and open issues.

PDF HTML Abstract

Comprehensive Survey on the Impact of Generative AI on Fake News

Introduction

The proliferation of fake news in the digital age has been significantly influenced by advancements in Generative Artificial Intelligence (GenAI). This survey meticulously examines the dual role of Generative AI in both creating and detecting fake news, offering a systematic review of the current literature spanning from technological underpinnings to ethical implications. By delineating the contributions and identifying gaps within the corpus of existing research, this survey sets a foundation for future investigations into the ways Generative AI continues to shape the landscape of information dissemination.

Enabling Technologies

Central to the creation and detection of fake news are enabling technologies like NLP and transformers. These technologies have evolved to understand and produce human-like text, bringing forth models such as BERT and GPT that are capable of generating content indistinguishable from that created by humans. This section explores how these technologies have been utilized for fake news detection, underscoring their importance in developing more sophisticated detection mechanisms.

Creation of Fake News

The capability of GenAI to automate content creation has profound implications for the production of fake news. By reviewing recent advancements in AI models, this section highlights the increasing realism and complexity of generated text, images, audio, and video content. The survey examines the generation of deepfake technology and its implications for creating synthetic media, emphasizing the need for ongoing research in detection strategies to counteract these advancements.

Case Study: Social Media as Distribution Channel

This section focuses on social media's pivotal role in disseminating fake news. It examines the impact of generative AI on the spread of misinformation across platforms, analyzing how AI-generated content can manipulate public discourse. With social media being the primary distribution channel for fake news, the survey emphasizes the necessity for platforms to employ advanced detection technologies to mitigate the spread of misinformation.

Detection of Fake News

Technological advancements in GenAI have not only facilitated the creation of fake content but also improved methods for detecting such content. This section reviews state-of-the-art technologies and methodologies employed in the detection of fake news. From using transformer-based models to leveraging user behavior and interaction patterns on social media, the survey showcases diverse approaches to identifying and combating fake news.

Deepfakes as Upcoming Technology

The emergence of deepfakes presents new challenges in distinguishing between real and AI-generated content. This section outlines the current state of deepfake technology and its potential for creating highly convincing fake news. The survey reviews detection methods that utilize machine learning algorithms and deep learning models, highlighting the ongoing arms race between deepfake creation and detection technologies.

Open Issues and Future Directions

Despite the significant progress in understanding and mitigating the impact of GenAI on fake news, several open issues remain unaddressed. This section outlines the unresolved challenges, including the rapid evolution of GenAI technologies, ethical concerns surrounding AI-generated content, and the lack of comprehensive datasets for training detection models. The survey emphasizes the importance of interdisciplinary research and collaboration among stakeholders to tackle these challenges effectively.

Conclusion

The interplay between Generative AI and fake news is a dynamic and evolving field that presents both challenges and opportunities. By offering a comprehensive review of the current state of research, this survey provides valuable insights into the capabilities and limitations of GenAI in the context of information dissemination. As the technology continues to advance, future research must focus on developing innovative solutions to detect and mitigate fake news, ensuring the integrity of information in the digital age.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. H. Berghel, “Malice domestic: The cambridge analytica dystopia,” vol. 51, no. 5, pp. 84–89, conference Name: Computer. [Online]. Available: https://ieeexplore.ieee.org/document/8364652
  2. A. Loth. OpenAI GPT-4: Exploring the evolution and impact of generative AI. [Online]. Available: https://alexloth.com/gpt-4-launches-today-the-rise-of-generative-ai-from-neural-networks-to-deepmind-and-openai/
  3. K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, “Fake news detection on social media: A data mining perspective.” [Online]. Available: http://arxiv.org/abs/1708.01967
  4. K. Sharma, F. Qian, H. Jiang, N. Ruchansky, M. Zhang, and Y. Liu, “Combating fake news: A survey on identification and mitigation techniques,” vol. 10, no. 3, pp. 21:1–21:42. [Online]. Available: https://doi.org/10.1145/3305260
  5. R. Oshikawa, J. Qian, and W. Y. Wang, “A survey on natural language processing for fake news detection.” [Online]. Available: http://arxiv.org/abs/1811.00770
  6. H. F. Villela, F. Corrêa, J. S. d. A. N. Ribeiro, A. Rabelo, and D. B. F. Carvalho, “Fake news detection: a systematic literature review of machine learning algorithms and datasets,” vol. 14, no. 1, pp. 47–58, number: 1. [Online]. Available: https://sol.sbc.org.br/journals/index.php/jis/article/view/3020
  7. M. Leo, S. Sharma, and K. Maddulety, “Machine learning in banking risk management: A literature review,” vol. 7, no. 1, p. 29, publisher: MDPI.
  8. Y. Cao, S. Li, Y. Liu, Z. Yan, Y. Dai, P. S. Yu, and L. Sun, “A comprehensive survey of AI-generated content (AIGC): A history of generative AI from GAN to ChatGPT.” [Online]. Available: http://arxiv.org/abs/2303.04226
  9. K. S. Kalyan, A. Rajasekharan, and S. Sangeetha, “Ammus: A survey of transformer-based pretrained models in natural language processing.”
  10. C. Melchior and M. Oliveira, “A systematic literature review of the motivations to share fake news on social media platforms and how to fight them,” p. 14614448231174224, publisher: SAGE Publications. [Online]. Available: https://doi.org/10.1177/14614448231174224
  11. X. Zhou and R. Zafarani, “A survey of fake news: Fundamental theories, detection methods, and opportunities,” vol. 53, no. 5, pp. 109:1–109:40. [Online]. Available: https://doi.org/10.1145/3395046
  12. A. Bondielli and F. Marcelloni, “A survey on fake news and rumour detection techniques,” vol. 497, pp. 38–55. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0020025519304372
  13. Z. Khanjani, G. Watson, and V. P. Janeja, “Audio deepfakes: A survey,” vol. 5, p. 1001063. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9869423/
  14. C. Biever, “ChatGPT broke the turing test — the race is on for new ways to assess AI,” vol. 619, no. 7971, pp. 686–689, bandiera_abtest: a Cg_type: News Feature Number: 7971 Publisher: Nature Publishing Group Subject_term: Computer science, Mathematics and computing, Technology, Society. [Online]. Available: https://www.nature.com/articles/d41586-023-02361-7
  15. R. Gozalo-Brizuela and E. C. Garrido-Merchan, “ChatGPT is not all you need. a state of the art review of large generative AI models.” [Online]. Available: http://arxiv.org/abs/2301.04655
  16. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding.”
  17. H. Jwa, D. Oh, K. Park, J. M. Kang, and H. Lim, “exBAKE: Automatic fake news detection model based on bidirectional encoder representations from transformers (BERT),” vol. 9, no. 19, p. 4062, number: 19 Publisher: Multidisciplinary Digital Publishing Institute. [Online]. Available: https://www.mdpi.com/2076-3417/9/19/4062
  18. A. Maronikolakis, H. Schutze, and M. Stevenson, “Identifying automatically generated headlines using transformers.” [Online]. Available: http://arxiv.org/abs/2009.13375
  19. J. Cao, P. Qi, Q. Sheng, T. Yang, J. Guo, and J. Li, “Exploring the role of visual content in fake news detection.” [Online]. Available: http://arxiv.org/abs/2003.05096
  20. C. Liu, X. Wu, M. Yu, G. Li, J. Jiang, W. Huang, and X. Lu, “A two-stage model based on BERT for short fake news detection,” in Knowledge Science, Engineering and Management, ser. Lecture Notes in Computer Science, C. Douligeris, D. Karagiannis, and D. Apostolou, Eds.   Springer International Publishing, pp. 172–183.
  21. A. Karnyoto, C. Sun, B. Liu, and X. Wang, “Transfer learning and GRU-CRF augmentation for covid-19 fake news detection,” vol. 19, no. 2, pp. 639–658. [Online]. Available: https://doiserbia.nb.rs/Article.aspx?ID=1820-02142100053K
  22. T. N. F. Novo, “MisInfoBot: fight misinformation about COVID on social media.”
  23. K. Bansal, S. Agarwal, and N. Vyas, “Deepfake detection using CNN and DCGANS to drop-out fake multimedia content: A hybrid approach,” in 2023 International Conference on IoT, Communication and Automation Technology (ICICAT), pp. 1–6. [Online]. Available: https://ieeexplore.ieee.org/document/10263628
  24. R. Wang, F. Juefei-Xu, L. Ma, X. Xie, Y. Huang, J. Wang, and Y. Liu, “FakeSpotter: A simple yet robust baseline for spotting AI-synthesized fake faces.” [Online]. Available: http://arxiv.org/abs/1909.06122
  25. J. P. Cardenuto, J. Yang, R. Padilha, R. Wan, D. Moreira, H. Li, S. Wang, F. Andaló, S. Marcel, and A. Rocha, “The age of synthetic realities: Challenges and opportunities.” [Online]. Available: http://arxiv.org/abs/2306.11503
  26. A. Mosallanezhad, K. Shu, and H. Liu, “Topic-preserving synthetic news generation: An adversarial deep reinforcement learning approach.” [Online]. Available: http://arxiv.org/abs/2010.16324
  27. F. Cocchi, L. Baraldi, S. Poppi, M. Cornia, L. Baraldi, and R. Cucchiara, “Unveiling the impact of image transformations on deepfake detection: An experimental analysis,” in Image Analysis and Processing – ICIAP 2023, ser. Lecture Notes in Computer Science, G. L. Foresti, A. Fusiello, and E. Hancock, Eds.   Springer Nature Switzerland, pp. 345–356.
  28. H. Allcott and M. Gentzkow, “Social media and fake news in the 2016 election.” [Online]. Available: https://papers.ssrn.com/abstract=2903810
  29. D. M. J. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, M. Schudson, S. A. Sloman, C. R. Sunstein, E. A. Thorson, D. J. Watts, and J. L. Zittrain, “The science of fake news,” vol. 359, no. 6380, pp. 1094–1096. [Online]. Available: http://arxiv.org/abs/2307.07903
  30. X. Zhou and R. Zafarani, “Network-based fake news detection: A pattern-driven approach.” [Online]. Available: http://arxiv.org/abs/1906.04210
  31. S. Bradshaw, H. Bailey, and P. N. Howard, “Industrialized disinformation: 2020 global inventory of organized social media manipulation (computational propaganda research project).”
  32. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need.” [Online]. Available: http://arxiv.org/abs/1706.03762
  33. L. Sandrini and R. Somogyi, “Generative AI and deceptive news consumption,” vol. 232, p. 111317. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0165176523003427
  34. E. Ferrara, “GenAI against humanity: nefarious applications of generative artificial intelligence and large language models.” [Online]. Available: https://doi.org/10.1007/s42001-024-00250-1
  35. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks.” [Online]. Available: http://arxiv.org/abs/1406.2661
  36. D. P. Kingma and M. Welling, “Auto-encoding variational bayes.” [Online]. Available: http://arxiv.org/abs/1312.6114
  37. S. Ma, H. Wang, L. Ma, L. Wang, W. Wang, S. Huang, L. Dong, R. Wang, J. Xue, and F. Wei, “The era of 1-bit LLMs: All large language models are in 1.58 bits.” [Online]. Available: http://arxiv.org/abs/2402.17764
  38. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training.”
  39. M. Schütz, A. Schindler, M. Siegel, and K. Nazemi, “Automatic fake news detection with pre-trained transformer models,” in Pattern Recognition. ICPR International Workshops and Challenges, A. Del Bimbo, R. Cucchiara, S. Sclaroff, G. M. Farinella, T. Mei, M. Bertini, H. J. Escalante, and R. Vezzani, Eds.   Springer International Publishing, vol. 12667, pp. 627–641, series Title: Lecture Notes in Computer Science. [Online]. Available: http://link.springer.com/10.1007/978-3-030-68787-8_45
  40. A. Loth, “The rise of generative AI: Revolutionizing innovation and enhancing human collaboration.” [Online]. Available: https://www.researchgate.net/publication/369625534_The_Rise_of_Generative_AI_Revolutionizing_Innovation_and_Enhancing_Human_Collaboration
  41. C. L. Otis, True Or False: A CIA Analyst’s Guide to Spotting Fake News.   Feiwel & Friends.
  42. A. Loth, Decisively Digital: From Creating a Culture to Designing Strategy.   John Wiley & Sons.
  43. M. Schütz, J. Böck, M. Andresel, A. Kirchknopf, D. Liakhovets, D. Slijepčević, and A. Schindler, “AIT_fhstp at CheckThat! 2022: Cross-lingual fake news detection with a large pre-trained transformer.”
  44. A. Loth, Visual analytics with Tableau.   John Wiley & Sons.
  45. M. E. Peters, M. Neumann, L. Zettlemoyer, and W.-t. Yih, “Dissecting contextual word embeddings: Architecture and representation.”
  46. N. R. de Oliveira, P. S. Pisa, M. A. Lopez, D. S. V. de Medeiros, and D. M. F. Mattos, “Identifying fake news on social networks based on natural language processing: Trends and challenges,” vol. 12, no. 1, p. 38, number: 1 Publisher: Multidisciplinary Digital Publishing Institute. [Online]. Available: https://www.mdpi.com/2078-2489/12/1/38
  47. J. Howard and S. Ruder, “Universal language model fine-tuning for text classification.”
  48. R. Vijjali, P. Potluri, S. Kumar, and S. Teki, “Two stage transformer model for COVID-19 fake news detection and fact checking.” [Online]. Available: http://arxiv.org/abs/2011.13253
  49. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y. Zhang, “Sparks of artificial general intelligence: Early experiments with GPT-4.” [Online]. Available: http://arxiv.org/abs/2303.12712
  50. A. Zhavoronkov, P. Mamoshina, Q. Vanhaelen, M. Scheibye-Knudsen, A. Moskalev, and A. Aliper, “Artificial intelligence for aging and longevity research: Recent advances and perspectives,” vol. 49, pp. 49–66. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S156816371830240X
  51. L. Jin, F. Tan, S. Jiang, and R. Köker, “Generative adversarial network technologies and applications in computer vision,” vol. 2020. [Online]. Available: https://doi.org/10.1155/2020/1459107
  52. A. Arora and A. Arora, “Generative adversarial networks and synthetic patient data: current challenges and future perspectives,” vol. 9, no. 2, pp. 190–193, publisher: Cambridge University Press. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9345230/
  53. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners.”
  54. J. D. Weisz, M. Muller, J. He, and S. Houde, “Toward general design principles for generative AI applications.” [Online]. Available: http://arxiv.org/abs/2301.05578
  55. F. M. Simon, S. Altay, and H. Mercier, “Misinformation reloaded? fears about the impact of generative AI on misinformation are overblown.” [Online]. Available: https://misinforeview.hks.harvard.edu/article/misinformation-reloaded-fears-about-the-impact-of-generative-ai-on-misinformation-are-overblown/
  56. S. Bradshaw, R. DiResta, and C. Miller, “Playing both sides: Russian state-backed media coverage of the #BlackLivesMatter movement,” p. 19401612221082052, publisher: SAGE Publications Inc. [Online]. Available: https://doi.org/10.1177/19401612221082052
  57. S. Lewandowsky and S. Van Der Linden, “Countering misinformation and fake news through inoculation and prebunking,” vol. 32, no. 2, pp. 348–384. [Online]. Available: https://www.tandfonline.com/doi/full/10.1080/10463283.2021.1876983
  58. K. Shu, D. Mahudeswaran, S. Wang, D. Lee, and H. Liu, “Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media,” vol. 8, no. 3, pp. 171–188, publisher: Mary Ann Liebert, Inc., publishers 140 Huguenot Street, 3rd Floor New ….
  59. J. Yang and Y. Tian, ““others are more vulnerable to fake news than i am”: Third-person effect of COVID-19 fake news on social media users,” vol. 125, p. 106950. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8867061/
  60. G. Pennycook and D. G. Rand, “The psychology of fake news,” vol. 25, no. 5, pp. 388–402, publisher: Elsevier. [Online]. Available: https://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(21)00051-6
  61. Y. Liu and Y.-F. Wu, “Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks,” vol. 32, no. 1, number: 1. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/11268
  62. P. Przybyla, “Capturing the style of fake news,” vol. 34, no. 1, pp. 490–497, number: 01. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/5386
  63. F. Qian, C. Gong, K. Sharma, and Y. Liu, “Neural user response generator: Fake news detection with collective user intelligence,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence.   International Joint Conferences on Artificial Intelligence Organization, pp. 3834–3840. [Online]. Available: https://www.ijcai.org/proceedings/2018/533
  64. K. Shu, S. Wang, and H. Liu, “Beyond news contents: The role of social context for fake news detection,” in Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, ser. WSDM ’19.   Association for Computing Machinery, pp. 312–320. [Online]. Available: https://dl.acm.org/doi/10.1145/3289600.3290994
  65. R. M. Silva, R. L. S. Santos, T. A. Almeida, and T. A. S. Pardo, “Towards automatically filtering fake news in portuguese,” vol. 146, p. 113199. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0957417420300257
  66. G. Pennycook and D. G. Rand, “Who falls for fake news? the roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking,” vol. 88, no. 2, pp. 185–200, _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/jopy.12476. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1111/jopy.12476
  67. E. C. Tandoc Jr, Z. W. Lim, and R. Ling, “Defining “fake news” a typology of scholarly definitions,” vol. 6, no. 2, pp. 137–153, publisher: Taylor & Francis.
  68. J. L. Egelhofer and S. Lecheler, “Fake news as a two-dimensional phenomenon: A framework and research agenda,” vol. 43, no. 2, pp. 97–116, publisher: Taylor & Francis.
  69. M. Osmundsen, A. Bor, P. B. Vahlstrup, A. Bechmann, and M. B. Petersen, “Partisan polarization is the primary psychological motivation behind political fake news sharing on twitter,” vol. 115, no. 3, pp. 999–1015, publisher: Cambridge University Press. [Online]. Available: https://www.cambridge.org/core/journals/american-political-science-review/article/abs/partisan-polarization-is-the-primary-psychological-motivation-behind-political-fake-news-sharing-on-twitter/3F7D2098CD87AE5501F7AD4A7FA83602
  70. Pôle d’excellence cyber: Lutte contre les manipulations de l’information avril 2023. [Online]. Available: https://www.pole-excellence-cyber.org/wp-content/uploads/2024/01/LMI_PEC_2023.pdf
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Alexander Loth (1 paper)
  2. Martin Kappes (2 papers)
  3. Marc-Oliver Pahl (7 papers)
Citations (1)
View on Semantic Scholar
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube