Papers
Topics
Authors
Recent
Search
2000 character limit reached

Challenges in Pre-Training Graph Neural Networks for Context-Based Fake News Detection: An Evaluation of Current Strategies and Resource Limitations

Published 28 Feb 2024 in cs.CL | (2402.18179v1)

Abstract: Pre-training of neural networks has recently revolutionized the field of NLP and has before demonstrated its effectiveness in computer vision. At the same time, advances around the detection of fake news were mainly driven by the context-based paradigm, where different types of signals (e.g. from social media) form graph-like structures that hold contextual information apart from the news article to classify. We propose to merge these two developments by applying pre-training of Graph Neural Networks (GNNs) in the domain of context-based fake news detection. Our experiments provide an evaluation of different pre-training strategies for graph-based misinformation detection and demonstrate that transfer learning does currently not lead to significant improvements over training a model from scratch in the domain. We argue that a major current issue is the lack of suitable large-scale resources that can be used for pre-training.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Kai Shu. Combating Disinformation on Social Media and Its Challenges: A Computational Perspective. Proceedings of the AAAI Conference on Artificial Intelligence, 37(13):15454–15454, Sep. 2023.
  2. A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities. ACM Comput. Surv., 53(5), sep 2020.
  3. Uncertainty-aware Propagation Structure Reconstruction for Fake News Detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2759–2768, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics.
  4. Divide-and-Conquer: Post-User Interaction Network for Fake News Detection on Social Media. In Proceedings of the ACM Web Conference 2022, WWW ’22, page 1148–1158, New York, NY, USA, 2022. Association for Computing Machinery.
  5. Tackling Fake News Detection by Continually Improving Social Context Representations using Graph Neural Networks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1363–1380, Dublin, Ireland, May 2022. Association for Computational Linguistics.
  6. A Survey on Automated Fact-Checking. Transactions of the Association for Computational Linguistics, 10:178–206, 02 2022.
  7. Beyond News Contents: The Role of Social Context for Fake News Detection. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM ’19, page 312–320, New York, NY, USA, 2019. Association for Computing Machinery.
  8. Detecting Users Prone to Spread Fake News on Arabic Twitter. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 12–22, Marseille, France, June 2022. European Language Resources Association.
  9. GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 505–514, Online, July 2020. Association for Computational Linguistics.
  10. True or False? Detecting False Information on Social Media Using Graph Neural Networks. In Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022), pages 222–229, Gyeongju, Republic of Korea, October 2022. Association for Computational Linguistics.
  11. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  12. Language Models are Few-Shot Learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.
  13. OpenAI. GPT-4 Technical Report. arXiv, 2023. 10.48550/arxiv.2303.08774.
  14. Shaina Raza. Automatic Fake News Detection in Political Platforms - A Transformer-based Approach. In Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021), pages 68–78, Online, August 2021. Association for Computational Linguistics.
  15. Applying Automatic Text Summarization for Fake News Detection. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2702–2713, Marseille, France, June 2022. European Language Resources Association.
  16. Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations, 2020.
  17. Debunking Rumors on Twitter with Tree Transformer. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5455–5466, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.
  18. Exploiting Microblog Conversation Structures to Detect Rumors. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5420–5429, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.
  19. Graph-based Modeling of Online Communities for Fake News Detection. CoRR, abs/2008.06274, 2020.
  20. Tackling Fake News Detection by Interactively Learning Representations using Graph Neural Networks. In Proceedings of the First Workshop on Interactive Learning for Natural Language Processing, pages 46–53, Online, August 2021. Association for Computational Linguistics.
  21. FANG: Leveraging Social Context for Fake News Detection Using Graph Representation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 1165–1174, New York, NY, USA, 2020. Association for Computing Machinery.
  22. Fake News Detection with Heterogeneous Transformer. arXiv, 2022. 10.48550/arxiv.2205.03100.
  23. Exploring Fake News Detection with Heterogeneous Social Media Context Graphs. In Advances in Information Retrieval, pages 396–405, Cham, 2023. Springer Nature Switzerland.
  24. Pre-Training Graph Neural Networks for Cold-Start Users and Items Representation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21, page 265–273, New York, NY, USA, 2021. Association for Computing Machinery.
  25. Pre-training graph neural networks for link prediction in biomedical networks. Bioinformatics, 38(8):2254–2262, 02 2022.
  26. Pre-Training on Large-Scale Heterogeneous Graph. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 756–766, New York, NY, USA, 2021. Association for Computing Machinery.
  27. PF-HIN:Pre-Training for Heterogeneous Information Networks. IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2022.
  28. Self-supervised heterogeneous graph pre-training based on structural clustering. Advances in Neural Information Processing Systems, 35:16962–16974, 2022.
  29. Learning to Pre-train Graph Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5):4276–4284, May 2021.
  30. GPT-GNN: Generative Pre-Training of Graph Neural Networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 1857–1867, New York, NY, USA, 2020. Association for Computing Machinery.
  31. Detect Rumors in Microblog Posts Using Propagation Structure via Kernel Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 708–717, Vancouver, Canada, July 2017. Association for Computational Linguistics.
  32. All-in-one: Multi-task Learning for Rumour Verification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3402–3413, Santa Fe, New Mexico, USA, August 2018. Association for Computational Linguistics.
  33. Detecting Rumors from Microblogs with Recurrent Neural Networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, page 3818–3824. AAAI Press, 2016.
  34. FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media. Big Data, 8(3):171–188, 2020. PMID: 32491943.
  35. FbMultiLingMisinfo: Challenging Large-Scale Multilingual Benchmark for Misinformation Detection. In 2022 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2022.
  36. Heterogeneous Graph Transformer. In Proceedings of The Web Conference 2020, WWW ’20, page 2704–2710, New York, NY, USA, 2020. Association for Computing Machinery.
  37. Attention is All you Need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  38. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
  39. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 2024. 3rd edition (draft), 3rd February 2024. https://web.stanford.edu/~jurafsky/slp3/.
  40. Dynamic graph neural network for fake news detection. Neurocomputing, 505:362–374, 2022.
  41. Reflections on 30 Years of Language Resource Development and Sharing. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 543–550, Marseille, France, June 2022. European Language Resources Association.
  42. Interactively learning social media representations improves news source factuality detection. In Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi, editors, Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings), pages 291–311, Nusa Dua, Bali, November 2023. Association for Computational Linguistics.
  43. Predicting factuality of reporting and bias of news media sources. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3528–3539, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up