Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VGA: Vision and Graph Fused Attention Network for Rumor Detection (2401.01759v1)

Published 3 Jan 2024 in cs.SI, cs.CL, cs.CV, and cs.MM

Abstract: With the development of social media, rumors have been spread broadly on social media platforms, causing great harm to society. Beside textual information, many rumors also use manipulated images or conceal textual information within images to deceive people and avoid being detected, making multimodal rumor detection be a critical problem. The majority of multimodal rumor detection methods mainly concentrate on extracting features of source claims and their corresponding images, while ignoring the comments of rumors and their propagation structures. These comments and structures imply the wisdom of crowds and are proved to be crucial to debunk rumors. Moreover, these methods usually only extract visual features in a basic manner, seldom consider tampering or textual information in images. Therefore, in this study, we propose a novel Vision and Graph Fused Attention Network (VGA) for rumor detection to utilize propagation structures among posts so as to obtain the crowd opinions and further explore visual tampering features, as well as the textual information hidden in images. We conduct extensive experiments on three datasets, demonstrating that VGA can effectively detect multimodal rumors and outperform state-of-the-art methods significantly.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Rumor detection on social media with bi-directional graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 549–556.
  2. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013).
  3. Information Credibility on Twitter. In Proceedings of the 20th International Conference on World Wide Web (Hyderabad, India) (WWW ’11). Association for Computing Machinery, New York, NY, USA, 675–684. https://doi.org/10.1145/1963405.1963500
  4. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  5. Call attention to rumors: Deep attention based recurrent neural networks for early rumor detection. In Pacific-Asia conference on knowledge discovery and data mining. Springer, 40–52.
  6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 4171–4186.
  7. Jessica Fridrich and Jan Kodovsky. 2012. Rich models for steganalysis of digital images. IEEE Transactions on information Forensics and Security 7, 3 (2012), 868–882.
  8. Rumor detection with hierarchical social attention network. In Proceedings of the 27th ACM international conference on information and knowledge management. 943–951.
  9. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  10. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In Proceedings of the 25th ACM international conference on Multimedia. 795–816.
  11. MVAE: Multimodal Variational Autoencoder for Fake News Detection. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19). Association for Computing Machinery, New York, NY, USA, 2915–2921. https://doi.org/10.1145/3308558.3313552
  12. Interpretable rumor detection in microblogs by attending to user interactions. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 8783–8790.
  13. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  14. Prominent Features of Rumor Propagation in Online Social Media. In 2013 IEEE 13th International Conference on Data Mining. 1103–1108. https://doi.org/10.1109/ICDM.2013.61
  15. Entity-oriented multi-modal alignment and fusion network for fake news detection. IEEE Transactions on Multimedia (2021).
  16. Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning. In Findings of the Association for Computational Linguistics: NAACL 2022. Association for Computational Linguistics, Seattle, United States, 2543–2556. https://doi.org/10.18653/v1/2022.findings-naacl.194
  17. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10012–10022.
  18. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems 32 (2019).
  19. Detecting Rumors from Microblogs with Recurrent Neural Networks. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (New York, New York, USA) (IJCAI’16). AAAI Press, 3818–3824.
  20. Detect Rumors in Microblog Posts Using Propagation Structure via Kernel Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 708–717. https://doi.org/10.18653/v1/P17-1066
  21. Rumor Detection on Twitter with Tree-structured Recursive Neural Networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 1980–1989. https://doi.org/10.18653/v1/P18-1184
  22. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, Vol. 30. Atlanta, Georgia, USA, 3.
  23. Exploiting multi-domain visual information for fake news detection. In 2019 IEEE international conference on data mining (ICDM). IEEE, 518–527.
  24. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
  25. Spotfake: A multi-modal framework for fake news detection. In 2019 IEEE fifth international conference on multimedia big data (BigMM). IEEE, 39–47.
  26. A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networks. Information Processing Management 58, 1 (2021), 102437.
  27. CED: credible early detection of social media rumors. IEEE Transactions on Knowledge and Data Engineering 33, 8 (2019), 3035–3047.
  28. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
  29. Attention is all you need. Advances in neural information processing systems 30 (2017).
  30. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  31. Extracting and Composing Robust Features with Denoising Autoencoders. In Proceedings of the 25th International Conference on Machine Learning (Helsinki, Finland) (ICML ’08). Association for Computing Machinery, New York, NY, USA, 1096–1103. https://doi.org/10.1145/1390156.1390294
  32. EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery; Data Mining (London, United Kingdom) (KDD ’18). Association for Computing Machinery, New York, NY, USA, 849–857. https://doi.org/10.1145/3219819.3219903
  33. Multimodal Emergent Fake News Detection via Meta Neural Process Networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery; Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 3708–3716. https://doi.org/10.1145/3447548.3467153
  34. Towards Propagation Uncertainty: Edge-enhanced Bayesian Graph Convolutional Networks for Rumor Detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 3845–3854. https://doi.org/10.18653/v1/2021.acl-long.297
  35. A State-independent and Time-evolving Network for Early Rumor Detection in Social Media. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 9042–9051. https://doi.org/10.18653/v1/2020.emnlp-main.727
  36. Detecting fake news by exploring the consistency of multimodal data. Information Processing Management 58, 5 (2021), 102610.
  37. Automatic Detection of Rumor on Sina Weibo. In Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics (Beijing, China) (MDS ’12). Association for Computing Machinery, New York, NY, USA, Article 13, 7 pages. https://doi.org/10.1145/2350190.2350203
  38. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems 33 (2020), 5812–5823.
  39. A Convolutional Approach for Misinformation Identification. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. 3901–3907. https://doi.org/10.24963/ijcai.2017/545
  40. Multi-modal knowledge-aware event memory network for social media rumor detection. In Proceedings of the 27th ACM international conference on multimedia. 1942–1951.
  41. MFAN: Multi-modal Feature-enhanced Attention Networks for Rumor Detection. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 2413–2419. https://doi.org/10.24963/ijcai.2022/335 Main Track.
  42. Learning rich features for image manipulation detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1053–1061.
  43. SAFE: Similarity-Aware Multi-modal Fake News Detection. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 354–367.
  44. Detection and Resolution of Rumours in Social Media: A Survey. ACM Comput. Surv. 51, 2, Article 32 (feb 2018), 36 pages. https://doi.org/10.1145/3161603
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Lin Bai (34 papers)
  2. Caiyan Jia (21 papers)
  3. Ziying Song (23 papers)
  4. Chaoqun Cui (1 paper)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now