Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Universally Robust Graph Neural Networks by Preserving Neighbor Similarity (2401.09754v1)

Published 18 Jan 2024 in cs.LG, cs.CR, and cs.SI

Abstract: Despite the tremendous success of graph neural networks in learning relational data, it has been widely investigated that graph neural networks are vulnerable to structural attacks on homophilic graphs. Motivated by this, a surge of robust models is crafted to enhance the adversarial robustness of graph neural networks on homophilic graphs. However, the vulnerability based on heterophilic graphs remains a mystery to us. To bridge this gap, in this paper, we start to explore the vulnerability of graph neural networks on heterophilic graphs and theoretically prove that the update of the negative classification loss is negatively correlated with the pairwise similarities based on the powered aggregated neighbor features. This theoretical proof explains the empirical observations that the graph attacker tends to connect dissimilar node pairs based on the similarities of neighbor features instead of ego features both on homophilic and heterophilic graphs. In this way, we novelly introduce a novel robust model termed NSPGNN which incorporates a dual-kNN graphs pipeline to supervise the neighbor similarity-guided propagation. This propagation utilizes the low-pass filter to smooth the features of node pairs along the positive kNN graphs and the high-pass filter to discriminate the features of node pairs along the negative kNN graphs. Extensive experiments on both homophilic and heterophilic graphs validate the universal robustness of NSPGNN compared to the state-of-the-art methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Abien Fred Agarap. 2018. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375 (2018).
  2. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261 (2018).
  3. Beyond low-frequency information in graph convolutional networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3950–3957.
  4. Adaptive Universal Generalized PageRank Graph Neural Network. In International Conference on Learning Representations. https://openreview.net/forum?id=n6jl7fLxrP
  5. Applications of link prediction in social networks: A review. Journal of Network and Computer Applications 166 (2020), 102716.
  6. GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks. In The First Learning on Graphs Conference. https://openreview.net/forum?id=kvwWjYQtmw
  7. Gbk-gnn: Gated bi-kernel graph neural networks for modeling both homophily and heterophily. In Proceedings of the ACM Web Conference 2022. 1550–1558.
  8. All You Need Is Low (Rank): Defending Against Adversarial Attacks on Graphs. In Proceedings of the 13th International Conference on Web Search and Data Mining (Houston, TX, USA) (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 169–177. https://doi.org/10.1145/3336191.3371789
  9. Matthias Fey and Jan E. Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
  10. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017).
  11. Simon Haykin. 1994. Neural networks: a comprehensive foundation. Prentice Hall PTR.
  12. Block modeling-guided graph convolutional neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 36. 4022–4029.
  13. Graph structure learning for robust graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining. 66–74.
  14. Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs.LG]
  15. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=SJU4ayYgl
  16. Semi-supervised graph classification: A hierarchical graph perspective. In The World Wide Web Conference. 972–982.
  17. Recent Advances in Reliable Deep Graph Learning: Inherent Noise, Distribution Shift, and Adversarial Attack. arXiv:2202.07114 [cs.LG]
  18. Adversarial Attack on Community Detection by Hiding Individuals. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW ’20). Association for Computing Machinery, New York, NY, USA, 917–927. https://doi.org/10.1145/3366423.3380171
  19. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
  20. Deeprobust: A pytorch library for adversarial attacks and defenses. arXiv preprint arXiv:2005.06149 (2020).
  21. Graph Neural Networks with Adaptive Residual. In Advances in Neural Information Processing Systems, A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). https://openreview.net/forum?id=hfkER_KJiNw
  22. Elastic graph neural networks. In International Conference on Machine Learning. PMLR, 6837–6849.
  23. Revisiting heterophily for graph neural networks. Advances in neural information processing systems 35 (2022), 1362–1375.
  24. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43–52.
  25. Automating the construction of internet portals with machine learning. Information Retrieval 3 (2000), 127–163.
  26. Birds of a feather: Homophily in social networks. Annual review of sociology 27, 1 (2001), 415–444.
  27. Separators for Sphere-Packings and Nearest Neighbor Graphs. J. ACM 44, 1 (jan 1997), 1–29. https://doi.org/10.1145/256292.256294
  28. Ryan A. Rossi and Nesreen K. Ahmed. 2015. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI. https://networkrepository.com
  29. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research 12, 9 (2011).
  30. Link prediction in citation networks. Journal of the American society for information science and technology 63, 1 (2012), 78–85.
  31. Amit Singhal et al. 2001. Modern information retrieval: A brief overview. IEEE Data Eng. Bull. 24, 4 (2001), 35–43.
  32. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
  33. Anti-money laundering in bitcoin: Experimenting with graph convolutional networks for financial forensics. arXiv preprint arXiv:1908.02591 (2019).
  34. Simplifying Graph Convolutional Networks. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 6861–6871. https://proceedings.mlr.press/v97/wu19e.html
  35. Adversarial Examples for Graph Data: Deep Insights into Attack and Defense. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 4816–4823. https://doi.org/10.24963/ijcai.2019/669
  36. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4–24.
  37. Topology Attack and Defense for Graph Neural Networks: An Optimization Perspective. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 3961–3967. https://doi.org/10.24963/ijcai.2019/550
  38. Xiang Zhang and Marinka Zitnik. 2020. GNNGUARD: Defending Graph Neural Networks against Adversarial Attacks. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS’20). Curran Associates Inc., Red Hook, NY, USA, Article 777, 13 pages.
  39. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering 34, 1 (2020), 249–270.
  40. Graph neural networks: A review of methods and applications. AI open 1 (2020), 57–81.
  41. Robust Graph Convolutional Networks Against Adversarial Attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Anchorage, AK, USA) (KDD ’19). Association for Computing Machinery, New York, NY, USA, 1399–1407. https://doi.org/10.1145/3292500.3330851
  42. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in neural information processing systems 33 (2020), 7793–7804.
  43. BinarizedAttack: Structural Poisoning Attacks to Graph-based Anomaly Detection. In 2022 IEEE 38th International Conference on Data Engineering (ICDE). 14–26. https://doi.org/10.1109/ICDE53745.2022.00006
  44. Adversarial Attacks on Neural Networks for Graph Data. In SIGKDD. 2847–2856.
  45. Daniel Zügner and Stephan Günnemann. 2019. Adversarial Attacks on Graph Neural Networks via Meta Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=Bylnx209YX

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets