Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GPatcher: A Simple and Adaptive MLP Model for Alleviating Graph Heterophily (2306.14340v1)

Published 25 Jun 2023 in cs.LG, cs.AI, and cs.SI

Abstract: While graph heterophily has been extensively studied in recent years, a fundamental research question largely remains nascent: How and to what extent will graph heterophily affect the prediction performance of graph neural networks (GNNs)? In this paper, we aim to demystify the impact of graph heterophily on GNN spectral filters. Our theoretical results show that it is essential to design adaptive polynomial filters that adapts different degrees of graph heterophily to guarantee the generalization performance of GNNs. Inspired by our theoretical findings, we propose a simple yet powerful GNN named GPatcher by leveraging the MLP-Mixer architectures. Our approach comprises two main components: (1) an adaptive patch extractor function that automatically transforms each node's non-Euclidean graph representations to Euclidean patch representations given different degrees of heterophily, and (2) an efficient patch mixer function that learns salient node representation from both the local context information and the global positional information. Through extensive experiments, the GPatcher model demonstrates outstanding performance on node classification compared with popular homophily GNNs and state-of-the-art heterophily GNNs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Skipgnn: predicting molecular interactions with skip-graph networks. Scientific reports, 10(1):1–16, 2020.
  2. Identifying drug–target interactions based on graph convolutional network and deep neural network. Briefings in bioinformatics, 22(2):2141–2150, 2021.
  3. Could graph neural networks learn better molecular representation for drug discovery? a comparison study of descriptor-based and graph-based models. Journal of cheminformatics, 13(1):1–23, 2021.
  4. A semi-supervised graph attentive network for financial fraud detection. In 2019 IEEE International Conference on Data Mining (ICDM), pages 598–607. IEEE, 2019.
  5. Graph neural networks for natural language processing: A survey. Foundations and Trends in Machine Learning, 16(2):119–328, 2023.
  6. Semantic graph convolutional networks for 3d human pose regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3425–3435, 2019.
  7. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5177–5186, 2019.
  8. Revisiting graph neural networks: All we have is low-pass filters. ArXiv, abs/1905.09550, 2019.
  9. Graph neural networks with heterophily. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11168–11176, 2021.
  10. Graph neural networks for graphs with heterophily: A survey, 2022.
  11. Revisiting heterophily for graph neural networks. In NeurIPS, 2022.
  12. Graph-less neural networks: Teaching old MLPs new tricks via distillation. In International Conference on Learning Representations, 2022.
  13. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems, 33, 2020.
  14. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In international conference on machine learning, pages 21–29. PMLR, 2019.
  15. Beyond smoothing: Unsupervised graph representation learning with edge heterophily discriminating. In AAAI, 2023.
  16. Block modeling-guided graph convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4022–4029, 2022.
  17. A survey of visual transformers. IEEE Transactions on Neural Networks and Learning Systems, 2023.
  18. A survey of transformers. AI Open, 2022.
  19. LINE: large-scale information network embedding. In Aldo Gangemi, Stefano Leonardi, and Alessandro Panconesi, editors, Proceedings of the 24th International Conference on World Wide Web, WWW 2015, Florence, Italy, May 18-22, 2015, pages 1067–1077. ACM, 2015.
  20. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR), 2017.
  21. Position-aware graph neural networks. In International conference on machine learning, pages 7134–7143. PMLR, 2019.
  22. Simplifying graph convolutional networks. In International conference on machine learning, pages 6861–6871. PMLR, 2019.
  23. Multi-scale attributed node embedding. Journal of Complex Networks, 9(2):cnab014, 2021.
  24. Social influence analysis in large-scale networks. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’09, page 807–816, New York, NY, USA, 2009. Association for Computing Machinery.
  25. Using fuzzy logic to leverage html markup for web page representation. IEEE Transactions on Fuzzy Systems, 25(4):919–933, 2016.
  26. Beyond low-pass filters: Adaptive feature propagation on graphs. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II 21, pages 450–465. Springer, 2021.
  27. Rethinking graph neural networks for the graph coloring problem. arXiv preprint arXiv:2208.06975, 2022.
  28. A graph neural network with negative message passing for graph coloring. arXiv preprint arXiv:2301.11164, 2023.
  29. Graph embedding using frequency filtering. IEEE transactions on pattern analysis and machine intelligence, 43(2):473–484, 2019.
  30. Protein interface prediction using graph convolutional networks. Advances in neural information processing systems, 30, 2017.
  31. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  32. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, volume 8, page 1, 2012.
  33. Adaptive universal generalized pagerank graph neural network. In International Conference on Learning Representations, 2021.
  34. Chebnet: Efficient and stable constructions of deep neural networks with rectified power units using chebyshev approximations. arXiv preprint arXiv:1911.05467, 2019.
  35. Predict then propagate: Graph neural networks meet personalized pagerank. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  36. Representation learning on graphs with jumping knowledge networks. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 5449–5458. PMLR, 2018.
  37. Graph Attention Networks. International Conference on Learning Representations, 2018.
  38. Inductive representation learning on large graphs. In NIPS, 2017.
  39. Hl-net: Heterophily learning network for scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19476–19485, June 2022.
  40. Do we really need complicated model architectures for temporal networks? arXiv preprint arXiv:2302.11636, 2023.
  41. A generalization of vit/mlp-mixer to graphs, 2022.
  42. Beyond low-frequency information in graph convolutional networks. In AAAI. AAAI Press, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now