Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AGS-GNN: Attribute-guided Sampling for Graph Neural Networks (2405.15218v1)

Published 24 May 2024 in cs.LG

Abstract: We propose AGS-GNN, a novel attribute-guided sampling algorithm for Graph Neural Networks (GNNs) that exploits node features and connectivity structure of a graph while simultaneously adapting for both homophily and heterophily in graphs. (In homophilic graphs vertices of the same class are more likely to be connected, and vertices of different classes tend to be linked in heterophilic graphs.) While GNNs have been successfully applied to homophilic graphs, their application to heterophilic graphs remains challenging. The best-performing GNNs for heterophilic graphs do not fit the sampling paradigm, suffer high computational costs, and are not inductive. We employ samplers based on feature-similarity and feature-diversity to select subsets of neighbors for a node, and adaptively capture information from homophilic and heterophilic neighborhoods using dual channels. Currently, AGS-GNN is the only algorithm that we know of that explicitly controls homophily in the sampled subgraph through similar and diverse neighborhood samples. For diverse neighborhood sampling, we employ submodularity, which was not used in this context prior to our work. The sampling distribution is pre-computed and highly parallel, achieving the desired scalability. Using an extensive dataset consisting of 35 small ($\le$ 100K nodes) and large (>100K nodes) homophilic and heterophilic graphs, we demonstrate the superiority of AGS-GNN compare to the current approaches in the literature. AGS-GNN achieves comparable test accuracy to the best-performing heterophilic GNNs, even outperforming methods using the entire graph for node classification. AGS-GNN also converges faster compared to methods that sample neighborhoods randomly, and can be incorporated into existing GNN models that employ node or graph sampling.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Mixhop: Higher-order graph convolutional architectures via sparsified neighborhood mixing. In International Conference on Machine Learning. PMLR, 21–29.
  2. Label-aware graph convolutional networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1977–1980.
  3. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling. In International Conference on Learning Representations.
  4. Stochastic Training of Graph Convolutional Networks with Variance Reduction. In International Conference on Machine Learning. PMLR, 942–950.
  5. Simple and deep graph convolutional networks. In International Conference on Machine Learning. PMLR, 1725–1735.
  6. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 257–266.
  7. Davide Chicco. 2021. Siamese neural networks: An overview. Artificial neural networks (2021), 73–94.
  8. Adaptive universal generalized pagerank graph neural network. arXiv preprint arXiv:2006.07988 (2020).
  9. Venkatesan Nallampatti Ekambaram. 2014. Graph-structured data viewed through a Fourier lens. University of California, Berkeley.
  10. Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428 (2019).
  11. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of The Web Conference 2020. 2331–2341.
  12. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018).
  13. CiteSeer: An automatic citation indexing system. In Proceedings of the third ACM Conference on Digital Libraries. 89–98.
  14. Deep learning. MIT press.
  15. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1024–1034.
  16. Convolutional neural networks on graphs with chebyshev approximation, revisited. Advances in Neural Information Processing Systems 35 (2022), 7264–7276.
  17. Combining label propagation and simple models out-performs graph neural networks. arXiv preprint arXiv:2010.13993 (2020).
  18. Adaptive sampling towards fast graph representation learning. Advances in Neural Information Processing Systems 31 (2018).
  19. Node similarity preserving graph convolutional networks. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 148–156.
  20. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  21. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  22. Andreas Krause and Daniel Golovin. 2014. Submodular function maximization. Tractability 3, 71-104 (2014), 3.
  23. Finding global homophily in graph neural networks when meeting heterophily. In International Conference on Machine Learning. PMLR, 13242–13256.
  24. LD2: Scalable Heterophilous Graph Neural Network with Decoupled Embeddings. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=7zkFc9TGKz
  25. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems 34 (2021), 20887–20902.
  26. Non-local graph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 12 (2021), 10270–10276.
  27. Revisiting heterophily for graph neural networks. arXiv preprint arXiv:2210.07606 (2022).
  28. Complete the missing half: Augmenting aggregation filtering with diversification for graph convolutional networks. arXiv preprint arXiv:2008.08844 (2020).
  29. Michel Minoux. 1978. Accelerated greedy algorithms for maximizing submodular set functions. Optimization Techniques (1978), 234–243.
  30. Query-driven active surveying for collective classification. In 10th International Workshop on Mining and Learning with Graphs, Vol. 8. 1.
  31. An analysis of approximations for maximizing submodular set functions—I. Mathematical programming 14 (1978), 265–294.
  32. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32 (2019).
  33. Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 (2020).
  34. Characterizing graph datasets for node classification: Beyond homophily-heterophily dichotomy. arXiv preprint arXiv:2209.06177 (2022).
  35. A critical look at the evaluation of GNNs under heterophily: are we really making progress? arXiv preprint arXiv:2302.11640 (2023).
  36. Multi-scale attributed node embedding. Journal of Complex Networks 9, 2 (2021), cnab014.
  37. apricot: Submodular selection for data summarization in Python. The Journal of Machine Learning Research 21, 1 (2020), 6474–6479.
  38. Collective classification in network data. AI magazine 29, 3 (2008), 93–93.
  39. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).
  40. Jesper E Van Engelen and Holger H Hoos. 2020. A survey on semi-supervised learning. Machine learning 109, 2 (2020), 373–440.
  41. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
  42. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019).
  43. Simplifying graph convolutional networks. In International Conference on Machine Learning. PMLR, 6861–6871.
  44. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
  45. Representation learning on graphs with jumping knowledge networks. In International Conference on Machine Learning. PMLR, 5453–5462.
  46. Node Classification Beyond Homophily: Towards a General Solution. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2862–2873.
  47. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. In 2022 IEEE International Conference on Data Mining (ICDM). IEEE, 1287–1292.
  48. Diverse message passing for attribute with heterophily. Advances in Neural Information Processing Systems 34 (2021), 4751–4763.
  49. Graph pointer neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8832–8839.
  50. Yang Ye and Shihao Ji. 2021. Sparse graph attention networks. IEEE Transactions on Knowledge and Data Engineering (2021).
  51. GraphSAINT: Graph Sampling Based Inductive Learning Method. In International Conference on Learning Representations.
  52. Elena Zheleva and Lise Getoor. 2009. To join or not to join: the illusion of privacy in social networks with mixed public and private user profiles. In Proceedings of the 18th International Conference on World Wide Web. 531–540.
  53. Robust graph representation learning via neural sparsification. In International Conference on Machine Learning. PMLR, 11458–11468.
  54. Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv:2202.07082 (2022).
  55. Learning with local and global consistency. Advances in Neural Information Processing Systems 16 (2003).
  56. Graph neural networks with heterophily. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11168–11176.
  57. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in Neural Information Processing Systems 33 (2020), 7793–7804.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets