Papers
Topics
Authors
Recent
Search
2000 character limit reached

Effective Structural Encodings via Local Curvature Profiles

Published 24 Nov 2023 in cs.LG and stat.ML | (2311.14864v2)

Abstract: Structural and Positional Encodings can significantly improve the performance of Graph Neural Networks in downstream tasks. Recent literature has begun to systematically investigate differences in the structural properties that these approaches encode, as well as performance trade-offs between them. However, the question of which structural properties yield the most effective encoding remains open. In this paper, we investigate this question from a geometric perspective. We propose a novel structural encoding based on discrete Ricci curvature (Local Curvature Profiles, short LCP) and show that it significantly outperforms existing encoding approaches. We further show that combining local structural encodings, such as LCP, with global positional encodings improves downstream performance, suggesting that they capture complementary geometric information. Finally, we compare different encoding types with (curvature-based) rewiring techniques. Rewiring has recently received a surge of interest due to its ability to improve the performance of Graph Neural Networks by mitigating over-smoothing and over-squashing effects. Our results suggest that utilizing curvature information for structural encodings delivers significantly larger performance increases than rewiring.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. On the bottleneck of graph neural networks and its practical implications. In International Conference on Learning Representations, 2021.
  2. Improving graph neural network expressivity via subgraph isomorphism counting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):657–668, 2022.
  3. A simple yet effective baseline for non-attributed graph classification. ICRL Representation Learning on Graphs and Manifolds, 2018.
  4. An optimal lower bound on the number of variables for graph identification. In 30th Annual Symposium on Foundations of Computer Science, pages 612–617, 1989.
  5. Graph neural networks with learnable structural and positional representations. In International Conference on Learning Representations, 2022a.
  6. Long range graph benchmark. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022b.
  7. Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1–48, 2023.
  8. How powerful are k-hop message passing graph neural networks. Advances in Neural Information Processing Systems, 35:4776–4790, 2022.
  9. Mitigating over-smoothing and over-squashing using augmentations of forman-ricci curvature. arXiv preprint arXiv:2309.09384, 2023.
  10. Robin Forman. Bochner’s Method for Cell Complexes and Combinatorial Ricci Curvature. volume 29, pages 323–374, 2003.
  11. Structure-based protein function prediction using graph convolutional networks. Nature communications, 12(1):3168, 2021.
  12. A new model for learning in graph domains. In Proceedings. 2005 IEEE international joint conference on neural networks, volume 2, pages 729–734, 2005.
  13. Inductive Representation Learning on Large Graphs. In NIPS, pages 1024–1034, 2017.
  14. Local structure in social networks. Sociological methodology, 7:1–45, 1976.
  15. J. Jost and S. Liu. Ollivier’s ricci curvature, local clustering and curvature-dimension inequalities on graphs. Discrete & Computational Geometry, 51(2):300–322, 2014.
  16. Characterizations of forman curvature. arXiv preprint arXiv:2110.04554, 2021.
  17. FoSR: First-order spectral rewiring for addressing oversquashing in GNNs. In The Eleventh International Conference on Learning Representations, 2023.
  18. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR, 2017.
  19. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34:21618–21629, 2021.
  20. Deeper exploiting graph structure information by discrete ricci curvature in a graph transformer. Entropy, 25(6), 2023.
  21. Distance encoding: Design provably more powerful neural networks for graph representation learning. Advances in Neural Information Processing Systems, 33:4465–4478, 2020.
  22. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  23. Andreas Loukas. What graph neural networks cannot learn: depth vs width. In International Conference on Learning Representations, 2019.
  24. Provably powerful graph networks. Advances in neural information processing systems, 32, 2019.
  25. Graphit: Encoding graph structure in transformers. arXiv preprint arXiv:2106.05667, 2021.
  26. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 4602–4609, 2019.
  27. Tudataset: A collection of benchmark datasets for learning with graphs. CoRR, abs/2007.08663, 2020. URL https://arxiv.org/abs/2007.08663.
  28. Revisiting over-smoothing and over-squashing using ollivier-ricci curvature. In International Conference on Machine Learning, pages 25956–25979. PMLR, 2023.
  29. Community detection on networks with ricci flow. Scientific reports, 9(1):1–12, 2019.
  30. Y. Ollivier. Ricci curvature of markov chains on metric spaces. Journal of Functional Analysis, 256(3):810–864, 2009.
  31. Grpe: Relative positional encoding for graph transformer. arXiv preprint arXiv:2201.12787, 2022.
  32. Recipe for a general, powerful, scalable graph transformer. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
  33. Graph neural networks in particle physics. Machine Learning: Science and Technology, 2(2):021001, 2020.
  34. Ollivier-ricci curvature-based method to community detection in complex networks. Scientific reports, 9(1):1–12, 2019.
  35. On the expressive power of ollivier-ricci curvature on graphs. In TAG-ML Workshop, 2023.
  36. Curvature-based clustering on graphs. arXiv preprint arXiv:2307.10155, 2023.
  37. Understanding over-squashing and bottlenecks on graphs via curvature. In International Conference on Learning Representations, 2022.
  38. Graph Attention Networks. In ICLR, 2018.
  39. Characterizing complex networks with forman-ricci curvature and associated geometric flows. Journal of Complex Networks, 5(4):527–550, 2017.
  40. B Weisfeiler and A Leman. The reduction of a graph to canonical form and the algebra which appears therein. Nauchno-Technicheskaya Informatsia, vol. 2(9):12-16, 1968.
  41. Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 55(5):1–37, 2022.
  42. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  43. Revisiting semi-supervised learning with graph embeddings. CoRR, abs/1603.08861, 2016. URL http://arxiv.org/abs/1603.08861.
  44. Curvature graph network. In International Conference on Learning Representations, 2020.
  45. From stars to subgraphs: Uplifting any GNN with local structure awareness. In International Conference on Learning Representations, 2022.
  46. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics, 34(13):i457–i466, 2018.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free