Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Link Prediction under Heterophily: A Physics-Inspired Graph Neural Network Approach (2402.14802v1)

Published 22 Feb 2024 in cs.LG, cs.IR, and cs.SI

Abstract: In the past years, Graph Neural Networks (GNNs) have become the `de facto' standard in various deep learning domains, thanks to their flexibility in modeling real-world phenomena represented as graphs. However, the message-passing mechanism of GNNs faces challenges in learnability and expressivity, hindering high performance on heterophilic graphs, where adjacent nodes frequently have different labels. Most existing solutions addressing these challenges are primarily confined to specific benchmarks focused on node classification tasks. This narrow focus restricts the potential impact that link prediction under heterophily could offer in several applications, including recommender systems. For example, in social networks, two users may be connected for some latent reason, making it challenging to predict such connections in advance. Physics-Inspired GNNs such as GRAFF provided a significant contribution to enhance node classification performance under heterophily, thanks to the adoption of physics biases in the message-passing. Drawing inspiration from these findings, we advocate that the methodology employed by GRAFF can improve link prediction performance as well. To further explore this hypothesis, we introduce GRAFF-LP, an extension of GRAFF to link prediction. We evaluate its efficacy within a recent collection of heterophilic graphs, establishing a new benchmark for link prediction under heterophily. Our approach surpasses previous methods, in most of the datasets, showcasing a strong flexibility in different contexts, and achieving relative AUROC improvements of up to 26.7%.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Lada A Adamic and Eytan Adar. 2003. Friends and neighbors on the Web. Social Networks 25, 3 (2003), 211–230. https://doi.org/10.1016/S0378-8733(03)00009-1
  2. Do More Negative Samples Necessarily Hurt in Contrastive Learning? arXiv:2205.01789 [cs.LG]
  3. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv:1409.0473 [cs.CL]
  4. A Survey on Physics Informed Reinforcement Learning: Review and Open Problems. arXiv:2309.01909 [cs.LG]
  5. Barabasi and Albert. 1999. Emergence of Scaling in Random Networks. Science 286, 5439 (oct 1999), 509–512. https://doi.org/10.1126/science.286.5439.509
  6. Graph Convolutional Encoders for Syntax-aware Neural Machine Translation. arXiv:1704.04675 [cs.CL]
  7. Interaction Networks for Learning about Objects, Relations and Physics. arXiv:1612.00222 [cs.AI]
  8. Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs. arXiv:2202.04579 [cs.LG]
  9. Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30, 1 (1998), 107–117. https://doi.org/10.1016/S0169-7552(98)00110-X Proceedings of the Seventh International World Wide Web Conference.
  10. Chen Cai and Yusu Wang. 2020. A Note on Over-Smoothing for Graph Neural Networks. arXiv:2006.13318 [cs.LG]
  11. Question Answering by Reasoning Across Documents with Graph Convolutional Networks. arXiv:1808.09920 [cs.CL]
  12. GRAND: Graph Neural Diffusion. arXiv:2106.10934 [cs.LG]
  13. Graph Neural Networks for Link Prediction with Subgraph Sketching. arXiv:2209.15486 [cs.LG]
  14. Adaptive Universal Generalized PageRank Graph Neural Network. arXiv:2006.07988 [cs.LG]
  15. GREAD: Graph Neural Reaction-Diffusion Networks. arXiv:2211.14208 [cs.LG]
  16. Hyperspherical Variational Auto-Encoders. arXiv:1804.00891 [stat.ML]
  17. Graph Neural Networks for Social Recommendation. arXiv:1902.07243 [cs.IR]
  18. Matthias Fey and Jan Eric Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. arXiv:1903.02428 [cs.LG]
  19. Victor Garcia and Joan Bruna. 2018. Few-Shot Learning with Graph Neural Networks. arXiv:1711.04043 [stat.ML]
  20. Neural Message Passing for Quantum Chemistry. arXiv:1704.01212 [cs.LG]
  21. Understanding convolution on graphs via energies. arXiv:2206.10991 [cs.LG]
  22. Regularization Theory and Neural Networks Architectures. Neural Computation 7, 2 (03 1995), 219–269. https://doi.org/10.1162/neco.1995.7.2.219 arXiv:https://direct.mit.edu/neco/article-pdf/7/2/219/812917/neco.1995.7.2.219.pdf
  23. Hamiltonian Neural Networks. arXiv:1906.01563 [cs.NE]
  24. Knowledge Base Completion with Out-of-Knowledge-Base Entities: A Graph Neural Network Approach. Transactions of the Japanese Society for Artificial Intelligence 33, 2 (2018), F–H72_1–10. https://doi.org/10.1527/tjsai.f-h72
  25. Inductive Representation Learning on Large Graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 1025–1035.
  26. Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs.CV]
  27. Relation Networks for Object Detection. arXiv:1711.11575 [cs.CV]
  28. Physics-informed machine learning. (05 2021), 1–19. https://doi.org/10.1038/s42254-021-00314-5
  29. Physics-informed machine learning: Case studies for weather and climate modelling. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 379 (02 2021), 20200093. https://doi.org/10.1098/rsta.2020.0093
  30. Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs.LG]
  31. Thomas N. Kipf and Max Welling. 2016. Variational Graph Auto-Encoders. arXiv:1611.07308 [stat.ML]
  32. Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. arXiv:1609.02907 [cs.LG]
  33. Datasets: A Community Library for Natural Language Processing. arXiv:2109.02846 [cs.CL]
  34. Gated Graph Sequence Neural Networks. arXiv:1511.05493 [cs.LG]
  35. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 3 (March 2021), 218–229. https://doi.org/10.1038/s42256-021-00302-5
  36. Revisiting Heterophily For Graph Neural Networks. arXiv:2210.07606 [cs.LG]
  37. Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks. arXiv:1804.08313 [cs.CL]
  38. When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning. arXiv:2203.16797 [cs.LG]
  39. Hoang NT and Takanori Maehara. 2019. Revisiting Graph Neural Networks: All We Have is Low-Pass Filters. arXiv:1905.09550 [stat.ML]
  40. Kenta Oono and Taiji Suzuki. 2021. Graph Neural Networks Exponentially Lose Expressive Power for Node Classification. arXiv:1905.10947 [cs.LG]
  41. Adversarially Regularized Graph Autoencoder for Graph Embedding. arXiv:1802.04407 [cs.LG]
  42. Netprobe: A Fast and Scalable System for Fraud Detection in Online Auction Networks. In Proceedings of the 16th International Conference on World Wide Web (Banff, Alberta, Canada) (WWW ’07). Association for Computing Machinery, New York, NY, USA, 201–210. https://doi.org/10.1145/1242572.1242600
  43. Geom-GCN: Geometric Graph Convolutional Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=S1e2agrFvS
  44. Large-Scale Hierarchical Text Classification with Recursively Regularized Deep Graph-CNN. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1063–1072. https://doi.org/10.1145/3178876.3186005
  45. Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyond. arXiv:2209.06177 [cs.SI]
  46. A critical look at the evaluation of GNNs under heterophily: are we really making progress? arXiv:2302.11640 [cs.LG]
  47. A Survey on Oversmoothing in Graph Neural Networks. arXiv:2303.10993 [cs.LG]
  48. Exploring Graph-structured Passage Representation for Multi-hop Reading Comprehension with Graph Neural Networks. arXiv:1809.02040 [cs.CL]
  49. Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. ACM. https://doi.org/10.1145/3447548.3467373
  50. Graph-Structured Representations for Visual Question Answering. arXiv:1609.05600 [cs.CV]
  51. Talip Ucar. 2023. NESS: Node Embeddings from Static SubGraphs. arXiv:2303.08958 [cs.LG]
  52. Graph Convolutional Matrix Completion. arXiv:1706.02263 [stat.ML]
  53. Petar Veličković. 2023. Everything is Connected: Graph Neural Networks. arXiv:2301.08210 [cs.LG]
  54. Graph Attention Networks. arXiv:1710.10903 [stat.ML]
  55. MGAE: Marginalized Graph Autoencoder for Graph Clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (Singapore, Singapore) (CIKM ’17). Association for Computing Machinery, New York, NY, USA, 889–898. https://doi.org/10.1145/3132847.3132967
  56. Zero-shot Recognition via Semantic Embeddings and Knowledge Graphs. arXiv:1803.08035 [cs.CV]
  57. Xiyuan Wang and Muhan Zhang. 2022. How Powerful are Spectral Graph Neural Networks. arXiv:2205.11172 [cs.LG]
  58. Frank. Wilcoxon. 1945. Individual Comparisons by Ranking Methods. Biometrics 1 (1945), 196–202. https://api.semanticscholar.org/CorpusID:53662922
  59. Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems. In The World Wide Web Conference. ACM. https://doi.org/10.1145/3308558.3313442
  60. Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network. arXiv:1905.11605 [cs.LG]
  61. Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks. arXiv:2102.06462 [cs.LG]
  62. Graph Convolutional Networks for Text Classification. arXiv:1809.05679 [cs.CL]
  63. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM. https://doi.org/10.1145/3219819.3219890
  64. Muhan Zhang. 2022. Graph Neural Networks: Link Prediction. In Graph Neural Networks: Foundations, Frontiers, and Applications, Lingfei Wu, Peng Cui, Jian Pei, and Liang Zhao (Eds.). Springer Singapore, Singapore, 195–223.
  65. Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. arXiv:1802.09691 [cs.LG]
  66. Dirichlet Energy Constrained Learning for Deep Graph Neural Networks. arXiv:2107.02392 [cs.LG]
  67. Link Prediction on Heterophilic Graphs via Disentangled Representation Learning. arXiv:2208.01820 [cs.LG]
  68. Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs. arXiv:2006.11468 [cs.LG]

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets