Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Scalable Structural Representations for Link Prediction with Bloom Signatures (2312.16784v1)

Published 28 Dec 2023 in cs.LG and cs.SI

Abstract: Graph neural networks (GNNs) have shown great potential in learning on graphs, but they are known to perform sub-optimally on link prediction tasks. Existing GNNs are primarily designed to learn node-wise representations and usually fail to capture pairwise relations between target nodes, which proves to be crucial for link prediction. Recent works resort to learning more expressive edge-wise representations by enhancing vanilla GNNs with structural features such as labeling tricks and link prediction heuristics, but they suffer from high computational overhead and limited scalability. To tackle this issue, we propose to learn structural link representations by augmenting the message-passing framework of GNNs with Bloom signatures. Bloom signatures are hashing-based compact encodings of node neighborhoods, which can be efficiently merged to recover various types of edge-wise structural features. We further show that any type of neighborhood overlap-based heuristic can be estimated by a neural network that takes Bloom signatures as input. GNNs with Bloom signatures are provably more expressive than vanilla GNNs and also more scalable than existing edge-wise models. Experimental results on five standard link prediction benchmarks show that our proposed model achieves comparable or better performance than existing edge-wise GNN models while being 3-200 $\times$ faster and more memory-efficient for online inference.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Lada A Adamic and Eytan Adar. 2003. Friends and neighbors on the web. Social networks 25, 3 (2003), 211–230.
  2. Albert-László Barabási and Réka Albert. 1999. Emergence of scaling in random networks. Science 286, 5439 (1999), 509–512.
  3. The netflix prize. In Proceedings of KDD Cup and Workshop, Vol. 2007. New York, 35.
  4. Burton H. Bloom. 1970. Space/Time Trade-Offs in Hash Coding with Allowable Errors. Commun. ACM 13, 7 (Jul 1970), 422–426.
  5. Andrei Broder and Michael Mitzenmacher. 2004. Network applications of bloom filters: A survey. Internet mathematics 1, 4 (2004), 485–509.
  6. Graph Neural Networks for Link Prediction with Subgraph Sketching. In International Conference on Learning Representations.
  7. Can graph neural networks count substructures? Advances in Neural Information Processing Systems 33 (2020), 10383–10395.
  8. P ERDdS and A R&wi. 1959. On random graphs I. Publ. math. debrecen 6, 290-297 (1959), 18.
  9. Neural message passing for quantum chemistry. In International Conference on Machine Learning. PMLR, 1263–1272.
  10. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.
  11. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems 30 (2017), 1025–1035.
  12. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems 33 (2020), 22118–22133.
  13. Paul Jaccard. 1901. Distribution de la flore alpine dans le bassin des Dranses et dans quelques régions voisines. Bull Soc Vaudoise Sci Nat 37 (1901), 241–272.
  14. Highly accurate protein structure prediction with AlphaFold. Nature 596, 7873 (2021), 583–589.
  15. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.
  16. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
  17. Geodesic Graph Neural Network for Efficient Graph Representation Learning. Advances in Neural Information Processing Systems 35 (2022), 5896–5909.
  18. Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  19. Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning. Advances in Neural Information Processing Systems 33 (2020), 4465–4478.
  20. David Liben-Nowell and Jon Kleinberg. 2003. The link prediction problem for social networks. In Proceedings of the 12th International Conference on Information and Knowledge Management. 556–559.
  21. Whole Brain Vessel Graphs: A Dataset and Benchmark for Graph Learning and Neuroscience. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  22. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701–710.
  23. Efficient sketching algorithm for sparse binary data. In IEEE International Conference on Data Mining. IEEE, 508–517.
  24. Network embedding as matrix factorization: Unifying deepwalk, line, pte, and node2vec. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 459–467.
  25. The deletable Bloom filter: a new member of the Bloom family. IEEE Communications Letters 14, 6 (2010), 557–559.
  26. Balasubramaniam Srinivasan and Bruno Ribeiro. 2020. On the equivalence between positional node embeddings and structural graph representations. In International Conference on Learning Representations.
  27. Drug response prediction as a link prediction problem. Scientific reports 7, 1 (2017), 40321.
  28. STRING v11: protein–protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic acids research 47, D1 (2019), D607–D613.
  29. Inductive relation prediction by subgraph reasoning. In International Conference on Machine Learning. PMLR, 9448–9457.
  30. Graph attention networks. In International Conference on Learning Representations.
  31. Improving Graph Neural Networks on Multi-node Tasks with Labeling Tricks. arXiv preprint arXiv:2304.10074 (2023).
  32. Neural Common Neighbor with Completion for Link Prediction. arXiv preprint arXiv:2302.00890 (2023).
  33. How Powerful are Graph Neural Networks?. In International Conference on Learning Representations.
  34. SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning. Proceedings of the VLDB Endowment 16, 11 (2023), 2939–2948.
  35. Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning. Proceedings of the VLDB Endowment 15, 11 (2022), 2788–2796.
  36. Identity-aware graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 10737–10745.
  37. Neo-gnns: Neighborhood overlap-aware graph neural networks for link prediction. Advances in Neural Information Processing Systems 34 (2021), 13683–13694.
  38. Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. Advances in Neural Information Processing Systems 31 (2018), 5165–5175.
  39. Muhan Zhang and Yixin Chen. 2020. Inductive Matrix Completion Based on Graph Neural Networks. In International Conference on Learning Representations.
  40. Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning. Advances in Neural Information Processing Systems 34 (2021), 9061–9073.
  41. Predicting missing links via local information. The European Physical Journal B 71 (2009), 623–630.
  42. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems 34 (2021), 29476–29490.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets