Papers
Topics
Authors
Recent
Search
2000 character limit reached

You Only Transfer What You Share: Intersection-Induced Graph Transfer Learning for Link Prediction

Published 27 Feb 2023 in cs.LG, cs.AI, and cs.SI | (2302.14189v2)

Abstract: Link prediction is central to many real-world applications, but its performance may be hampered when the graph of interest is sparse. To alleviate issues caused by sparsity, we investigate a previously overlooked phenomenon: in many cases, a densely connected, complementary graph can be found for the original graph. The denser graph may share nodes with the original graph, which offers a natural bridge for transferring selective, meaningful knowledge. We identify this setting as Graph Intersection-induced Transfer Learning (GITL), which is motivated by practical applications in e-commerce or academic co-authorship predictions. We develop a framework to effectively leverage the structural prior in this setting. We first create an intersection subgraph using the shared nodes between the two graphs, then transfer knowledge from the source-enriched intersection subgraph to the full target graph. In the second step, we consider two approaches: a modified label propagation, and a multi-layer perceptron (MLP) model in a teacher-student regime. Experimental results on proprietary e-commerce datasets and open-source citation graphs show that the proposed workflow outperforms existing transfer learning baselines that do not explicitly utilize the intersection structure.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. Friends and neighbors on the web. Social networks, 25(3):211–230, 2003.
  2. A survey of link prediction in social networks. In Social network data analytics, pp.  243–275. Springer, 2011.
  3. Sparse local embeddings for extreme multi-label classification. In NIPS, volume 29, pp.  730–738, 2015.
  4. Meta-graph: Few shot link prediction via meta learning. arXiv preprint arXiv:1912.09867, 2019.
  5. Graph domain adaptation: A generative view. arXiv preprint arXiv:2106.07482, 2021.
  6. Link prediction approach to collaborative filtering. In Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’05), pp.  141–142. IEEE, 2005.
  7. Scalable graph neural networks via bidirectional propagation. arXiv preprint arXiv:2010.15421, 2020.
  8. Ssne: Effective node representation for link prediction in sparse networks. IEEE Access, 9:57874–57885, 2021a.
  9. Bag of tricks for training deeper graph neural networks: A comprehensive benchmark study. arXiv preprint arXiv:2108.10521, 2021b.
  10. Decoupled smoothing on graphs. In The World Wide Web Conference, pp.  263–272, 2019.
  11. Gobinda G Chowdhury. Introduction to modern information retrieval. Facet publishing, 2010.
  12. A comprehensive study on large-scale graph training: Benchmarking and rethinking. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  13. Label propagation via teaching-to-learn and learning-to-teach. IEEE transactions on neural networks and learning systems, 28(6):1452–1465, 2016.
  14. Graph transfer learning. 2021.
  15. Adaptive transfer learning on graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.  565–574, 2021.
  16. BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages. In Nicoletta Calzolari (Conference chair), Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Koiti Hasida, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis, and Takenobu Tokunaga (eds.), Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 7-12, 2018 2018. European Language Resources Association (ELRA). ISBN 979-10-95546-00-9.
  17. Strategies for pre-training graph neural networks. In International Conference on Learning Representations (ICLR), 2020a.
  18. Graph-mlp: Node classification without message passing in graph. arXiv preprint arXiv:2106.04051, 2021.
  19. Gpt-gnn: Generative pre-training of graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  1857–1867, 2020b.
  20. Combining label propagation and simple models out-performs graph neural networks. arXiv preprint arXiv:2010.13993, 2020.
  21. Self-supervised auxiliary learning with meta-paths for heterogeneous graphs. Advances in Neural Information Processing Systems, 33:10294–10305, 2020.
  22. Social recommendation with cross-domain transferable knowledge. IEEE transactions on knowledge and data engineering, 27(11):3084–3097, 2015.
  23. Thorsten Joachims. Transductive learning via spectral graph partitioning. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp.  290–297, 2003.
  24. Zero-shot scene graph relation prediction through commonsense knowledge integration. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  466–482. Springer, 2021.
  25. Manifold-based similarity adaptation for label propagation. Advances in neural information processing systems, 26:1547–1555, 2013.
  26. Simple embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp.  4289–4300, 2018.
  27. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.
  28. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885–1894. PMLR, 2017.
  29. Unifying guilt-by-association approaches: Theorems and fast algorithms. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  245–260. Springer, 2011.
  30. Node classification on graphs with few-shot novel labels via meta transformed network embedding. Advances in Neural Information Processing Systems, 33, 2020.
  31. Cold start link prediction. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.  393–402, 2010.
  32. The link-prediction problem for social networks. Journal of the American society for information science and technology, 58(7):1019–1031, 2007.
  33. New perspectives and methods in link prediction. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.  243–252, 2010.
  34. Deep learning for extreme multi-label text classification. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp.  115–124, 2017.
  35. Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002, 2018.
  36. Impact of centrality measures on the common neighbors in link prediction for multiplex networks. Big Data, 10(2):138–150, 2022.
  37. Mark EJ Newman. Clustering and preferential attachment in growing networks. Physical review E, 64(2):025102, 2001.
  38. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
  39. Graphon neural networks and the transferability of graph neural networks. Advances in Neural Information Processing Systems, 33, 2020.
  40. Reciprocity, community detection, and link prediction in dynamic networks. Journal of Physics: Complexity, 3(1):015010, 2022.
  41. Edge proposal sets for link prediction. arXiv preprint arXiv:2106.15810, 2021.
  42. Plums: predicting links using multiple sources. In Proceedings of the 2015 SIAM International Conference on Data Mining, pp.  370–378. SIAM, 2015.
  43. Scalable and adaptive graph neural networks with self-label-enhanced training. arXiv preprint arXiv:2104.09376, 2021.
  44. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2020a.
  45. A benchmarking study of embedding-based entity alignment for knowledge graphs. arXiv preprint arXiv:2003.07743, 2020b.
  46. Deep graph infomax. ICLR (Poster), 2(3):4, 2019.
  47. Deep graph infomax. In International Conference on Learning Representations, 2019.
  48. Label propagation through linear neighborhoods. IEEE Transactions on Knowledge and Data Engineering, 20(1):55–67, 2007.
  49. Unifying graph convolutional neural networks and label propagation. arXiv preprint arXiv:2002.06755, 2020.
  50. Data dropout: Optimizing training data for convolutional neural networks. In 2018 IEEE 30th international conference on tools with artificial intelligence (ICTAI), pp.  39–46. IEEE, 2018.
  51. Instance-based deep transfer learning. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.  367–375. IEEE, 2019.
  52. Pairwise learning for neural link prediction. arXiv preprint arXiv:2112.02936, 2021.
  53. Simplifying graph convolutional networks. In International conference on machine learning, pp. 6861–6871. PMLR, 2019.
  54. Unsupervised domain adaptive graph convolutional networks. In Proceedings of The Web Conference 2020, pp.  1457–1467, 2020.
  55. Few-shot link prediction in dynamic networks. The 15th ACM International Conference on Web Search and Data Mining (WSDM), 2022a.
  56. Few-shot link prediction in dynamic networks. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp.  1245–1255, 2022b.
  57. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812–5823, 2020a.
  58. When does self-supervision help graph convolutional networks? In International Conference on Machine Learning, pp. 10871–10880. PMLR, 2020b.
  59. Neo-gnns: Neighborhood overlap-aware graph neural networks for link prediction. Advances in Neural Information Processing Systems, 34:13683–13694, 2021.
  60. Link prediction based on graph neural networks. Advances in Neural Information Processing Systems, 31:5165–5175, 2018.
  61. Labeling trick: A theory of using graph neural networks for multi-node representation learning. Advances in Neural Information Processing Systems, 34, 2021a.
  62. A comprehensive survey on knowledge graph entity alignment via representation learning. arXiv preprint arXiv:2103.15059, 2021b.
  63. Graph-less neural networks: Teaching old mlps new tricks via distillation. arXiv preprint arXiv:2110.08727, 2021c.
  64. Cold brew: Distilling graph node representations with incomplete or missing neighborhoods. arXiv preprint arXiv:2111.04840, 2021.
  65. Learning with local and global consistency. In Advances in neural information processing systems, pp. 321–328, 2004.
  66. Predicting missing links via local information. The European Physical Journal B, 71(4):623–630, 2009.
  67. Collective multi-type entity alignment between knowledge graphs. In Proceedings of The Web Conference 2020, pp.  2241–2252, 2020.
  68. Transfer learning of graph neural networks with ego-graph information maximization. Advances in Neural Information Processing Systems, 34:1766–1779, 2021a.
  69. Xiaojin Zhu. Semi-supervised learning with graphs. Carnegie Mellon University, 2005.
  70. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pp.  912–919, 2003.
  71. Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems, 34:29476–29490, 2021b.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up