Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Long-range Meta-path Search on Large-scale Heterogeneous Graphs (2307.08430v5)

Published 17 Jul 2023 in cs.AI

Abstract: Utilizing long-range dependency, a concept extensively studied in homogeneous graphs, remains underexplored in heterogeneous graphs, especially on large ones, posing two significant challenges: Reducing computational costs while maximizing effective information utilization in the presence of heterogeneity, and overcoming the over-smoothing issue in graph neural networks. To address this gap, we investigate the importance of different meta-paths and introduce an automatic framework for utilizing long-range dependency on heterogeneous graphs, denoted as Long-range Meta-path Search through Progressive Sampling (LMSPS). Specifically, we develop a search space with all meta-paths related to the target node type. By employing a progressive sampling algorithm, LMSPS dynamically shrinks the search space with hop-independent time complexity. Through a sampling evaluation strategy, LMSPS conducts a specialized and effective meta-path selection, leading to retraining with only effective meta-paths, thus mitigating costs and over-smoothing. Extensive experiments across diverse heterogeneous datasets validate LMSPS's capability in discovering effective long-range meta-paths, surpassing state-of-the-art methods. Our code is available at https://github.com/JHL-HUST/LMSPS.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. On the bottleneck of graph neural networks and its practical implications. In 9th International Conference on Learning Representations, ICLR, 2021.
  2. Hierarchical Optimization: An Introduction. Annals of Operations Research, pp.  1–11, 1992.
  3. Spectral clustering with graph neural networks for graph pooling. In International conference on machine learning, ICML, pp.  874–883. PMLR, 2020.
  4. MEGNN: Meta-path Extracted Graph Neural Network for Heterogeneous Graph Representation Learning. Knowledge-Based Systems, pp.  107611, 2022.
  5. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. In KDD ’19: In Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  257–266, 2019.
  6. An Overview of Bilevel Optimization. Annals of Operations Research, pp.  235–256, 2007.
  7. DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks. In KDD ’21: In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  279–288, 2021.
  8. Searching for a robust neural architecture in four gpu hours. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ICCV, pp.  1761–1770, 2019.
  9. Erridge, P. The pareto principle. British Dental Journal, 201(7):419–419, 2006.
  10. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In Proceedings of the ACM Web Conference, WWW, pp.  2331–2341, 2020.
  11. Understanding the Difficulty of Training Deep Feedforward Neural Networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS, pp.  249–256, 2010.
  12. Single Path One-Shot Neural Architecture Search with Uniform Sampling. In European Conference on Computer Vision, ECCV, pp.  544–560, 2020.
  13. Inductive representation learning on large graphs. Advances in neural information processing systems, NeurIPS, 30, 2017.
  14. Genetic Meta-Structure Search for Recommendation on Heterogeneous Information Network. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, pp.  455–464, 2020.
  15. A generalization of vit/mlp-mixer to graphs. In International conference on machine learning, ICML, pp.  12724–12745. PMLR, 2023.
  16. An Attention-based Graph Neural Network for Heterogeneous Structural Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, pp.  4132–4139, 2020.
  17. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, NeurIPS, 33:22118–22133, 2020a.
  18. OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
  19. Heterogeneous Graph Transformer. In Proceedings of the ACM Web Conference, WWW, pp.  2704–2710, 2020b.
  20. Heterogeneous Graph Propagation Network. IEEE Transactions on Knowledge and Data Engineering, pp.  521–532, 2021.
  21. Keriven, N. Not too little, not too much: a theoretical analysis of graph (over) smoothing. Advances in Neural Information Processing Systems, NeurIPS, 35:2268–2281, 2022.
  22. Adam: A Method for Stochastic Optimization. In Bengio, Y. and LeCun, Y. (eds.), International Conference on Learning Representations, ICLR, 2015.
  23. Differentiable Meta Multigraph Search with Partial Message Propagation on Heterogeneous Information Networks. Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 2023.
  24. Deepgcns: Can gcns go as deep as cnns? In Proceedings of the IEEE/CVF international conference on computer vision, ICCV, pp.  9267–9276, 2019.
  25. Higher-order Attribute-enhancing Heterogeneous Graph Neural Networks. IEEE Transactions on Knowledge and Data Engineering, pp.  560–574, 2021a.
  26. Deeper Insights Into Graph Convolutional Networks for Semi-supervised Learning. In Proceedings of the AAAI conference on artificial intelligence, AAAI, 2018.
  27. GraphMSE: Efficient Meta-path Selection in Semantically Aligned Feature Space for Graph Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, pp.  4206–4214, 2021b.
  28. DARTS: differentiable architecture search. In 7th International Conference on Learning Representations, ICLR, 2019a.
  29. DARTS: Architecture Search. In 7th International Conference on Learning Representations, ICLR, 2019b.
  30. Are We Really Making Much Progress? Revisiting, Benchmarking and Refining Heterogeneous Graph Neural Networks. In KDD ’21: In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  1150–1160, 2021.
  31. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712, 2016.
  32. Hinormer: Representation learning on heterogeneous information networks with graph transformer. In Proceedings of the ACM Web Conference, WWW, pp.  599–610, 2023.
  33. Automatic meta-path discovery for effective graph-based recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp.  1563–1572, 2022.
  34. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, NeurIPS, 32, 2019.
  35. Large-scale evolution of image classifiers. In International conference on machine learning, ICML, pp.  2902–2911, 2017.
  36. Sanders, R. The pareto principle: its use and abuse. Journal of Services Marketing, 1(2):37–40, 1987.
  37. Modeling Relational Data with Graph Convolutional Networks. In European semantic web conference, pp.  593–607, 2018.
  38. Masked Label Prediction: Unified Message Passing Model for Semi-supervised Classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI, pp.  1548–1554, 2021.
  39. Scalable and Adaptive Graph Neural Networks with Self-label-enhanced Training. arXiv preprint arXiv:2104.09376, 2021.
  40. Multi-stage Self-supervised Learning for Graph Convolutional Networks on Graphs with Few Labeled Nodes. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, pp.  5892–5899, 2020.
  41. Mining Heterogeneous Information Networks: A Structural Analysis Approach. SIGKDD Explor., pp.  20–28, 2012.
  42. PathSim: Meta Path-Based Top-K Similarity Search in Heterogeneous Information Networks. Proc. VLDB Endow., pp.  992–1003, 2011.
  43. Mlp-mixer: An all-mlp architecture for vision. Advances in Neural Information Processing Systems, NeurIPS, 34:24261–24272, 2021.
  44. Unifying Graph Convolutional Neural Networks and Label Propagation. arXiv preprint arXiv:2002.06755, 2020.
  45. Rethinking architecture selection in differentiable nas. In International Conference on Learning Representation, ICLR, 2021.
  46. Heterogeneous Graph Attention Network. In Proceedings of the ACM Web Conference, WWW, pp.  2022–2032, 2019.
  47. Dynamixer: a vision mlp architecture with dynamic mixing. In International conference on machine learning, ICML, pp.  22691–22701. PMLR, 2022.
  48. Genetic CNN. In IEEE International Conference on Computer Vision, ICCV, pp.  1388–1397, 2017.
  49. SNAS: Stochastic Neural Architecture Search. In 7th International Conference on Learning Representations, ICLR, 2019.
  50. Rethinking Bi-Level Optimization in Neural Architecture Search: A Gibbs Sampling Perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, pp.  10551–10559, 2021.
  51. Simple and Efficient Heterogeneous Graph Neural Network. Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 2023.
  52. Efficient Neural Architecture Search via Proximal Iterations. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 2020.
  53. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, NeurIPS, 34:28877–28888, 2021.
  54. Scalable Graph Neural Networks for Heterogeneous Graphs. arXiv preprint arXiv:2011.09679, 2020.
  55. Graph Transformer Networks. In Advances in Neural Information Processing Systems, NeurIPS, pp.  11960–11970, 2019.
  56. Heterogeneous Graph Neural Network. In KDD ’19: In Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  793–803, 2019.
  57. Graph Attention Multi-layer Perceptron. In KDD ’22: In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  4560–4570, 2022.
  58. Reinforcement learning enhanced heterogeneous graph neural network. arXiv preprint arXiv:2010.13735, 2020.
  59. Relation Structure-aware Heterogeneous Graph Neural Network. In 2019 IEEE International Conference on Data Mining, ICDM, pp.  1534–1539, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Chao Li (429 papers)
  2. Zijie Guo (4 papers)
  3. Qiuting He (2 papers)
  4. Hao Xu (351 papers)
  5. Kun He (177 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now