Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks (2304.11574v2)

Published 23 Apr 2023 in cs.AI

Abstract: Meta-structures are widely used to define which subset of neighbors to aggregate information in heterogeneous information networks (HINs). In this work, we investigate existing meta-structures, including meta-path and meta-graph, and observe that they are initially designed manually with fixed patterns and hence are insufficient to encode various rich semantic information on diverse HINs. Through reflection on their limitation, we define a new concept called meta-multigraph as a more expressive and flexible generalization of meta-graph, and propose a stable differentiable search method to automatically optimize the meta-multigraph for specific HINs and tasks. As the flexibility of meta-multigraphs may propagate redundant messages, we further introduce a complex-to-concise (C2C) meta-multigraph that propagates messages from complex to concise along the depth of meta-multigraph. Moreover, we observe that the differentiable search typically suffers from unstable search and a significant gap between the meta-structures in search and evaluation. To this end, we propose a progressive search algorithm by implicitly narrowing the search space to improve search stability and reduce inconsistency. Extensive experiments are conducted on six medium-scale benchmark datasets and one large-scale benchmark dataset over two representative tasks, i.e., node classification and recommendation. Empirical results demonstrate that our search methods can automatically find expressive meta-multigraphs and C2C meta-multigraphs, enabling our model to outperform state-of-the-art heterogeneous graph neural networks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Heterogeneous graph attention network. In The World Wide Web Conference, WWW, pages 2022–2032, 2019.
  2. Genetic meta-structure search for recommendation on heterogeneous information network. In The 29th ACM International Conference on Information and Knowledge Management, CIKM, pages 455–464, 2020.
  3. Heterogeneous graph neural network with distance encoding. In 2021 IEEE International Conference on Data Mining, ICDM, pages 1138–1143. IEEE, 2021.
  4. MEGNN: Meta-path extracted graph neural network for heterogeneous graph representation learning. Knowl. Based Syst., 235:107611, 2022.
  5. Heterogeneous information network embedding based on multiperspective metapath for question routing. Knowl. Based Syst., 240:107842, 2022.
  6. Attentive meta-graph embedding for item recommendation in heterogeneous information networks. Knowl. Based Syst., 211:106524, 2021.
  7. PathSim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the Vldb Endowment, 4(11):992–1003, 2011.
  8. Semantic proximity search on graphs with metagraph-based learning. In ICDE, pages 277–288, 2016.
  9. Meta Structure: Computing relevance in large heterogeneous information networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1595–1604, 2016.
  10. Meta-graph based recommendation fusion over heterogeneous information networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 635–644, 2017.
  11. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In The World Wide Web Conference, WWW, pages 2331–2341, 2020.
  12. Graph transformer networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems, NeurIPS, pages 11960–11970, 2019.
  13. Heterogeneous graph transformer. In The World Wide Web Conference, WWW, pages 2704–2710, 2020.
  14. Neural architecture search with reinforcement learning. In 5th International Conference on Learning Representations, ICLR, Toulon, France, Conference Track Proceedings, 2017.
  15. Efficient neural architecture search via parameters sharing. In ICML, pages 4095–4104, 2018.
  16. DARTS: Architecture search. In 7th International Conference on Learning Representations, ICLR, New Orleans, LA, USA, 2019.
  17. DiffMG: Differentiable meta graph search for heterogeneous graph neural networks. In KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, pages 279–288, 2021.
  18. Heterogeneous graph neural architecture search. In 2021 IEEE International Conference on Data Mining, ICDM, pages 1066–1071, 2021.
  19. Differentiable meta multigraph search with partial message propagation on heterogeneous information networks. Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, 2023.
  20. Ista-nas: Efficient and consistent neural architecture search by sparse coding. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems, NeurIPS, 33:10503–10513, 2020.
  21. Towards improving the consistency, efficiency, and flexibility of differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pages 6667–6676, 2021.
  22. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. PVLDB, 4(11):992–1003, 2011.
  23. Auto-GNN: Neural architecture search of graph neural networks. arXiv preprint arXiv:1909.03184, 2019.
  24. Graph differentiable architecture search with structure learning. Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, NeurIPS, 34:16860–16872, 2021.
  25. Search to aggregate neighborhood for graph neural network. In ICDE, 2021.
  26. Designing the topology of graph neural networks: A novel feature fusion perspective. In Proceedings of the ACM Web Conference, pages 1381–1391, 2022.
  27. Pasca: A graph neural architecture search system under the scalable paradigm. In Proceedings of the ACM Web Conference, pages 1817–1828, 2022.
  28. G Anandalingam and Terry L Friesz. Hierarchical optimization: An introduction. Annals of Operations Research, 34(1):1–11, 1992.
  29. An overview of bilevel optimization. Annals of operations research, 153(1):235–256, 2007.
  30. Rethinking bi-level optimization in neural architecture search: A gibbs sampling perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, number 12, pages 10551–10559, 2021.
  31. Single path one-shot neural architecture search with uniform sampling. In European Conference on Computer Vision, ECCV, pages 544–560, 2020.
  32. DARTS: Differentiable architecture search. In 7th International Conference on Learning Representations, ICLR, 2019.
  33. DATA: differentiable architecture approximation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems, NeurIPS, pages 874–884, 2019.
  34. Efficient neural interaction function search for collaborative filtering. In The World Wide Web Conference, WWW, pages 1660–1670, 2020.
  35. Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In IEEE/CVF International Conference on Computer Vision, ICCV, Seoul, Korea (South), pages 1294–1303, 2019.
  36. PC-DARTS: Partial channel connections for memory-efficient architecture search. In 8th International Conference on Learning Representations, ICLR, 2020.
  37. Efficient neural architecture search via proximal iterations. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI, volume 34, pages 6664–6671, 2020.
  38. Proxylessnas: Direct neural architecture search on target task and hardware. In 7th International Conference on Learning Representations, ICLR, 2019.
  39. Heterogeneous network representation learning: A unified framework with survey and benchmark. IEEE Transactions on Knowledge and Data Engineering, 2020.
  40. Ogb-lsc: A large-scale challenge for machine learning on graphs. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021.
  41. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR, Conference Track Proceedings, 2017.
  42. Graph attention networks. In 6th International Conference on Learning Representations, ICLR, Conference Track Proceedings, 2018.
  43. Graph attention multi-layer perceptron. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, page 4560–4570, 2022.
  44. Modeling relational data with graph convolutional networks. In ESWC, 2018.
  45. Heterogeneous graph neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD, Anchorage, AK, USA, pages 793–803, 2019.
  46. Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1150–1160, 2021.
  47. Scalable graph neural networks for heterogeneous graphs. arXiv preprint arXiv:2011.09679, 2020.
  48. Scalable and adaptive graph neural networks with self-label-enhanced training. arXiv preprint arXiv:2104.09376, 2021.
  49. Maurice G Kendall. A new measure of rank correlation. Biometrika, 30(1/2):81–93, 1938.
  50. Laurens Van Der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2605):2579–2605, 2008.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now