Papers
Topics
Authors
Recent
Search
2000 character limit reached

IRWE: Inductive Random Walk for Joint Inference of Identity and Position Network Embedding

Published 1 Jan 2024 in cs.SI | (2401.00651v3)

Abstract: Network embedding, which maps graphs to distributed representations, is a unified framework for various graph inference tasks. According to the topology properties (e.g., structural roles and community memberships of nodes) to be preserved, it can be categorized into the identity and position embedding. Most existing methods can only capture one type of property. Some approaches can support the inductive inference that generalizes the embedding model to new nodes or graphs but relies on the availability of attributes. Due to the complicated correlations between topology and attributes, it is unclear for some inductive methods which type of property they can capture. In this study, we explore a unified framework for the joint inductive inference of identity and position embeddings without attributes. An inductive random walk embedding (IRWE) method is proposed, which combines multiple attention units to handle the random walk (RW) on graph topology and simultaneously derives identity and position embeddings that are jointly optimized. We demonstrate that some RW statistics can characterize node identities and positions while supporting the inductive inference. Experiments validate the superior performance of IRWE over various baselines for the transductive and inductive inference of identity and position embeddings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. An algorithmic theory of learning: Robust concepts and random projection. Machine learning, 63:161–182, 2006.
  2. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International on Conference on Information & Knowledge Management, pp.  891–900, 2015.
  3. Structure-aware transformer for graph representation learning. In Proceedings of the 2022 International Conference on Machine Learning, pp.  3469–3489, 2022.
  4. Csgcl: Community-strength-enhanced graph contrastive learning. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence, pp.  2059–2067, 2023.
  5. Learning structural node embeddings via diffusion wavelets. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  1320–1329, 2018.
  6. Raftgp: Random fast graph partitioning. In 2023 IEEE High Performance Extreme Computing Conference, pp.  1–7, 2023.
  7. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  855–864, 2016.
  8. Spine: Structural identity preserved inductive network embedding. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp.  2399–2405, 2019.
  9. Role-oriented graph auto-encoder guided by structural information. In Proceedings of the 25th International Conference on Database Systems for Advanced Applications, pp.  466–481, 2020.
  10. Inductive representation learning on large graphs. In Proceedings of the 2017 Advances in Neural Information Processing Systems, pp.  1024–1034, 2017.
  11. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.  594–604, 2022.
  12. Graphmae2: A decoding-enhanced masked self-supervised graph learner. In Proceedings of the 2023 ACM Web Conference, pp.  737–746, 2023.
  13. Anonymous walk embeddings. In Proceedings of the 2018 International Conference on Machine Learning, pp.  2186–2195, 2018.
  14. Gralsp: Graph neural networks with local structural patterns. In Proceedings of the 2020 AAAI Conference on Artificial Intelligence, pp.  4361–4368, 2020.
  15. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations, 2017.
  16. Adaptive multiple non-negative matrix factorization for temporal link prediction in dynamic networks. In Proceedings of the 2018 ACM SIGCOMM Workshop on Network Meets AI & ML, pp.  28–34, 2018.
  17. Gcn-gan: A non-linear temporal link prediction model for weighted dynamic networks. In Proceedings of the 2019 IEEE Conference on Computer Communications, pp.  388–396, 2019.
  18. Distance encoding: Design provably more powerful neural networks for graph representation learning. Proceedings of the 2020 Advances in Neural Information Processing Systems, 33:4465–4478, 2020.
  19. Identifying interpretable link communities with user interactions and messages in social networks. In Proceedings of the 2019 IEEE International Conference on Parallel & Distributed Processing with Applications, pp.  271–278, 2019.
  20. Rsc: accelerate graph neural networks training via randomized sparse computations. In International Conference on Machine Learning, pp.  21951–21968, 2023.
  21. Reconstructing markov processes from independent and anonymous experiments. Discrete Applied Mathematics, 200:108–122, 2016.
  22. Mark EJ Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577–8582, 2006.
  23. A self-attention network based node embedding model. In Proceedings of the 2021 Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.  364–377, 2021.
  24. Universal graph transformer self-attention networks. In Companion Proceedings of the Web Conference 2022, pp.  193–196, 2022.
  25. struc2gauss: Structural role preserving network embedding via gaussian embedding. Data Mining & Knowledge Discovery, 34(4):1072–1103, 2020.
  26. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery & data Mining, pp.  701–710, 2014.
  27. Dual-channel hybrid community detection in attributed networks. Information Sciences, 551:146–167, 2021.
  28. Temporal link prediction: A unified framework, taxonomy, and review. ACM Computing Surveys, 56(4):1–40, 2023.
  29. Adaptive community detection incorporating topology and content in social networks. Knowledge-Based Systems, 161:342–356, 2018.
  30. Towards a better trade-off between quality and efficiency of community detection: An inductive embedding method across graphs. ACM Transactions on Knowledge Discovery from Data, 2023a.
  31. High-quality temporal link prediction for weighted dynamic graphs via inductive embedding aggregation. IEEE Transactions on Knowledge and Data Engineering, 2023b.
  32. struc2vec: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  385–394, 2017.
  33. On proximity and structural role-based embeddings in networks: Misconceptions, techniques, and applications. ACM Transactions on Knowledge Discovery from Data, 14(5):1–37, 2020.
  34. On the equivalence between positional node embeddings and structural graph representations. In Proceedings of the 8th International Conference on Learning Representations, 2020.
  35. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pp.  1067–1077, 2015.
  36. Attention is all you need. In Proceedings of the 2017 Advances in Neural Information Processing Systems, pp.  5998–6008, 2017.
  37. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations, 2018.
  38. Deep graph infomax. In Proceedings of the 7th International Conference on Learning Representations, 2019.
  39. Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics & Computing, 17(4):395–416, 2007.
  40. Community preserving network embedding. In Proceedings of the 2017 AAAI Conference on Artificial Iintelligence, pp.  203–209, 2017.
  41. Am-gcn: Adaptive multi-channel graph convolutional networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  1243–1253, 2020.
  42. Demo-net: Degree-specific graph neural networks for node and graph classification. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.  406–415, 2019.
  43. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks & Learning Systems, 32(1):4–24, 2020.
  44. How powerful are graph neural networks? In Proceedings of the 7th International Conference on Learning Representations, 2019.
  45. RAIN: social role-aware information diffusion. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, pp.  367–373, 2015.
  46. Community preserving mapping for network hyperbolic embedding. Knowledge-Based Systems, 246:108699, 2022.
  47. Position-aware graph neural networks. In Proceedings of th 2019 International Conference on Machine Learning, pp.  7134–7143, 2019.
  48. Identity-aware graph neural networks. In Proceedings of the 2012 AAAI Conference on Artificial Intelligence, pp.  10737–10745, 2021.
  49. Pasca: A graph neural architecture search system under the scalable paradigm. In Proceedings of the 2022 ACM Web Conference, pp.  1817–1828, 2022.
  50. Node proximity is all you need: Unified structural and positional node and graph embedding. In Proceedings of the 2021 SIAM International Conference on Data Mining, pp.  163–171, 2021.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Meng Qin 
  2. Dit-Yan Yeung 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up