Papers
Topics
Authors
Recent
Search
2000 character limit reached

LSEnet: Lorentz Structural Entropy Neural Network for Deep Graph Clustering

Published 20 May 2024 in cs.LG | (2405.11801v1)

Abstract: Graph clustering is a fundamental problem in machine learning. Deep learning methods achieve the state-of-the-art results in recent years, but they still cannot work without predefined cluster numbers. Such limitation motivates us to pose a more challenging problem of graph clustering with unknown cluster number. We propose to address this problem from a fresh perspective of graph information theory (i.e., structural information). In the literature, structural information has not yet been introduced to deep clustering, and its classic definition falls short of discrete formulation and modeling node features. In this work, we first formulate a differentiable structural information (DSI) in the continuous realm, accompanied by several theoretical results. By minimizing DSI, we construct the optimal partitioning tree where densely connected nodes in the graph tend to have the same assignment, revealing the cluster structure. DSI is also theoretically presented as a new graph clustering objective, not requiring the predefined cluster number. Furthermore, we design a neural LSEnet in the Lorentz model of hyperbolic space, where we integrate node features to structural information via manifold-valued graph convolution. Extensive empirical results on real graphs show the superiority of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Dynamic capacity networks. In Proceedings of the ICML, pp.  2549–2558. PMLR, 2016.
  2. Constant curvature graph convolutional networks. In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119, pp.  486–496. PMLR, 2020.
  3. Riemannian adaptive optimization methods. In Proceedings of 7th International Conference on Learning Representation (ICLR), 2019.
  4. Riemannian Adaptive Optimization Methods. In International Conference on Learning Representations, 2018.
  5. Bianconi, G. Entropy of network ensembles. Physical Review E, 79(3):036114, 2009.
  6. The laplacian of a graph as a density matrix: a basic combinatorial approach to separability of mixed states. Annals of Combinatorics, 10(3):291–317, 2006.
  7. Hyperbolic graph convolutional neural networks. In Advances in the 32nd NeurIPS, pp.  4869–4880, 2019.
  8. From trees to continuous embeddings and back: Hyperbolic hierarchical clustering. In Advances in NeurIPS, volume 33, pp.  15065–15076, 2020.
  9. Fully hyperbolic neural networks. In Proceedings of the 60th ACL, pp.  5672–5686, 2022.
  10. Hierarchical clustering: Objective functions and algorithms. Journal of the ACM, 66(4):26:1–26:42, 2019.
  11. S3GC: Scalable self-supervised graph clustering. In Advances in NeurIPS, pp.  3248–3261, 2022.
  12. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd KDD, pp.  226–231. AAAI Press, 1996.
  13. ACE-HGNN: adaptive curvature exploration hyperbolic graph neural network. In Proceedings of the ICDM, pp.  111–120. IEEE, 2021.
  14. Hyperbolic geometric graph representation learning for hierarchy-imbalance node classification. In Proceedings of the ACM Web Conference 2023, pp. 460–468. ACM, 2023.
  15. A Tutorial on Bayesian Nonparametric Models, 2011.
  16. Inductive representation learning on large graphs. In Advances in the 30th NeurIPS, pp.  1024–1034, 2017.
  17. Contrastive multi-view representation learning on graphs. In Proceedings of the 37th ICML, volume 119, pp.  4116–4126, 2020.
  18. Communitygan: Community detection with generative adversarial nets. In Proceedings of the WWW, pp.  784–794. ACM, 2019.
  19. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.
  20. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th ICLR, 2017.
  21. Hyperbolic geometry of complex networks. Physics Review E, 82(036106), 2010.
  22. Structural information and dynamical complexity of networks. IEEE Transactions on Information Theory, 62(6):3290–3339, 2016.
  23. Graph communal contrastive learning. In Proceedings of The ACM Web Conference, pp.  1203–1213. ACM, 2022.
  24. REM: from structural entropy to community structure deception. In Advances in the 32nd NeurIPS, pp.  12918–12928, 2019.
  25. Deep graph clustering via dual correlation reduction. In Proceedings of the 36th AAAI, pp.  7603–7611. AAAI Press, 2022a.
  26. Towards unsupervised deep graph structure learning. In Proceedings of the ACM Web Conference 2022, pp. 1392–1403, 2022b.
  27. Reinforcement graph clustering with unknown cluster number. In Proceedings of the 31st ACM MM, pp.  3528–3537. ACM, 2023a.
  28. Dink-net: Neural clustering on large graphs. In Proceedings of the ICML, volume 202, pp.  21794–21812. PMLR, 2023b.
  29. A Survey of Deep Graph Clustering: Taxonomy, Challenge, Application, and Open Resource, 2023c.
  30. Differentiating through the fréchet mean. In Proceedings of the 37th ICML, volume 119 of Proceedings of Machine Learning Research, pp.  6393–6403. PMLR, 2020.
  31. Escaping feature twist: A variational graph auto-encoder for node clustering. In Proceedings of the 31st IJCAI, pp.  3351–3357. ijcai.org, 2022.
  32. Poincaré embeddings for learning hierarchical representations. In Advances in 30th NeurIPS, pp.  6338–6347, 2017.
  33. Multi-view contrastive graph clustering. In Advances in NeurIPS, pp.  2148–2159, 2021.
  34. Adversarially regularized graph autoencoder for graph embedding. In Proceedings of the 27th IJCAI, pp.  2609–2615. ijcai.org, 2018.
  35. Deepwalk: Online learning of social representations. In Proceedings of the 20th KDD, pp.  701–710. ACM, 2014.
  36. Sarkar, R. Low distortion delaunay embedding of trees in hyperbolic plane. In Proceedings of the 19th International Symposium on Graph Drawing (Eindhoven, The Netherlands), volume 7034, pp.  355–366. Springer, 2011.
  37. Schubert, E. Stop using the elbow criterion for k-means and how to choose the number of clusters instead. SIGKDD Explor., 25(1):36–42, 2023.
  38. Shannon, C. E. A mathematical theory of communication. The Bell System Technical Journal, 27(3):379–423, 1948.
  39. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the 35th AAAI, pp.  4375–4383, 2021.
  40. A self-supervised riemannian GNN with time varying curvature for temporal graph learning. In Proceedings of the 31st ACM CIKM, pp.  1827–1836, 2022a.
  41. A self-supervised mixed-curvature graph neural network. In Proceedings of the 36th AAAI, pp.  4146–4155, 2022b.
  42. Deepricci: Self-supervised graph structure-feature co-refinement for alleviating over-squashing. In Proceedings of the 23rd ICDM, pp.  558–567, 2023a.
  43. CONGREGATE: contrastive graph clustering in curvature spaces. In Proceedings of the 32nd IJCAI, pp.  2296–2305. ijcai.org, 2023b.
  44. Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the 37th AAAI, pp.  4633–4642, 2023c.
  45. R-ode: Ricci curvature tells when you will be informed. In Proceedings of the ACM SIGIR, 2024a.
  46. Riccinet: Deep clustering via a riemannian generative model. In Proceedings of the ACM Web Conference 2024 (WWW’24), 2024b.
  47. Motif-aware riemannian graph neural network with generative-contrastive learning. In Proceedings of the 38th AAAI, pp.  9044–9052, 2024c.
  48. Graph attention networks. In Proceedings of the 6th ICLR, 2018.
  49. Gc-flow: A graph-based flow network for effective clustering. In Proceedings of the ICML, volume 202, pp.  36157–36173. PMLR, 2023.
  50. Structural entropy guided graph hierarchical pooling. In Proceedings of the 39th ICML, pp.  24017–24030. PMLR, 2022.
  51. SEGA: structural entropy guided anchor view for graph contrastive learning. In Proceedings of the 40th ICML, volume 202, pp. 37293–37312. PMLR, 2023.
  52. Progcl: Rethinking hard negative mining in graph contrastive learning. In Proceedings of the ICML, volume 162, pp.  24332–24346. PMLR, 2022.
  53. Unsupervised deep embedding for clustering analysis. In Proceedings of the ICML, pp.  478–487. PMLR, 2016.
  54. Pseudo-riemannian graph convolutional networks. In Advances in NeurIPS, 2022.
  55. Graph clustering with dynamic embedding. arXiv preprint arXiv:1712.08249, 2017.
  56. JANE: jointly adversarial network embedding. In Proceedings of the 29th IJCAI, pp.  1381–1387. ijcai.org, 2020.
  57. Hyperbolic representation learning: Revisiting and advancing. In Proceedings of the 40th ICML. PMLR, 2023a.
  58. Minimum entropy principle guided graph neural networks. In Proceedings of the 16th ACM WSDM, pp.  114–122, 2023b.
  59. Local higher-order graph clustering. In Proceedings of the 23rd ACM SIGKDD, pp.  555–564. ACM, 2017.
  60. Lorentzian graph convolutional networks. In Proceedings of the Web Conference 2021, pp.  1249–1261. ACM / IW3C2, 2021.
  61. Se-gsl: A general and effective graph structure learning framework through structural entropy optimization. In Proceedings of the ACM Web Conference, pp.  499–510, 2023.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up