Every Node is Different: Dynamically Fusing Self-Supervised Tasks for Attributed Graph Clustering (2401.06595v1)
Abstract: Attributed graph clustering is an unsupervised task that partitions nodes into different groups. Self-supervised learning (SSL) shows great potential in handling this task, and some recent studies simultaneously learn multiple SSL tasks to further boost performance. Currently, different SSL tasks are assigned the same set of weights for all graph nodes. However, we observe that some graph nodes whose neighbors are in different groups require significantly different emphases on SSL tasks. In this paper, we propose to dynamically learn the weights of SSL tasks for different nodes and fuse the embeddings learned from different SSL tasks to boost performance. We design an innovative graph clustering approach, namely Dynamically Fusing Self-Supervised Learning (DyFSS). Specifically, DyFSS fuses features extracted from diverse SSL tasks using distinct weights derived from a gating network. To effectively learn the gating network, we design a dual-level self-supervised strategy that incorporates pseudo labels and the graph structure. Extensive experiments on five datasets show that DyFSS outperforms the state-of-the-art multi-task SSL methods by up to 8.66% on the accuracy metric. The code of DyFSS is available at: https://github.com/q086/DyFSS.
- Structural Deep Clustering Network. In WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, 1400–1410. ACM / IW3C2.
- Improved Deep Embedded Clustering with Local Structure Preservation. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, 1753–1759. ijcai.org.
- A K-Means Clustering Algorithm. Journal of the Royal Statistical Society: Series C (Applied Statistics), 28(1): 100–108.
- GraphMAE: Self-Supervised Masked Graph Autoencoders. In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022, 594–604. ACM.
- Automated Self-Supervised Learning for Graphs. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 1–13. OpenReview.net.
- Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalization. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, 1–13. OpenReview.net.
- Variational Graph Auto-Encoders. CoRR, abs/1611.07308.
- Rethinking Graph Auto-Encoder Models for Attributed Graph Clustering (Extended abstract). In 39th IEEE International Conference on Data Engineering, ICDE 2023, Anaheim, CA, USA, April 3-7, 2023, 3891–3892. IEEE.
- Adversarially Regularized Graph Autoencoder for Graph Embedding. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence.
- Self-Supervised Graph Representation Learning via Global Context Prediction. CoRR, abs/2003.01604.
- DeepWalk: online learning of social representations. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014, 701–710. ACM.
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
- Large-Scale Representation Learning on Graphs via Bootstrapping. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
- RWR-GAE: Random Walk Regularization for Graph Auto Encoders. CoRR, abs/1908.04003.
- Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86): 2579–2605.
- Deep Graph Infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 1–13. OpenReview.net.
- Attributed Graph Clustering: A Deep Attentional Embedding Approach. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, 3670–3676. ijcai.org.
- Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive. IEEE Trans. Knowl. Data Eng., 35(4): 4216–4235.
- Self-consistent Contrastive Attributed Graph Clustering with Pseudo-label Prompt. IEEE Transactions on Multimedia, 1–13.
- Unsupervised Deep Embedding for Clustering Analysis. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, 478–487. JMLR.org.
- Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, 3861–3870. PMLR.
- Cluster-Guided Contrastive Graph Clustering Network. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, 10834–10842. AAAI Press.
- When Does Self-Supervision Help Graph Convolutional Networks? In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, 10871–10880. PMLR.
- From Canonical Correlation Analysis to Self-supervised Graph Neural Networks. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 76–89.
- Collaborative Decision-Reinforced Self-Supervision for Attributed Graph Clustering. IEEE Transactions on Neural Networks and Learning Systems, 1–13.
- Graph Contrastive Learning with Adaptive Augmentation. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, 2069–2080. ACM / IW3C2.
- Pengfei Zhu (76 papers)
- Qian Wang (453 papers)
- Yu Wang (939 papers)
- Jialu Li (53 papers)
- Qinghua Hu (83 papers)