Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Efficient and Effective Deep Clustering with Dynamic Grouping and Prototype Aggregation (2401.13581v2)

Published 24 Jan 2024 in cs.CV

Abstract: Previous contrastive deep clustering methods mostly focus on instance-level information while overlooking the member relationship within groups/clusters, which may significantly undermine their representation learning and clustering capability. Recently, some group-contrastive methods have been developed, which, however, typically rely on the samples of the entire dataset to obtain pseudo labels and lack the ability to efficiently update the group assignments in a batch-wise manner. To tackle these critical issues, we present a novel end-to-end deep clustering framework with dynamic grouping and prototype aggregation, termed as DigPro. Specifically, the proposed dynamic grouping extends contrastive learning from instance-level to group-level, which is effective and efficient for timely updating groups. Meanwhile, we perform contrastive learning on prototypes in a spherical feature space, termed as prototype aggregation, which aims to maximize the inter-cluster distance. Notably, with an expectation-maximization framework, DigPro simultaneously takes advantage of compact intra-cluster connections, well-separated clusters, and efficient group updating during the self-supervised training. Extensive experiments on six image benchmarks demonstrate the superior performance of our approach over the state-of-the-art. Code is available at https://github.com/Regan-Zhang/DigPro.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Self-labelling via simultaneous clustering and representation learning. arXiv preprint arXiv:1911.05371, 2019.
  2. Locality preserving nonnegative matrix factorization. In Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2009.
  3. Deep clustering for unsupervised learning of visual features. In Proceedings of the European conference on computer vision (ECCV), pages 132–149, 2018.
  4. Unsupervised learning of visual features by contrasting cluster assignments. In Advanced in Neural Information Processing Systems (NeurIPS), 2020.
  5. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision, pages 5879–5887, 2017.
  6. A simple framework for contrastive learning of visual representations. In Proc. of International Conference on Machine Learning (ICML), 2020.
  7. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750–15758, 2021.
  8. Deep and structure-preserving autoencoders for clustering data with missing information. IEEE Transactions on Emerging Topics in Computational Intelligence, 5(4):639–650, 2021.
  9. Enhanced model-based clustering, density estimation,and discriminant analysis software: MCLUST. Journal of Classification, 20(2):263–286, 2003.
  10. Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recognition, 10(2):105–12, 1978.
  11. Bootstrap your own latent-a new approach to self-supervised learning. In Advanced in Neural Information Processing Systems (NeurIPS), 2020.
  12. Improved deep embedded clustering with local structure preservation. In Proc. of International Joint Conference on Artificial Intelligence (IJCAI), 2017.
  13. Adaptive self-paced deep clustering with data augmentation. IEEE Transactions on Knowledge and Data Engineering, 32(9):1680–1693, 2019.
  14. Hcsc: hierarchical contrastive selective coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9706–9715, 2022.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  16. Momentum contrast for unsupervised visual representation learning. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  17. Adco: Adversarial contrast for efficient learning of unsupervised representations from self-trained negative adversaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1074–1083, 2021.
  18. Ultra-scalable spectral clustering and ensemble clustering. IEEE Transactions on Knowledge and Data Engineering, 32(6):1212–1226, 2020.
  19. Toward multidiversified ensemble clustering of high-dimensional data: From subspaces to metrics and beyond. IEEE Transactions on Cybernetics, pages 1–14, 2021.
  20. Learning representation for clustering via prototype scattering and positive sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–16, 2022.
  21. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
  22. Prototypical contrastive learning of unsupervised representations. arXiv preprint arXiv:2005.04966, 2020.
  23. Contrastive clustering. In Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2021.
  24. Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5):2486–2498, 2017.
  25. James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proc. of Mathematical Statistics and Probability, 1967.
  26. Pca versus lda. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):228–233, 2001.
  27. Unsupervised visual representation learning by synchronous momentum grouping. In European Conference on Computer Vision, pages 265–282. Springer, 2022.
  28. Clustering-friendly representation learning via instance discrimination and feature decorrelation. arXiv preprint arXiv:2106.00131, 2021.
  29. Mice: Mixture of contrastive experts for unsupervised image clustering. In International conference on learning representations, 2020.
  30. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
  31. Scan: Learning to classify images without labels. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X, pages 268–285. Springer, 2020.
  32. Unsupervised feature learning by cross-level instance-group discrimination. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12586–12595, 2021.
  33. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3733–3742, 2018.
  34. Unsupervised deep embedding for clustering analysis. In Proc. of International Conference on Machine Learning (ICML), 2016.
  35. Bag-of-visual-words and spatial extensions for land-use classification. In Proc. of SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2010.
  36. Self-tuning spectral clustering. In Advanced in Neural Information Processing Systems (NeurIPS), 2005.
  37. BIRCH: An efficient data clustering method for very large databases. In Proc. of SIGMOD International Conference on Management of Data, 1996.
  38. Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing, 54(4):2108–2123, 2015.
  39. Graph contrastive clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9224–9233, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Haixin Zhang (6 papers)
  2. Dong Huang (102 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now