Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Embedding Clustering Driven by Sample Stability (2401.15989v1)

Published 29 Jan 2024 in cs.LG

Abstract: Deep clustering methods improve the performance of clustering tasks by jointly optimizing deep representation learning and clustering. While numerous deep clustering algorithms have been proposed, most of them rely on artificially constructed pseudo targets for performing clustering. This construction process requires some prior knowledge, and it is challenging to determine a suitable pseudo target for clustering. To address this issue, we propose a deep embedding clustering algorithm driven by sample stability (DECS), which eliminates the requirement of pseudo targets. Specifically, we start by constructing the initial feature space with an autoencoder and then learn the cluster-oriented embedding feature constrained by sample stability. The sample stability aims to explore the deterministic relationship between samples and all cluster centroids, pulling samples to their respective clusters and keeping them away from other clusters with high determinacy. We analyzed the convergence of the loss using Lipschitz continuity in theory, which verifies the validity of the model. The experimental results on five datasets illustrate that the proposed method achieves superior performance compared to state-of-the-art clustering approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Pattern recognition and machine learning, volume 4. Springer, 2006.
  2. Large scale spectral clustering with landmark-based representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 25, pages 313–318, 2011.
  3. Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In Proceedings of the IEEE international conference on computer vision, pages 5736–5745, 2017.
  4. Improved deep embedded clustering with local structure preservation. In Ijcai, volume 17, pages 1753–1759, 2017.
  5. Deep clustering with convolutional autoencoders. In Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24, pages 373–382. Springer, 2017.
  6. Deep embedded clustering with data augmentation. In Asian conference on machine learning, pages 550–565. PMLR, 2018.
  7. Adaptive self-paced deep clustering with data augmentation. IEEE Transactions on Knowledge and Data Engineering, 32(9):1680–1693, 2019.
  8. Deep clustering: On the link between discriminative models and k-means. IEEE transactions on pattern analysis and machine intelligence, 43(6):1887–1896, 2019.
  9. A survey on contrastive self-supervised learning. Technologies, 9(1):2, 2020.
  10. Variational deep embedding: An unsupervised and generative approach to clustering. arXiv preprint arXiv:1611.05148, 2016.
  11. Self-supervised visual feature learning with deep neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(11):4037–4058, 2020.
  12. Stephen C Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):241–254, 1967.
  13. Fast agglomerative hierarchical clustering algorithm using locality-sensitive hashing. Knowledge and Information Systems, 12:25–53, 2007.
  14. Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recognition, 83:161–173, 2018.
  15. Clustering ensemble based on sample’s stability. Artificial Intelligence, 273:37–55, 2019.
  16. Clustering method based on samples stability. Sci. Sin. Inf., 50(8):1239–1254, 2020.
  17. Adaptive graph auto-encoder for general data clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):9725–9732, 2021.
  18. Improved deep convolutional embedded clustering with re-selectable sample training. Pattern Recognition, 127:108611, 2022.
  19. James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.
  20. J MacQueen. Classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, pages 281–297, 1967.
  21. Deep clustering with a dynamic autoencoder: From reconstruction towards centroids construction. Neural Networks, 130:206–228, 2020.
  22. Yurii Nesterov. Introductory lectures on convex programming volume i: Basic course. Lecture notes, 3(4):5, 1998.
  23. On spectral clustering: Analysis and an algorithm. Advances in neural information processing systems, 14, 2001.
  24. Gatcluster: Self-supervised gaussian-attention network for image clustering. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16, pages 735–751. Springer, 2020.
  25. Spice: Semantic pseudo-labeling for image clustering. IEEE Transactions on Image Processing, 31:7264–7278, 2022.
  26. Xai beyond classification: Interpretable neural clustering. The Journal of Machine Learning Research, 23(1):227–254, 2022.
  27. Semi-supervised deep embedded clustering. Neurocomputing, 325:121–130, 2019.
  28. Douglas A Reynolds et al. Gaussian mixture models. Encyclopedia of biometrics, 741(659-663), 2009.
  29. Learning statistical representation with joint deep embedded clustering. arXiv preprint arXiv:2109.05232, 1, 2021.
  30. Deepdpm: Deep clustering with an unknown number of clusters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9861–9870, 2022.
  31. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence, 22(8):888–905, 2000.
  32. Numerical taxonomy. Nature, 193:855–860, 1962.
  33. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, pages 478–487. PMLR, 2016.
  34. Survey of clustering algorithms. IEEE Transactions on neural networks, 16(3):645–678, 2005.
  35. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5147–5156, 2016.
  36. New l2, 1-norm relaxation of multi-way graph cut for clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  37. Deep spectral clustering using dual autoencoder network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4066–4075, 2019.
  38. Tdec: Deep embedded image clustering with transformer and distribution information. In Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, pages 280–288, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now