Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Clustering Using the Soft Silhouette Score: Towards Compact and Well-Separated Clusters (2402.00608v1)

Published 1 Feb 2024 in cs.LG and cs.CV

Abstract: Unsupervised learning has gained prominence in the big data era, offering a means to extract valuable insights from unlabeled datasets. Deep clustering has emerged as an important unsupervised category, aiming to exploit the non-linear mapping capabilities of neural networks in order to enhance clustering performance. The majority of deep clustering literature focuses on minimizing the inner-cluster variability in some embedded space while keeping the learned representation consistent with the original high-dimensional dataset. In this work, we propose soft silhoutte, a probabilistic formulation of the silhouette coefficient. Soft silhouette rewards compact and distinctly separated clustering solutions like the conventional silhouette coefficient. When optimized within a deep clustering framework, soft silhouette guides the learned representations towards forming compact and well-separated clusters. In addition, we introduce an autoencoder-based deep learning architecture that is suitable for optimizing the soft silhouette objective function. The proposed deep clustering method has been tested and compared with several well-studied deep clustering methods on various benchmark datasets, yielding very satisfactory clustering results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. A comprehensive survey and analysis of generative models in machine learning. Computer Science Review, 38:100285, 2020.
  2. Network representation learning: A survey. IEEE transactions on Big Data, 6(1):3–28, 2018.
  3. Clustering methods for big data analytics. Techniques, Toolboxes and Applications, 1:91–113, 2019.
  4. A survey of kernel and spectral methods for clustering. Pattern recognition, 41(1):176–190, 2008.
  5. Anil K Jain. Data clustering: 50 years beyond k-means. Pattern recognition letters, 31(8):651–666, 2010.
  6. A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects. Engineering Applications of Artificial Intelligence, 110:104743, 2022.
  7. Np-hardness of euclidean sum-of-squares clustering. Machine learning, 75(2):245–248, 2009.
  8. James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.
  9. Christopher M Bishop. Pattern recognition. Machine learning, 128(9), 2006.
  10. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226–231, 1996.
  11. Clustering by fast search and find of density peaks. science, 344(6191):1492–1496, 2014.
  12. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37–52, 1987.
  13. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788–791, 1999.
  14. On spectral clustering: Analysis and an algorithm. In Advances in neural information processing systems, pages 849–856, 2002.
  15. Minimum density hyperplanes. Journal of Machine Learning Research, 2016.
  16. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798–1828, 2013.
  17. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097–1105, 2012.
  18. Deep learning. nature, 521(7553):436–444, 2015.
  19. Clustering with deep learning: Taxonomy and new methods. arXiv preprint arXiv:1801.07648, 2018.
  20. A survey of clustering with deep learning: From the perspective of network architecture. IEEE Access, 6:39501–39514, 2018.
  21. An Introduction to Deep Clustering, pages 73–89. 01 2019.
  22. A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions. arXiv preprint arXiv:2206.07579, 2022.
  23. Deep clustering: A comprehensive survey. arXiv preprint arXiv:2210.04142, 2022.
  24. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
  25. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  26. Graph neural networks: A review of methods and applications. AI open, 1:57–81, 2020.
  27. Clustergan: Latent space clustering in generative adversarial networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4610–4617, 2019.
  28. Variational deep embedding: An unsupervised and generative approach to clustering. arXiv preprint arXiv:1611.05148, 2016.
  29. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5147–5156, 2016.
  30. Neural clustering based on implicit maximum likelihood. Neural Computing and Applications, pages 1–14, 2023.
  31. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In international conference on machine learning, pages 3861–3870. PMLR, 2017.
  32. Internal versus external cluster validation indexes. International Journal of computers and communications, 5(1):27–34, 2011.
  33. Normalized mutual information feature selection. IEEE Transactions on neural networks, 20(2):189–201, 2009.
  34. Information theoretic measures for clusterings comparison: Variants, properties, normalization and correction for chance. Journal of Machine Learning Research, 11(95):2837–2854, 2010.
  35. Comparing partitions. Journal of classification, 2(1):193–218, 1985.
  36. Minimum adjusted rand index for two clusterings of a given size. Advances in Data Analysis and Classification, pages 1–9, 2022.
  37. Joseph C Dunn. A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. 1973.
  38. A dendrite method for cluster analysis. Communications in Statistics-theory and Methods, 3(1):1–27, 1974.
  39. A cluster separation measure. IEEE transactions on pattern analysis and machine intelligence, (2):224–227, 1979.
  40. Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65, 1987.
  41. An extensive comparative study of cluster validity indices. Pattern recognition, 46(1):243–256, 2013.
  42. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
  43. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, pages 478–487. PMLR, 2016.
  44. Improved deep embedded clustering with local structure preservation. In Ijcai, pages 1753–1759, 2017.
  45. Auto-encoder based data clustering. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 18th Iberoamerican Congress, CIARP 2013, Havana, Cuba, November 20-23, 2013, Proceedings, Part I 18, pages 117–124. Springer, 2013.
  46. Martin Dietrich Buhmann. Radial basis functions. Acta numerica, 9:1–38, 2000.
  47. Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of machine learning research, 4(Jun):119–155, 2003.
  48. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500):2319–2323, 2000.
  49. MNIST handwritten digit database. 2010.
  50. A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert systems with applications, 40(1):200–210, 2013.
  51. Emnist: Extending mnist to handwritten letters. In Int’l Joint Conf. on Neural Networks, pages 2921–2926. IEEE, 2017.
  52. Human activity recognition using smartphones. In 2018 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pages 1–6, 2018.
  53. UCI machine learning repository, 2017.
  54. Laurens Van Der Maaten. Learning a parametric embedding by preserving local structure. In Artificial intelligence and statistics, pages 384–391. PMLR, 2009.
  55. Deep embedded clustering with data augmentation. In Asian conference on machine learning, pages 550–565. PMLR, 2018.
  56. Deep density-based image clustering. Knowledge-Based Systems, 197:105841, 2020.
  57. Deep embedded k-means clustering. In 2021 International Conference on Data Mining Workshops (ICDMW), pages 686–694. IEEE, 2021.
  58. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10), pages 807–814, 2010.
  59. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  60. William M Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, 66(336):846–850, 1971.
  61. Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on information theory, 28(2):129–137, 1982.
  62. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  63. Strongly augmented contrastive clustering. Pattern Recognition, 139:109470, 2023.
  64. Spectral clustering via ensemble deep autoencoder learning (sc-edae). Pattern Recognition, 108:107522, 2020.
  65. Adversarial learning for robust deep clustering. Advances in Neural Information Processing Systems, 33:9098–9108, 2020.
  66. Self-paced learning for latent variable models. Advances in neural information processing systems, 23, 2010.
  67. Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recognition, 83:161–173, 2018.
  68. Adaptive self-paced deep clustering with data augmentation. IEEE Transactions on Knowledge and Data Engineering, 32(9):1680–1693, 2019.
  69. Self-paced deep clustering with learning loss. Pattern Recognition Letters, 171:8–14, 2023.
  70. Dip-means: an incremental clustering method for estimating the number of clusters. Advances in neural information processing systems, 25, 2012.
  71. Dip-based deep embedded clustering with k-estimation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 903–913, 2021.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now