Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

One-step Multi-view Clustering with Diverse Representation (2306.05437v2)

Published 8 Jun 2023 in cs.LG and cs.AI

Abstract: Multi-view clustering has attracted broad attention due to its capacity to utilize consistent and complementary information among views. Although tremendous progress has been made recently, most existing methods undergo high complexity, preventing them from being applied to large-scale tasks. Multi-view clustering via matrix factorization is a representative to address this issue. However, most of them map the data matrices into a fixed dimension, limiting the model's expressiveness. Moreover, a range of methods suffers from a two-step process, i.e., multimodal learning and the subsequent $k$-means, inevitably causing a sub-optimal clustering result. In light of this, we propose a one-step multi-view clustering with diverse representation method, which incorporates multi-view learning and $k$-means into a unified framework. Specifically, we first project original data matrices into various latent spaces to attain comprehensive information and auto-weight them in a self-supervised manner. Then we directly use the information matrices under diverse dimensions to obtain consensus discrete clustering labels. The unified work of representation learning and clustering boosts the quality of the final results. Furthermore, we develop an efficient optimization algorithm with proven convergence to solve the resultant problem. Comprehensive experiments on various datasets demonstrate the promising clustering performance of our proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. J. Han, J. Xu, F. Nie, and X. Li, “Multi-view k-means clustering with adaptive sparse memberships and weight allocation,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 2, pp. 816–827, 2022.
  2. S. Wang, X. Liu, L. Liu, S. Zhou, and E. Zhu, “Late fusion multiple kernel clustering with proxy graph refinement,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–12, 2021.
  3. Z. Tao, H. Liu, S. Li, Z. Ding, and Y. Fu, “Marginalized multiview ensemble clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 2, pp. 600–611, 2020.
  4. X. Peng, H. Zhu, J. Feng, C. Shen, H. Zhang, and J. T. Zhou, “Deep clustering with sample-assignment invariance prior,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4857–4868, 2020.
  5. Q. Wang, Z. Qin, F. Nie, and X. Li, “Spectral embedded adaptive neighbors clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 4, pp. 1265–1271, 2019.
  6. Q. Zheng, “Large-scale multi-view clustering via fast essential subspace representation learning,” IEEE Signal Processing Letters, vol. 29, pp. 1893–1897, 2022.
  7. W. Yang, Y. Gao, Y. Shi, and L. Cao, “Mrm-lasso: A sparse multiview feature selection method via low-rank analysis,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 11, pp. 2801–2815, 2015.
  8. S. Huang, Y. Liu, I. W. Tsang, Z. Xu, and J. Lv, “Multi-view subspace clustering by joint measuring of consistency and diversity,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–12, 2022.
  9. G. F. Tzortzis and A. C. Likas, “Multiple view clustering using a weighted combination of exemplar-based mixture models,” IEEE Transactions on Neural Networks, vol. 21, no. 12, pp. 1925–1938, 2010.
  10. L. Huang, C.-D. Wang, H.-Y. Chao, and P. S. Yu, “Mvstream: Multiview data stream clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 9, pp. 3482–3496, 2020.
  11. Z. Ding and Y. Fu, “Robust multiview data analysis through collective low-rank subspace,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 5, pp. 1986–1997, 2018.
  12. Y. Wang, W. Zhang, L. Wu, X. Lin, and X. Zhao, “Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 1, pp. 57–70, 2017.
  13. U. Fang, M. Li, J. Li, L. Gao, T. Jia, and Y. Zhang, “A comprehensive survey on multi-view clustering,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–20, 2023.
  14. J. Xu, Y. Ren, H. Tang, Z. Yang, L. Pan, Y. Yang, X. Pu, P. S. Yu, and L. He, “Self-supervised discriminative feature learning for deep multi-view clustering,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 7, pp. 7470–7482, 2023.
  15. C. Zhang, Y. Cui, Z. Han, J. T. Zhou, H. Fu, and Q. Hu, “Deep partial multi-view learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2402–2415, 2022.
  16. H. Gao, F. Nie, X. Li, and H. Huang, “Multi-view subspace clustering,” in 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 4238–4246.
  17. J. Liu, X. Liu, Y. Yang, X. Guo, M. Kloft, and L. He, “Multiview subspace clustering via co-training robust data representation,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–13, 2021.
  18. S. Liu, S. Wang, P. Zhang, K. Xu, X. Liu, C. Zhang, and F. Gao, “Efficient one-pass multi-view subspace clustering with consensus anchors,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 7, 2022, pp. 7576–7584.
  19. M. Yin, J. Gao, S. Xie, and Y. Guo, “Multiview subspace clustering via tensorial t-product representation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 3, pp. 851–864, 2019.
  20. Z. Yue, H. Yong, D. Meng, Q. Zhao, Y. Leung, and L. Zhang, “Robust multiview subspace learning with nonindependently and nonidentically distributed complex noise,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 4, pp. 1070–1083, 2020.
  21. T. Zhang, X. Liu, L. Gong, S. Wang, X. Niu, and L. Shen, “Late fusion multiple kernel clustering with local kernel alignment maximization,” IEEE Transactions on Multimedia, pp. 1–1, 2021.
  22. L. Li, S. Wang, X. Liu, E. Zhu, L. Shen, K. Li, and K. Li, “Local sample-weighted multiple kernel clustering with consensus discriminative graph,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–14, 2022.
  23. J. Zhang, L. Li, S. Wang, J. Liu, Y. Liu, X. Liu, and E. Zhu, “Multiple kernel clustering with dual noise minimization,” in Proceedings of the 30th ACM International Conference on Multimedia, ser. MM ’22.   New York, NY, USA: Association for Computing Machinery, 2022, p. 3440–3450.
  24. X. Wan, J. Liu, W. Liang, X. Liu, Y. Wen, and E. Zhu, “Continual multi-view clustering,” in Proceedings of the 30th ACM International Conference on Multimedia, ser. MM ’22.   New York, NY, USA: Association for Computing Machinery, 2022, p. 3676–3684. [Online]. Available: https://doi.org/10.1145/3503161.3547864
  25. Z. Ren and Q. Sun, “Simultaneous global and local graph structure preserving for multiple kernel clustering,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 5, pp. 1839–1851, 2021.
  26. W. Liang, S. Zhou, J. Xiong, X. Liu, S. Wang, E. Zhu, Z. Cai, and X. Xu, “Multi-view spectral clustering with high-order optimal neighborhood laplacian matrix,” in IEEE Transactions on Knowledge and Data Engineering (TKDE), 2020.
  27. C. Tang, X. Liu, X. Zhu, E. Zhu, Z. Luo, L. Wang, and W. Gao, “Cgd: Multi-view clustering via cross-view graph diffusion,” in AAAI Conference on Artificial Intelligence (AAAI), 2020, pp. 5924–5931.
  28. Y. Liu, W. Tu, S. Zhou, X. Liu, L. Song, X. Yang, and E. Zhu, “Deep graph clustering via dual correlation reduction,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 7, 2022, pp. 7603–7611.
  29. Y. Liu, X. Yang, S. Zhou, and X. Liu, “Simple contrastive graph clustering,” arXiv preprint arXiv:2205.07865, 2022.
  30. W. Xia, Q. Gao, Q. Wang, X. Gao, C. Ding, and D. Tao, “Tensorized bipartite graph learning for multi-view clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  31. Y. Wang, L. Wu, X. Lin, and J. Gao, “Multiview spectral clustering via structured low-rank matrix factorization,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 10, pp. 4833–4843, 2018.
  32. H. Wang, Y. Yang, and T. Li, “Multi-view clustering via concept factorization with local manifold regularization,” in 2016 IEEE 16th International Conference on Data Mining (ICDM), 2016, pp. 1245–1250.
  33. G. A. Khan, J. Hu, T. Li, B. Diallo, and Q. Huang, “Weighted multi-view data clustering via joint non-negative matrix factorization,” in 2019 IEEE 14th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), 2019, pp. 1159–1165.
  34. S. Gao, Z. Yu, T. Jin, and M. Yin, “Multi-view low-rank matrix factorization using multiple manifold regularization,” Neurocomputing, vol. 335, pp. 143–152, 2019.
  35. C. Zhang, Q. Hu, H. Fu, P. Zhu, and X. Cao, “Latent multi-view subspace clustering,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4333–4341.
  36. X. Liu, “Incomplete multiple kernel alignment maximization for clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2021.
  37. X. Li, H. Zhang, R. Wang, and F. Nie, “Multiview clustering: A scalable and parameter-free bipartite graph fusion method,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 1, pp. 330–344, 2022.
  38. N. Liang, Z. Yang, Z. Li, S. Xie, and C.-Y. Su, “Semi-supervised multi-view clustering with graph-regularized partially shared non-negative matrix factorization,” Know.-Based Syst., vol. 190, no. C, feb 2020. [Online]. Available: https://doi.org/10.1016/j.knosys.2019.105185
  39. N. Liang, Z. Yang, Z. Li, W. Sun, and S. Xie, “Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints,” Knowledge-Based Systems, vol. 194, p. 105582, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950705120300642
  40. P. Luo, J. Peng, Z. Guan, and J. Fan, “Dual regularized multi-view non-negative matrix factorization for clustering,” Neurocomputing, vol. 294, pp. 1–11, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231217316740
  41. Z. Zhang, L. Liu, F. Shen, H. T. Shen, and L. Shao, “Binary multi-view clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 7, pp. 1774–1782, 2019.
  42. J. Liu, C. Wang, J. Gao, and J. Han, “Multi-view clustering via joint nonnegative matrix factorization,” in Proceedings of the 2013 SIAM international conference on data mining.   SIAM, 2013, pp. 252–260.
  43. J. Wang, F. Tian, H. Yu, C. H. Liu, K. Zhan, and X. Wang, “Diverse non-negative matrix factorization for multiview data representation5,” IEEE Transactions on Cybernetics, vol. 48, no. 9, pp. 2620–2632, 2018.
  44. J. Liu, X. Liu, Y. Yang, L. Liu, S. Wang, W. Liang, and J. Shi, “One-pass multi-view clustering for large-scale data,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 12 324–12 333.
  45. C. Tang, Z. Li, J. Wang, X. Liu, W. Zhang, and E. Zhu, “Unified one-step multi-view spectral clustering,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–1, 2022.
  46. X. Wan, X. Liu, J. Liu, S. Wang, Y. Wen, W. Liang, E. Zhu, Z. Liu, and L. Zhou, “Auto-weighted multi-view clustering for large-scale data,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  47. P. Zhang, X. Liu, J. Xiong, S. Zhou, W. Zhao, E. Zhu, and Z. Cai, “Consensus one-step multi-view subspace clustering,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 10, pp. 4676–4689, 2022.
  48. S. Dasgupta, “The hardness of k-means clustering,” 2007.
  49. S. Pei, H. Chen, F. Nie, R. Wang, and X. Li, “Centerless clustering: An efficient variant of k-means based on k-nn graph,” IEEE transactions on pattern analysis and machine intelligence, vol. PP, February 2022. [Online]. Available: https://doi.org/10.1109/TPAMI.2022.3150981
  50. J.-W. Liu, Y.-F. Wang, R.-K. Lu, and X.-L. Luo, “Multi-view non-negative matrix factorization discriminant learning via cross entropy loss,” in 2020 Chinese Control And Decision Conference (CCDC), 2020, pp. 3964–3971.
  51. C. Ding, L. Tao, P. Wei, and H. Park, “Orthogonal nonnegative matrix tri-factorizations for clustering,” in Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, August 20-23, 2006, 2006.
  52. S. Wang, X. Liu, E. Zhu, C. Tang, J. Liu, J. Hu, J. Xia, and J. Yin, “Multi-view clustering via late fusion alignment maximization,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19.   International Joint Conferences on Artificial Intelligence Organization, 7 2019, pp. 3778–3784. [Online]. Available: https://doi.org/10.24963/ijcai.2019/524
  53. L. Zong, X. Zhang, L. Zhao, H. Yu, and Q. Zhao, “Multi-view clustering via multi-manifold regularized non-negative matrix factorization,” Neural Networks, vol. 88, pp. 74–89, 2017. [Online]. Available: https://www-sciencedirect-com-s.libyc.nudt.edu.cn:443/science/article/pii/S089360801730028X
  54. H. Xu, L. Gong, H. Xuan, X. Zheng, Z. Gao, and X. Wen, “Multiview clustering via consistent and specific nonnegative matrix factorization with graph regularization,” Multimedia Syst., vol. 28, no. 5, p. 1559–1572, oct 2022. [Online]. Available: https://doi-org-s.libyc.nudt.edu.cn:443/10.1007/s00530-022-00905-x
  55. Q. Ou, S. Wang, S. Zhou, M. Li, X. Guo, and E. Zhu, “Anchor-based multiview subspace clustering with diversity regularization,” IEEE MultiMedia, vol. 27, no. 4, pp. 91–101, 2020.
  56. M. Sun, P. Zhang, S. Wang, S. Zhou, W. Tu, X. Liu, E. Zhu, and C. Wang, “Scalable multi-view subspace clustering with unified anchors,” in Proceedings of the 29th ACM International Conference on Multimedia, ser. MM ’21.   New York, NY, USA: Association for Computing Machinery, 2021, p. 3528–3536. [Online]. Available: https://doi.org/10.1145/3474085.3475516
  57. R. Li, C. Zhang, Q. Hu, P. Zhu, and Z. Wang, “Flexible multi-view representation learning for subspace clustering,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19.   International Joint Conferences on Artificial Intelligence Organization, 7 2019, pp. 2916–2922. [Online]. Available: https://doi.org/10.24963/ijcai.2019/404
  58. M. Sun, P. Zhang, S. Wang, S. Zhou, W. Tu, X. Liu, E. Zhu, and C. Wang, “Scalable multi-view subspace clustering with unified anchors,” Proceedings of the 29th ACM International Conference on Multimedia, 2021.
  59. S. Wang, X. Liu, X. Zhu, P. Zhang, Y. Zhang, F. Gao, and E. Zhu, “Fast parameter-free multi-view subspace clustering with consensus anchor guidance,” IEEE Transactions on Image Processing, vol. 31, pp. 556–568, 2022.
  60. Z. Kang, W. Zhou, Z. Zhao, J. Shao, M. Han, and Z. Xu, “Large-scale multi-view subspace clustering in linear time,” in AAAI, 2020.
  61. H. W. Kuhn, “The hungarian method for the assignment problem,” Naval Research Logistics (NRL), vol. 52, 1955.
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now