Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CCFC: Bridging Federated Clustering and Contrastive Learning (2401.06634v1)

Published 12 Jan 2024 in cs.LG and cs.AI

Abstract: Federated clustering, an essential extension of centralized clustering for federated scenarios, enables multiple data-holding clients to collaboratively group data while keeping their data locally. In centralized scenarios, clustering driven by representation learning has made significant advancements in handling high-dimensional complex data. However, the combination of federated clustering and representation learning remains underexplored. To bridge this, we first tailor a cluster-contrastive model for learning clustering-friendly representations. Then, we harness this model as the foundation for proposing a new federated clustering method, named cluster-contrastive federated clustering (CCFC). Benefiting from representation learning, the clustering performance of CCFC even double those of the best baseline methods in some cases. Compared to the most related baseline, the benefit results in substantial NMI score improvements of up to 0.4155 on the most conspicuous case. Moreover, CCFC also shows superior performance in handling device failures from a practical viewpoint.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. A faster secure content-based image retrieval using clustering for cloud. Expert Systems with Applications, 189:116070, 2022.
  2. Learning representations by maximizing mutual information across views. Advances in neural information processing systems, 32, 2019.
  3. Fcm: The fuzzy c-means clustering algorithm. Computers & geosciences, 10(2-3):191–203, 1984.
  4. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  5. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750–15758, 2021.
  6. Communication-efficient and model-heterogeneous personalized federated learning via clustered knowledge transfer. IEEE Journal of Selected Topics in Signal Processing, 17(1):234–247, 2023.
  7. Federated unsupervised clustering with generative models. In AAAI 2022 International Workshop on Trustable, Verifiable and Auditable Federated Learning, 2022.
  8. Heterogeneity for the win: One-shot federated clustering. In International Conference on Machine Learning, pages 2611–2620. PMLR, 2021.
  9. Self-supervised representation learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39(3):42–62, 2022.
  10. Client selection in federated learning: Principles, challenges, and opportunities. IEEE Internet of Things Journal, 2023.
  11. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271–21284, 2020.
  12. Fedx: Unsupervised federated learning with cross knowledge distillation. In European Conference on Computer Vision, pages 691–707. Springer, 2022.
  13. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  14. A k-means clustering and svm based hybrid concept drift detection technique for network anomaly detection. Expert Systems with Applications, 193:116510, 2022.
  15. Fuzzy logic-based ddos attacks and network traffic anomaly detection methods: Classification, overview, and future perspectives. Information Sciences, 2023.
  16. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  17. Model-contrastive federated learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713–10722, 2021.
  18. Evaluation of community detection methods. IEEE Transactions on Knowledge and Data Engineering, 32(9):1736–1746, 2019.
  19. Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on information theory, 28(2):129–137, 1982.
  20. Multi-center federated learning: clients clustering for better personalization. World Wide Web, 26(1):481–500, 2023.
  21. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017.
  22. Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal Processing, 2022.
  23. Fedproc: Prototypical contrastive federated learning on non-iid data. Future Generation Computer Systems, 143:93–104, 2023.
  24. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  25. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  26. Self-supervised learning for videos: A survey. ACM Computing Surveys, 55(13s):1–37, 2023.
  27. Towards federated clustering: A federated fuzzy c𝑐citalic_c-means algorithm (ffcm). In AAAI 2022 International Workshop on Trustable, Verifiable and Auditable Federated Learning, 2022.
  28. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of machine learning research, 3(Dec):583–617, 2002.
  29. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8432–8440, 2022.
  30. Privacy-preserving federated deep clustering based on gan. arXiv preprint arXiv:2211.16965, 2022a.
  31. Federated clustering with gan-based data synthesis. arXiv preprint arXiv:2210.16524, 2022b.
  32. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In International Conference on Machine Learning, pages 3861–3870. PMLR, 2017.
  33. Centerclip: Token clustering for efficient text-video retrieval. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 970–981, 2022.
  34. A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions. arXiv preprint arXiv:2206.07579, 2022.
  35. Collaborative unsupervised visual representation learning from decentralized data. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4912–4921, 2021.
  36. Divergence-aware federated self-supervised learning. arXiv preprint arXiv:2204.04385, 2022.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets