Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learning (2403.09284v1)

Published 14 Mar 2024 in cs.LG and cs.DC

Abstract: Personalized federated learning becomes a hot research topic that can learn a personalized learning model for each client. Existing personalized federated learning models prefer to aggregate similar clients with similar data distribution to improve the performance of learning models. However, similaritybased personalized federated learning methods may exacerbate the class imbalanced problem. In this paper, we propose a novel Dynamic Affinity-based Personalized Federated Learning model (DA-PFL) to alleviate the class imbalanced problem during federated learning. Specifically, we build an affinity metric from a complementary perspective to guide which clients should be aggregated. Then we design a dynamic aggregation strategy to dynamically aggregate clients based on the affinity metric in each round to reduce the class imbalanced risk. Extensive experiments show that the proposed DA-PFL model can significantly improve the accuracy of each client in three real-world datasets with state-of-the-art comparison methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, and B. He, “A survey on federated learning systems: vision, hype and reality for data privacy and protection,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 04, pp. 3347–3366, 2023.
  2. S. Chen, and B. Li, “Towards optimal multi-modal federated learning on non-iid data with hierarchical gradient blending,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications, 2022, pp. 1469–1478.
  3. J. Oh, S. Kim, and S.-Y. Yun, “FedBABU: Toward enhanced representation for federated image classification,” in The Tenth International Conference on Learning Representations, 2022.
  4. M. Luo, F. Chen, D. Hu, Y. Zhang, J. Liang, and J. Feng, “No fear of heterogeneity: Classifier calibration for federated learning with non-iid data,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 5972–5984.
  5. K. Muhammad, Q. Wang, D. O’Reilly-Morgan, E. Tragos, B. Smyth, N. Hurley, J. Geraci, and A. Lawlor, “Fedfast: Going beyond average for faster training of federated recommender systems,” in The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2020, pp. 1234–1242.
  6. Q. Wang, H. Yin, T. Chen, J. Yu, A. Zhou, and X. Zhang, “Fast-adapting and privacy-preserving federated recommender system,” The VLDB J., vol. 32, pp. 877-–896, 2022.
  7. S. Zhang, W. Yuan, and H. Yin, “Comprehensive privacy analysis on federated recommender system against attribute inference attacks,” IEEE Trans. Knowl. Data Eng., no. 01, pp. 1–13, 2023.
  8. W. Huang, J. Liu, T. Li, T. Huang, S. Ji, and J. Wan, “Feddsr: Daily schedule recommendation in a federated deep reinforcement learning framework,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 04, pp. 3912–3924, 2023.
  9. P. Zhou, K. Wang, L. Guo, S. Gong, and B. Zheng, “A privacy-preserving distributed contextual federated online learning framework with big data support in social recommender systems,” IEEE Trans. Knowl. Data Eng., vol. 33, no. 03, pp. 824–838, 2021.
  10. R. Yu and P. Li, “Toward resource-efficient federated learning in mobile edge computing,” IEEE Network, vol. 35, no. 1, pp. 148–155, 2021.
  11. K. Singhal, H. Sidahmed, Z. Garrett, S. Wu, J. Rush, and S. Prakash, “Federated reconstruction: Partially local federated learning,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 11220–11232.
  12. Q. Yang, J. Zhang, W. Hao, G. P. Spell, and L. Carin, “Flop: Federated learning on medical datasets using partial networks,” in The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021, pp. 3845–3853.
  13. I. Dayan, H. R. Roth, A. Zhong, A. Harouni, A. Gentili, A. Z. Abidin, A. Liu, A. B. Costa, B. J. Wood, C.-S. Tsai et al., “Federated learning for predicting clinical outcomes in patients with covid-19,” Nat. Med., vol. 27, no. 10, pp. 1735–1743, 2021.
  14. D. Chen, D. Gao, W. Kuang, Y. Li, and B. Ding, “pfl-bench: A comprehensive benchmark for personalized federated learning,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 9344–9360.
  15. A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient framework for clustered federated learning,” IEEE Trans. Inf. Theory, vol. 68, no. 12, pp. 8076–8091, 2022.
  16. Y. Ruan and C. Joe-Wong, “FedSoft: Soft clustered federated learning with proximal local updating,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, 2022, pp. 8124–8131.
  17. Y. Huang, L. Chu, Z. Zhou, L. Wang, J. Liu, J. Pei, and Y. Zhang, “Personalized cross-silo federated learning on non-iid data,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 7865–7873.
  18. B. Liu, Y. Guo, and X. Chen, “PFA: Privacy-preserving federated adaptation for effective model personalization,” in Proceedings of the Web Conference 2021, 2021, pp. 923–934.
  19. P.-E. Danielsson, “Euclidean distance mapping,” Comput. Graph. Image Proc. , vol. 14, no. 3, pp. 227–248, 1980.
  20. X. Ma, J. Zhang, S. Guo, and W. Xu, “Layer-wised model aggregation for personalized federated learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10092–10101.
  21. W. Jeong and S. J. Hwang, “Factorized-fl: Personalized federated learning with parameter factorization & similarity matching,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 35684–35695.
  22. X. Ouyang, Z. Xie, J. Zhou, J. Huang, and G. Xing, “Clusterfl: A similarity-aware federated learning system for human activity recognition,” in The 19th Annual International Conference on Mobile Systems, Applications, and Services, Virtual Event, 2021, pp. 54–66.
  23. C. Palihawadana, N. Wiratunga, A. Wijekoon, and H. Kalutarage, “Fedsim: Similarity guided model aggregation for federated learning,” Neurocomputing, vol. 483, pp. 432–445, 2022.
  24. S. Vahidian, M. Morafah, W. Wang, V. Kungurtsev, C. Chen, M. Shah, and B. Lin, “Efficient distribution similarity identification in clustered federated learning via principal angles between client data subspaces,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, 2023, pp. 10043–10052.
  25. T. Li, S. Hu, A. Beirami, and V. Smith, “Ditto: Fair and robust federated learning through personalization,” in Proceedings of the 38th International Conference on Machine Learning, 2021, pp. 6357–6368.
  26. C. T Dinh, N. Tran, and J. Nguyen, “Personalized federated learning with moreau envelopes,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 21394–21405.
  27. R. Yang, J. Tian, and Y. Zhang, “Regularized mutual learning for personalized federated learning,” in Asian Conference on Machine Learning, vol. 13, 2021, pp. 1521–1536.
  28. J. Kim, G. Kim, and B. Han, “Multi-level branched regularization for federated learning,” in Proceedings of the 39th International Conference on Machine Learning, 2022, pp. 11058–11073.
  29. A. Cheng, P. Wang, X. S. Zhang, and J. Cheng, “Differentially private federated learning with local regularization and sparsification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10122–10131.
  30. S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” in Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 5132–5143.
  31. X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” in Proceedings of the 8th International Conference on Learning Representations, 2020.
  32. A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 3557–3568.
  33. Q. Li, B. He, and D. Song, “Model-contrastive federated learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10713–10722.
  34. L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting shared representations for personalized federated learning,” in Proceedings of the 38th International Conference on Machine Learning, 2021, pp. 2089–2099.
  35. H. Zhu, H. Zhang, and Y. Jin, “From federated learning to federated neural architecture search: a survey,” Complex Intell. Syst., vol. 7, pp. 639–657, 2021.
  36. C. Thapa, P. C. M. Arachchige, S. Camtepe, and L. Sun, “Splitfed: When federated learning meets split learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, 2022, pp. 8485–8493.
  37. X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,” in Proceedings of the 9th International Conference on Learning Representations, 2021.
  38. C. Wang, B. Chen, G. Li, and H. Wang, “Automated graph neural network search under federated learning framework,” IEEE Trans. Knowl. Data Eng., no. 01, pp. 1–13, 2023.
  39. Z. Pan, L. Hu, W. Tang, J. Li, Y. He, and Z. Liu, “Privacy-preserving multi-granular federated neural architecture search – a general framework,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 03, pp. 2975–2986, 2023.
  40. B. Sun, H. Huo, Y. Yang, and B. Bai, “Partialfed: Cross-domain personalized federated learning via partial initialization,” in Advances in Neural Information Processing Systems, vol.34, 2021, pp. 23309–23320.
  41. Z. Chai, A. Ali, S. Zawad, S. Truex, A. Anwar, N. Baracaldo, Y. Zhou, H. Ludwig, F. Yan, and Y. Cheng, “Tifl: A tier-based federated learning system,” in Proceedings of the 29th international symposium on high-performance parallel and distributed computing, 2020, pp. 125–136.
  42. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017, pp. 1273–1282.
  43. F. Sattler, K.-R. Müller, T. Wiegand, and W. Samek, “On the byzantine robustness of clustered federated learning,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2020, pp. 8861–8865.
  44. C. Liu, C. Lou, R. Wang, A. Y. Xi, L. Shen, and J. Yan, “Deep neural network fusion via graph matching with applications to model ensemble and federated learning,” in Proceedings of the 39th International Conference on Machine Learning, 2022, pp. 13857–13869.
  45. J. Baek, W. Jeong, J. Jin, J. Yoon, and S. J. Hwang, “Personalized subgraph federated learning,” in Proceedings of the 40th International Conference on Machine Learning, 2023, pp. 1396–1415.
  46. L. Tu, X. Ouyang, J. Zhou, Y. He, and G. Xing, “Feddl: Federated learning via dynamic layer sharing for human activity recognition,” in Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, 2021, pp. 15–28.
  47. J. Zhang, S. Guo, X. Ma, H. Wang, W. Xu, and F. Wu, “Parameterized knowledge transfer for personalized federated learning,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 10 092–10 104.
  48. T. V. Vo, A. Bhattacharyya, Y. Lee, and T.-Y. Leong, “An adaptive kernel approach to federated learning of heterogeneous causal effects,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 459–24 473.
  49. D. Caldarola, M. Mancini, F. Galasso, M. Ciccone, E. Rodolà, and B. Caputo, “Cluster-driven graph federated learning over multiple domains,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 2749–2758.
  50. Y. J. Cho, J. Wang, T. Chirvolu and G. Joshi, “Communication-efficient and model-heterogeneous personalized federated learning via clustered knowledge transfer,” IEEE J. Sel. Top. Signal Process., vol. 17, no. 1, pp. 234–247, 2023.
  51. Z. Wang, H. Xu, J. Liu, Y. Xu, H. Huang, and Y. Zhao, “Accelerating federated learning with cluster construction and hierarchical aggregation,” IEEE Trans. Mob. Comput., vol. 22, no. 7, pp. 3805–3822, 2022.
  52. X. Ouyang, Z. Xie, J. Zhou, G. Xing, and J. Huang, “Clusterfl: A clustering-based federated learning system for human activity recognition,” ACM Trans. Sens. Netw., vol. 19, no. 1, pp. 1–32, 2022.
  53. Y. Fraboni, R. Vidal, L. Kameni, and M. Lorenzi, “Clustered sampling: Low-variance and improved representativity for clients selection in federated learning,” in Proceedings of the 38th International Conference on Machine Learning, 2021, pp. 3407–3416.
  54. D. Song, G. Shen, D. Gao, L. Yang, X. Zhou, S. Pan, W. Lou, and F. Zhou, “Fast heterogeneous federated learning with hybrid client selection,” in Uncertainty in Artificial Intelligence, 2023, pp. 2006–2015.
  55. C. Briggs, Z. Fan, and P. Andras, “Federated learning with hierarchical clustering of local updates to improve training on non-iid data,” in International Joint Conference on Neural Networks, 2020, pp. 1–9.
  56. M. Duan, D. Liu, X. Ji, R. Liu, L. Liang, X. Chen, and Y. Tan, “Fedgroup: Efficient federated learning via decomposed similarity-based clustering,” in IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking, 2021, pp. 228–237.
  57. Y. Tan, G. Long, L. Liu, T. Zhou, Q. Lu, J. Jiang, and C. Zhang, “Fedproto: Federated prototype learning across heterogeneous clients,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol.36, 2022, pp. 8432–8440.
  58. H.-Y. Chen and W.-L. Chao, “On bridging generic and personalized federated learning for image classification,” in Proceedings of the 9th International Conference on Learning Representations, 2021.
  59. X.-C. Li and D.-C. Zhan, “FedRS: Federated learning with restricted softmax for label distribution non-iid data,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021, pp. 995–1005.
  60. B. Sarwar, G. Karypis, J. Konstan, and J. Riedl, “Item-based collaborative filtering recommendation algorithms,” in Proceedings of the 10th international conference on World Wide Web, 2001, pp. 285–295.
  61. A. Krizhevsky, G. Hinton, “Learning multiple layers of features from tiny images,” Master’s thesis, Dept. Comput. Sci., Univ. Toronto, Toronto, ON, 2009.
  62. X. Shang, Y. Lu, G. Huang, and H. Wang, “Federated learning on heterogeneous and long-tailed data via classifier re-training with federated features,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 2022, pp. 2218–2224.
  63. I. Achituve, A. Shamsian, A. Navon, G. Chechik, and E. Fetaya, “Personalized federated learning with gaussian processes,” in Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 8392–8406.
  64. T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” in Advances in Neural Information Processing Systems, vol.33, 2020, pp. 2351–2363.
  65. Z. Shen, J. Cervino, H. Hassani, and A. Ribeiro, “An agnostic approach to federated learning with class imbalance,” in Proceedings of the 10th International Conference on Learning Representations, 2022.
  66. M. Yurochkin, M. Agarwal, S. Ghosh, K. H. Greenewald, T. N. Hoang, Y. Khazaeni, “ Bayesian nonparametric federated learning of neural networks,” in Proceedings of the 36th International Conference on Machine Learning, 2019, pp. 7252-7261.
  67. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proceedings of Machine Learning and Systems, 2020, pp. 429–450.
  68. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smithy, “Feddane: A federated newton-type method,” in Proceedings of the 53rd Asilomar Conference on Signals, Systems, and Computers, 2019, pp. 1227–1231.
  69. Y. J. Cho, A. Manoel, G. Joshi, R. Sim, and D. Dimitriadis, “Heterogeneous ensemble knowledge transfer for training large models in federated learning,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 2022, 2881–2887.
  70. H. Wang, Y. Li, W. Xu, R. Li, Y. Zhan and Z. Zeng, “DaFKD: Domain-aware federated knowledge distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, 20412–20421.
  71. Z. Zhu, J. Hong, S. Drew and J. Zhou, “Resilient and communication efficient learning for heterogeneous federated systems,” in Proceedings of the 39th International Conference on Machine Learning, 2022, 27504–27526.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Xu Yang (222 papers)
  2. Jiyuan Feng (3 papers)
  3. Songyue Guo (1 paper)
  4. Ye Wang (248 papers)
  5. Ye Ding (8 papers)
  6. Binxing Fang (16 papers)
  7. Qing Liao (42 papers)
Citations (1)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets