Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-Criteria Client Selection and Scheduling with Fairness Guarantee for Federated Learning Service (2312.14941v1)

Published 5 Dec 2023 in cs.DC and cs.LG

Abstract: Federated Learning (FL) enables multiple clients to train machine learning models collaboratively without sharing the raw training data. However, for a given FL task, how to select a group of appropriate clients fairly becomes a challenging problem due to budget restrictions and client heterogeneity. In this paper, we propose a multi-criteria client selection and scheduling scheme with a fairness guarantee, comprising two stages: 1) preliminary client pool selection, and 2) per-round client scheduling. Specifically, we first define a client selection metric informed by several criteria, such as client resources, data quality, and client behaviors. Then, we formulate the initial client pool selection problem into an optimization problem that aims to maximize the overall scores of selected clients within a given budget and propose a greedy algorithm to solve it. To guarantee fairness, we further formulate the per-round client scheduling problem and propose a heuristic algorithm to divide the client pool into several subsets such that every client is selected at least once while guaranteeing that the `integrated' dataset in a subset is close to an independent and identical distribution (iid). Our experimental results show that our scheme can improve the model quality especially when data are non-iid.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. S. AbdulRahman, H. Tout, A. Mourad, and C. Talhi, “Fedmccs: Multicriteria client selection model for optimal iot federated learning,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4723–4735, 2020.
  2. D. Babaev, M. Savchenko, A. Tuzhilin, and D. Umerenkov, “Et-rnn: Applying deep learning to credit loan applications,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 2183–2190.
  3. V. Cacchiani, M. Iori, A. Locatelli, and S. Martello, “Knapsack problems—an overview of recent advances. part ii: Multiple, multidimensional, and quadratic knapsack problems,” Computers & Operations Research, vol. 143, p. 105693, 2022.
  4. J. M. Calvin and J. Y.-T. Leung, “Average-case analysis of a greedy algorithm for the 0/1 knapsack problem,” Operations Research Letters, vol. 31, no. 3, pp. 202–210, 2003.
  5. A. M. Chekroud, R. J. Zotti, Z. Shehzad, R. Gueorguieva, M. K. Johnson, M. H. Trivedi, T. D. Cannon, J. H. Krystal, and P. R. Corlett, “Cross-trial prediction of treatment outcome in depression: a machine learning approach,” The Lancet Psychiatry, vol. 3, no. 3, pp. 243–250, 2016.
  6. W. Chen, S. Horvath, and P. Richtarik, “Optimal client sampling for federated learning,” arXiv preprint arXiv:2010.13723, 2020.
  7. Y. J. Cho, J. Wang, and G. Joshi, “Towards understanding biased client selection in federated learning,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 10 351–10 375.
  8. P. C. Chu and J. E. Beasley, “A genetic algorithm for the multidimensional knapsack problem,” Journal of heuristics, vol. 4, pp. 63–86, 1998.
  9. U. Fiore, A. De Santis, F. Perla, P. Zanetti, and F. Palmieri, “Using generative adversarial networks for improving classification effectiveness in credit card fraud detection,” Information Sciences, vol. 479, pp. 448–455, 2019.
  10. Y. Fraboni, R. Vidal, L. Kameni, and M. Lorenzi, “Clustered sampling: Low-variance and improved representativity for clients selection in federated learning,” in International Conference on Machine Learning.   PMLR, 2021, pp. 3407–3416.
  11. M. Hino, E. Benami, and N. Brooks, “Machine learning for environmental monitoring,” Nature Sustainability, vol. 1, no. 10, pp. 583–588, 2018.
  12. T. Huang, W. Lin, W. Wu, L. He, K. Li, and A. Y. Zomaya, “An efficiency-boosting client selection scheme for federated learning with fairness guarantee,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 7, pp. 1552–1564, 2020.
  13. A. R. Jadhav, A. Portnoy, and J. Tursunboev, “Federated-learning-pytorch,” https://github.com/AshwinRJ/Federated-Learning-PyTorch, 2019.
  14. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al., “Advances and open problems in federated learning,” arXiv preprint arXiv:1912.04977, 2019.
  15. N. Kourtellis, K. Katevas, and D. Perino, “Flaas: Federated learning as a service,” in Proceedings of the 1st workshop on distributed machine learning, 2020, pp. 7–13.
  16. J. Lee, H. Ko, S. Seo, and S. Pack, “Data distribution-aware online client selection algorithm for federated learning in heterogeneous networks,” IEEE Transactions on Vehicular Technology, 2022.
  17. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems, vol. 2, pp. 429–450, 2020.
  18. Z. Li, Y. He, H. Yu, J. Kang, X. Li, Z. Xu, and D. Niyato, “Data heterogeneity-robust federated learning via group client selection in industrial iot,” IEEE Internet of Things Journal, 2022.
  19. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics.   PMLR, 2017, pp. 1273–1282.
  20. X. Meng, J. Bradley, B. Yavuz, E. Sparks, S. Venkataraman, D. Liu, J. Freeman, D. Tsai, M. Amde, S. Owen et al., “Mllib: Machine learning in apache spark,” The Journal of Machine Learning Research, vol. 17, no. 1, pp. 1235–1241, 2016.
  21. A. Najjar, S. Kaneko, and Y. Miyanaga, “Combining satellite imagery and open data to map road safety,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017.
  22. T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” in ICC 2019-2019 IEEE International Conference on Communications (ICC).   IEEE, 2019, pp. 1–7.
  23. S. Perveen, M. Shahbaz, K. Keshavjee, and A. Guergachi, “Prognostic modeling and prevention of diabetes using machine learning technique,” Scientific reports, vol. 9, no. 1, pp. 1–9, 2019.
  24. Z. Qu, R. Duan, L. Chen, J. Xu, Z. Lu, and Y. Liu, “Context-aware online client selection for hierarchical federated learning,” IEEE Transactions on Parallel and Distributed Systems, 2022.
  25. A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, “Deep reinforcement learning framework for autonomous driving,” Electronic Imaging, vol. 2017, no. 19, pp. 70–76, 2017.
  26. H. Shi, M. Xu, and R. Li, “Deep learning for household load forecasting—a novel pooling deep rnn,” IEEE Transactions on Smart Grid, vol. 9, no. 5, pp. 5271–5280, 2017.
  27. M. J. Varnamkhasti, “Overview of the algorithms for solving the multidimensional knapsack problems,” Advanced Studies in Biology, vol. 4, no. 1, pp. 37–47, 2012.
  28. H. Wang, Z. Kaplan, D. Niu, and B. Li, “Optimizing federated learning on non-iid data with reinforcement learning,” in IEEE INFOCOM 2020-IEEE Conference on Computer Communications.   IEEE, 2020, pp. 1698–1707.
  29. J. Wolfrath, N. Sreekumar, D. Kumar, Y. Wang, and A. Chandra, “Haccs: Heterogeneity-aware clustered client selection for accelerated federated learning,” in IEEE IPDPS, 2022.
  30. W. Zhang, X. Wang, P. Zhou, W. Wu, and X. Zhang, “Client selection for federated learning with non-iid data in mobile edge computing,” IEEE Access, vol. 9, pp. 24 462–24 474, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now