2000 character limit reached
Boosting Fairness and Robustness in Over-the-Air Federated Learning
Published 7 Mar 2024 in cs.LG and cs.CY | (2403.04431v1)
Abstract: Over-the-Air Computation is a beyond-5G communication strategy that has recently been shown to be useful for the decentralized training of machine learning models due to its efficiency. In this paper, we propose an Over-the-Air federated learning algorithm that aims to provide fairness and robustness through minmax optimization. By using the epigraph form of the problem at hand, we show that the proposed algorithm converges to the optimal solution of the minmax problem. Moreover, the proposed approach does not require reconstructing channel coefficients by complex encoding-decoding schemes as opposed to state-of-the-art approaches. This improves both efficiency and privacy.
- B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–1282.
- V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in neural information processing systems, vol. 30, 2017.
- K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Konečnỳ, S. Mazzocchi, B. McMahan et al., “Towards federated learning at scale: System design,” Proceedings of Machine Learning and Systems, vol. 1, pp. 374–388, 2019.
- Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
- A. Nedic, “Distributed gradient methods for convex machine learning problems in networks: Distributed optimization,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 92–101, 2020.
- V. P. Chellapandi, A. Upadhyay, A. Hashemi, and S. H. Żak, “On the convergence of decentralized federated learning under imperfect information sharing,” IEEE Control Systems Letters, 2023.
- T. Omori and K. Kashima, “Combinatorial optimization approach to client scheduling for federated learning,” IEEE Control Systems Letters, 2023.
- T. Sery, N. Shlezinger, K. Cohen, and Y. C. Eldar, “Over-the-air federated learning from heterogeneous data,” IEEE Transactions on Signal Processing, vol. 69, pp. 3796–3811, 2021.
- T. Gafni, N. Shlezinger, K. Cohen, Y. C. Eldar, and H. V. Poor, “Federated learning: A signal processing perspective,” IEEE Signal Processing Magazine, vol. 39, no. 3, pp. 14–41, 2022.
- M. Ye, X. Fang, B. Du, P. C. Yuen, and D. Tao, “Heterogeneous federated learning: State-of-the-art and research challenges,” ACM Computing Surveys, vol. 56, no. 3, pp. 1–44, 2023.
- J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.
- T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020.
- P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
- H. Y. Oksuz, F. Molinari, H. Sprekeler, and J. Raisch, “Federated learning in wireless networks via over-the-air computations,” in 2023 62nd IEEE Conference on Decision and Control (CDC), 2023, pp. 4379–4386.
- F. Molinari, N. Agrawal, S. Stańczak, and J. Raisch, “Max-consensus over fading wireless channels,” IEEE Transactions on Control of Network Systems, vol. 8, no. 2, pp. 791–802, 2021.
- M. Frey, I. Bjelaković, and S. Stańczak, “Over-the-air computation in correlated channels,” IEEE Transactions on Signal Processing, vol. 69, pp. 5739–5755, 2021.
- K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via over-the-air computation,” IEEE Transactions on Wireless Communications, vol. 19, no. 3, pp. 2022–2035, 2020.
- F. Molinari and J. Raisch, “Exploiting wireless interference for distributively solving linear equations,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 2999–3006, 2020.
- M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” in International Conference on Machine Learning. PMLR, 2019, pp. 4615–4625.
- Z. Hu, K. Shaloudegi, G. Zhang, and Y. Yu, “Federated learning meets multi-objective optimization,” IEEE Transactions on Network Science and Engineering, vol. 9, no. 4, pp. 2039–2051, 2022.
- D. P. Bertsekas, “Necessary and sufficient conditions for a penalty method to be exact,” Mathematical programming, vol. 9, no. 1, pp. 87–99, 1975.
- K. Srivastava, A. Nedić, and D. Stipanović, “Distributed min-max optimization in networks,” in 2011 17th International Conference on Digital Signal Processing (DSP). IEEE, 2011, pp. 1–8.
- R. Ahlswede, “Multi-way communication channels,” in Second International Symposium on Information Theory: Tsahkadsor, Armenia, USSR, Sept. 2-8, 1971, 1973.
- A. Giridhar and P. Kumar, “Toward a theory of in-network computation in wireless sensor networks,” IEEE Communications Magazine, vol. 44, no. 4, pp. 98–107, apr 2006.
- F. Molinari, N. Agrawal, S. Stańczak, and J. Raisch, “Over-the-air max-consensus in clustered networks adopting half-duplex communication technology,” IEEE Transactions on Control of Network Systems, 2022.
- B. Sklar, “Rayleigh fading channels in mobile digital communication systems. i. characterization,” IEEE Communications magazine, vol. 35, no. 7, pp. 90–100, 1997.
- Y. I. Alber, A. N. Iusem, and M. V. Solodov, “On the projected subgradient method for nonsmooth convex optimization in a hilbert space,” Mathematical Programming, vol. 81, pp. 23–35, 1998.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
