Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learning (2305.16854v4)

Published 26 May 2023 in cs.LG and cs.DC

Abstract: Federated learning (FL) is a popular privacy-preserving distributed training scheme, where multiple devices collaborate to train machine learning models by uploading local model updates. To improve communication efficiency, over-the-air computation (AirComp) has been applied to FL, which leverages analog modulation to harness the superposition property of radio waves such that numerous devices can upload their model updates concurrently for aggregation. However, the uplink channel noise incurs considerable model aggregation distortion, which is critically determined by the device scheduling and compromises the learned model performance. In this paper, we propose a probabilistic device scheduling framework for over-the-air FL, named PO-FL, to mitigate the negative impact of channel noise, where each device is scheduled according to a certain probability and its model update is reweighted using this probability in aggregation. We prove the unbiasedness of this aggregation scheme and demonstrate the convergence of PO-FL on both convex and non-convex loss functions. Our convergence bounds unveil that the device scheduling affects the learning performance through the communication distortion and global update variance. Based on the convergence analysis, we further develop a channel and gradient-importance aware algorithm to optimize the device scheduling probabilities in PO-FL. Extensive simulation results show that the proposed PO-FL framework with channel and gradient-importance awareness achieves faster convergence and produces better models than baseline methods.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Y. Sun, Z. Lin, Y. Mao, S. Jin, and J. Zhang, “Probabilistic device scheduling for over-the-air federated learning,” in Proc. IEEE Int. Conf. Commun. Tech. (ICCT), Wuxi, China, Oct. 2023.
  2. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. Int. Conf. Artif. Intell. Stat. (AISTATS), Ft. Lauderdale, FL, USA, Apr. 2017.
  3. O. Shahid, S. Pouriyeh, R. M. Parizi, Q. Z. Sheng, G. Srivastava, and L. Zhao, “Communication efficiency in federated learning: Achievements and challenges,” [Online]. Available: https://arxiv.org/pdf/2107.10996.pdf.
  4. W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” IEEE Commun. Surveys Tuts., vol. 22, no. 3, pp. 2031–2063, Apr. 2020.
  5. B. Nazer and M. Gastpar, “Computation over multiple-access channels,” IEEE Trans. Inf. Theory, vol. 53, no. 10, pp. 3498–3516, Oct. 2007.
  6. G. Zhu, Y. Wang, and K. Huang, “Broadband analog aggregation for low-latency federated edge learning,” IEEE Trans. Wireless Commun., vol. 19, no. 1, pp. 491–506, Jan. 2020.
  7. K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via over-the-air computation,” IEEE Trans. Wireless Commun., vol. 19, no. 3, pp. 2022–2035, Mar. 2020.
  8. Z. Lin, H. Liu, and Y.-J. A. Zhang, “Relay-assisted cooperative federated learning,” IEEE Trans. Wireless Commun., vol. 21, no. 9, pp. 7148–7164, Sept. 2022.
  9. T. Sery, N. Shlezinger, K. Cohen, and Y. C. Eldar, “Over-the-air federated learning from heterogeneous data,” IEEE Trans. Signal Process., vol. 69, pp. 3796–3811, Jun. 2021.
  10. X. Fan, Y. Wang, Y. Huo, and Z. Tian, “Joint optimization of communications and federated learning over the air,” IEEE Trans. Wireless Commun., vol. 21, no. 6, pp. 4434–4449, Dec. 2021.
  11. N. Zhang and M. Tao, “Gradient statistics aware power control for over-the-air federated learning,” IEEE Trans. Wireless Commun., vol. 20, no. 8, pp. 5115–5128, Aug. 2021.
  12. X. Cao, G. Zhu, J. Xu, and S. Cui, “Transmission power control for over-the-air federated averaging at network edge,” IEEE J. Sel. Areas Commun., vol. 40, no. 5, pp. 1571–1586, May 2022.
  13. X. Ma, H. Sun, Q. Wang, and R. Q. Hu, “User scheduling for federated learning through over-the-air computation,” in Proc. IEEE Veh. Technol. Conf. (VTC-Fall), Norman, OK, USA, Sept. 2021.
  14. W. Luping, W. Wei, and L. Bo, “CMFL: Mitigating communication overhead for federated learning,” in Proc. Int. Conf. Distrib. Comput. Syst. (ICDCS), Dallas, TX, USA, Jul. 2019.
  15. J. Du, B. Jiang, C. Jiang, Y. Shi, and Z. Han, “Gradient and channel aware dynamic scheduling for over-the-air computation in federated edge learning systems,” IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 1035–1050, Apr. 2023.
  16. L. Su and V. K. Lau, “Data and channel-adaptive sensor scheduling for federated edge learning via over-the-air gradient aggregation,” IEEE Internet Things J., vol. 9, no. 3, pp. 1640–1654, Feb. 2021.
  17. Y. Sun, S. Zhou, Z. Niu, and D. Gündüz, “Dynamic scheduling for over-the-air federated edge learning with energy constraints,” IEEE J. Sel. Areas Commun., vol. 40, no. 1, pp. 227–242, Nov. 2021.
  18. D. C. Attota, V. Mothukuri, R. M. Parizi, and S. Pouriyeh, “An ensemble multi-view federated learning intrusion detection for IoT,” IEEE Access, vol. 9, pp. 117 734–117 745, Aug. 2021.
  19. L. Wang, Y. Guo, T. Lin, and X. Tang, “Client selection in nonconvex federated learning: Improved convergence analysis for optimal unbiased sampling strategy,” [Online]. Available: https://arxiv.org/pdf/2205.13925.pdf.
  20. H. Wu, X. Tang, Y.-J. A. Zhang, and L. Gao, “Incentive mechanism for federated learning based on random client sampling,” in Proc. IEEE Global Commun. Conf. Wkshps. (GLOBECOM Wkshps), Rio de Janeiro, Brazil, Dec. 2022.
  21. J. Ren, Y. He, D. Wen, G. Yu, K. Huang, and D. Guo, “Scheduling for cellular federated edge learning with importance and channel awareness,” IEEE Trans. Wireless Commun., vol. 19, no. 11, pp. 7690–7703, Nov. 2020.
  22. M. Zhang, G. Zhu, S. Wang, J. Jiang, Q. Liao, C. Zhong, and S. Cui, “Communication-efficient federated edge learning via optimal probabilistic device scheduling,” IEEE Trans. Wireless Commun., vol. 21, no. 10, pp. 8536–8551, Oct. 2022.
  23. Y. Sun, J. Shao, Y. Mao, S. Li, and J. Zhang, “Stochastic coded federated learning: Theoretical analysis and incentive mechanism design,” IEEE Trans. Wireless Commun., to appear.
  24. M. M. Amiri, D. Gündüz, S. R. Kulkarni, and H. V. Poor, “Convergence of update aware device scheduling for federated learning at the wireless edge,” IEEE Trans. Wireless Commun., vol. 20, no. 6, pp. 3643–3658, Jun. 2021.
  25. Z. Lin, H. Liu, and Y.-J. A. Zhang, “CFLIT: Coexisting federated learning and information transfer,” IEEE Trans. Wireless Commun., vol. 22, no. 11, pp. 8436–8453, Nov. 2023.
  26. Y. Sun, J. Shao, Y. Mao, J. H. Wang, and J. Zhang, “Semi-decentralized federated edge learning with data and device heterogeneity,” IEEE Trans. Netw. Service Manag., vol. 20, no. 2, pp. 1487–1501, Jun. 2023.
  27. Z. Chen, W. Yi, Y. Liu, and A. Nallanathan, “Knowledge-aided federated learning for energy-limited wireless networks,” IEEE Trans. Wireless Commun., vol. 71, no. 6, pp. 3368–3386, Jun. 2023.
  28. X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of FedAvg on non-IID data,” in Proc. Int. Conf. Learn. Repr. (ICLR), Addis Ababa, Ethiopia, Apr. 2020.
  29. L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale machine learning,” SIAM Rev., vol. 60, no. 2, pp. 223–311, Aug. 2018.
  30. J. Wang and G. Joshi, “Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms,” J. Mach. Learn. Res., vol. 22, no. 1, pp. 9709–9758, Jan. 2021.
  31. A. Barakat and P. Bianchi, “Convergence rates of a momentum algorithm with bounded adaptive step size for nonconvex optimization,” in Proc. Asian Conf. Mach. Learn. (ACML), Bangkok, Thailand, Nov. 2020.
  32. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
  33. A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” [Online]. Available: https://www.cs.toronto.edu/~kriz/cifar.html.
  34. Q. Li, Y. Diao, Q. Chen, and B. He, “Federated learning on non-IID data silos: An experimental study,” [Online]. Available: https://arxiv.org/pdf/2102.02079.pdf.
  35. L. Li, D. Ma, H. Ren, P. Wang, W. Lin, and Z. Han, “Toward energy-efficient multiple IRSs: federated learning-based configuration optimization,” IEEE Trans. Green Commun. Netw., vol. 6, no. 2, pp. 755–765, Jun. 2021.
  36. N. I. Mowla, N. H. Tran, I. Doh, and K. Chae, “Federated learning-based cognitive detection of jamming attack in flying Ad-Hoc network,” IEEE Access, vol. 8, pp. 4338–4350, Dec. 2019.
  37. J. Du, C. Jiang, J. Wang, Y. Ren, and M. Debbah, “Machine learning for 6G wireless networks: Carrying forward enhanced bandwidth, massive access, and ultrareliable/low-latency service,” IEEE Veh. Technol. Mag., vol. 15, no. 4, pp. 122–134, Sept. 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yuchang Sun (17 papers)
  2. Yuyi Mao (44 papers)
  3. Shi Jin (487 papers)
  4. Jun Zhang (1008 papers)
  5. Zehong Lin (21 papers)
Citations (10)
View on Semantic Scholar