Papers
Topics
Authors
Recent
Search
2000 character limit reached

Exploring Age-of-Information Weighting in Federated Learning under Data Heterogeneity

Published 24 May 2024 in eess.SP | (2405.15978v1)

Abstract: This paper investigates federated learning in a wireless communication system, where random device selection is employed with non-independent and identically distributed (non-IID) data. The analysis indicates that while training deep learning networks using federated stochastic gradient descent (FedSGD) on non-IID datasets, device selection can generate gradient errors that accumulate, leading to potential weight divergence. To mitigate training divergence, we design an age-weighted FedSGD to scale local gradients according to the previous state of devices. To further improve learning performance by increasing device participation under the maximum time consumption constraint, we formulate an energy consumption minimization problem by including resource allocation and sub-channel assignment. By transforming the resource allocation problem into convex and utilizing KKT conditions, we derived the optimal resource allocation solution. Moreover, this paper develops a matching based algorithm to generate the enhanced sub-channel assignment. Simulation results indicate that i) age-weighted FedSGD is able to outperform conventional FedSGD in terms of convergence rate and achievable accuracy, and ii) the proposed resource allocation and sub-channel assignment strategies can significantly reduce energy consumption and improve learning performance by increasing the number of selected devices.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. M. Chen, H. V. Poor, W. Saad, and S. Cui, “Wireless communications for collaborative federated learning,” IEEE Commun. Mag., vol. 58, no. 12, pp. 48–54, 2020.
  2. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  3. J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.
  4. S. Savazzi, M. Nicoli, M. Bennis, S. Kianoush, and L. Barbieri, “Opportunities of federated learning in connected, cooperative, and automated industrial systems,” IEEE Commun. Mag., vol. 59, no. 2, pp. 16–21, 2021.
  5. M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Distributed learning in wireless networks: Recent progress and future challenges,” IEEE J. Sel. Areas Commun., vol. 39, no. 12, pp. 3579–3605, 2021.
  6. Z. Yang, M. Chen, K.-K. Wong, H. V. Poor, and S. Cui, “Federated learning for 6G: Applications, challenges, and opportunities,” Engineering, vol. 8, pp. 33–41, 2022.
  7. T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” in ICC 2019 - 2019 IEEE International Commun. Conf. (ICC), 2019, pp. 1–7.
  8. L. Fu, H. Zhang, G. Gao, M. Zhang, and X. Liu, “Client selection in federated learning: Principles, challenges, and opportunities,” IEEE Internet of Things Journal, vol. 10, no. 24, pp. 21 811–21 819, 2023.
  9. K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via over-the-air computation,” IEEE Trans. Wireless Commun., vol. 19, no. 3, pp. 2022–2035, 2020.
  10. M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Trans. Wireless Commun., vol. 20, no. 1, 2021.
  11. W. Guo, R. Li, C. Huang, X. Qin, K. Shen, and W. Zhang, “Joint device selection and power control for wireless federated learning,” IEEE J. Sel. Areas Commun., vol. 40, no. 8, pp. 2395–2410, 2022.
  12. M. Ribero, H. Vikalo, and G. de Veciana, “Federated learning under intermittent client availability and time-varying communication constraints,” IEEE J. Sel. Top. Signal Process., vol. 17, no. 1, pp. 98–111, 2023.
  13. J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527, 2016.
  14. T. Gafni, N. Shlezinger, K. Cohen, Y. C. Eldar, and H. V. Poor, “Federated learning: A signal processing perspective,” IEEE Signal Process. Mag., vol. 39, no. 3, pp. 14–41, 2022.
  15. Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582, 2018.
  16. M. M. Amiri, D. Gündüz, S. R. Kulkarni, and H. V. Poor, “Convergence of update aware device scheduling for federated learning at the wireless edge,” IEEE Trans. Wireless Commun., vol. 20, no. 6, pp. 3643–3658, 2021.
  17. B. Luo, W. Xiao, S. Wang, J. Huang, and L. Tassiulas, “Tackling system and statistical heterogeneity for federated learning with adaptive client sampling,” in IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, 2022, pp. 1739–1748.
  18. Y. J. Cho, J. Wang, and G. Joshi, “Towards understanding biased client selection in federated learning,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 10 351–10 375.
  19. K. Wang, Z. Ding, D. K. C. So, and Z. Ding, “Age-of-information minimization in federated learning based networks with Non-IID dataset,” IEEE Trans. Wireless Commun., pp. 1–1, 2024.
  20. T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Process. Mag., vol. 37, no. 3, pp. 50–60, 2020.
  21. Z. Yang, M. Chen, W. Saad, C. S. Hong, and M. Shikh-Bahaei, “Energy efficient federated learning over wireless communication networks,” IEEE Trans. Wireless Commun., vol. 20, no. 3, pp. 1935–1949, 2021.
  22. T. Wang, N. Huang, Y. Wu, and T. Q. S. Quek, “Energy-efficient wireless federated learning: A secrecy oriented design via sequential artificial jamming,” IEEE Trans. Veh. Technol., vol. 72, no. 5, pp. 6412–6427, 2023.
  23. M. Alishahi, P. Fortier, W. Hao, X. Li, and M. Zeng, “Energy minimization for wireless-powered federated learning network with NOMA,” IEEE Wireless Commun. Lett., vol. 12, no. 5, pp. 833–837, 2023.
  24. Y. Ren, C. Wu, and D. K. So, “Joint edge association and aggregation frequency for energy-efficient hierarchical federated learning by deep reinforcement learning,” in ICC 2023 - IEEE International Commun. Conf. (ICC), 2023, pp. 3639–3645.
  25. Y. Lin, K. Wang, and Z. Ding, “Rethinking clustered federated learning in NOMA enhanced wireless networks,” arXiv preprint arXiv:2403.03157, 2024.
  26. E. Rizk, S. Vlaski, and A. H. Sayed, “Federated learning under importance sampling,” IEEE Trans. Signal Process., vol. 70, pp. 5381–5396, 2022.
  27. H. H. Yang, A. Arafa, T. Q. S. Quek, and H. Vincent Poor, “Age-based scheduling policy for federated learning in mobile edge networks,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 8743–8747.
  28. K. Wang, Y. Ma, M. B. Mashhadi, C. H. Foh, R. Tafazolli, and Z. Ding, “Age of information in federated learning over wireless networks,” arXiv preprint arXiv:2209.06623, 2022.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free