Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Mobility Accelerates Learning: Convergence Analysis on Hierarchical Federated Learning in Vehicular Networks (2401.09656v1)

Published 18 Jan 2024 in cs.LG, cs.AI, and cs.DC

Abstract: Hierarchical federated learning (HFL) enables distributed training of models across multiple devices with the help of several edge servers and a cloud edge server in a privacy-preserving manner. In this paper, we consider HFL with highly mobile devices, mainly targeting at vehicular networks. Through convergence analysis, we show that mobility influences the convergence speed by both fusing the edge data and shuffling the edge models. While mobility is usually considered as a challenge from the perspective of communication, we prove that it increases the convergence speed of HFL with edge-level heterogeneous data, since more diverse data can be incorporated. Furthermore, we demonstrate that a higher speed leads to faster convergence, since it accelerates the fusion of data. Simulation results show that mobility increases the model accuracy of HFL by up to 15.1% when training a convolutional neural network on the CIFAR-10 dataset.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. A. M. Elbir, B. Soner, S. Çöleri, D. Gündüz, and M. Bennis, “Federated learning in vehicular networks,” in IEEE Int. Mediterranean Conf. on Communications and Networking (MeditCom), Athens, Greece, Sep. 2022, pp. 72–77.
  2. Y. Sun, B. Xie, S. Zhou, and Z. Niu, “MEET: Mobility-Enhanced Edge inTelligence for Smart and Green 6G Networks,” IEEE Commun. Mag., vol. 61, no. 1, pp. 64–70, Jan. 2023.
  3. K. Tan, D. Bremner, J. Le Kernec, and M. Imran, “Federated machine learning in vehicular networks: A summary of recent applications,” in Int. Conf. on UK-China Emerging Technologies (UCET), Glasgow, UK, Aug. 2020, pp. 1–4.
  4. L. Liang, H. Ye, and G. Y. Li, “Toward intelligent vehicular networks: A machine learning framework,” IEEE Internet Things J., vol. 6, no. 1, pp. 124–135, Feb. 2019.
  5. D. Ye, R. Yu, M. Pan, and Z. Han, “Federated learning in vehicular edge computing: A selective model aggregation approach,” IEEE Access, vol. 8, pp. 23920–23935, 2020.
  6. J. Posner, L. Tseng, M. Aloqaily, and Y. Jararweh, “Federated learning in vehicular networks: Opportunities and solutions,” IEEE Netw., vol. 35, no. 2, pp. 152–159, Mar. 2021.
  7. H. Zhou, Y. Zheng, H. Huang, J. Shu, and X. Jia, “Toward robust hierarchical federated learning in Internet of vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 5, pp. 5600–5614, May 2023.
  8. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. Artificial Intelligence and Statistics (AIStats), Apr. 2017.
  9. Z. Yu, J. Hu, G. Min, Z. Zhao, W. Miao, and M. S. Hossain, “Mobility-aware proactive edge caching for connected vehicles using federated learning,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 8, pp. 5341–5351, Aug. 2021.
  10. S. Samarakoon, M. Bennis, W. Saad, and M. Debbah, “Distributed federated learning for ultra-reliable low-latency vehicular communications,” IEEE Trans. Commun., vol. 68, no. 2, pp. 1146–1159, Feb. 2020.
  11. F. Yin, Z. Lin, Q. Kong, Y. Xu, D. Li, S. Theodoridis, and S. R. Cui, “Fedloc: Federated learning framework for data-driven cooperative localization and location data processing,” IEEE Open J. Signal Process., vol. 1, pp. 187–215, Nov. 2020.
  12. W. Y. B. Lim, J. S. Ng, Z. Xiong, D. Niyato, C. Miao and D. I. Kim, “Dynamic Edge Association and Resource Allocation in Self-Organizing Hierarchical Federated Learning Networks,” in IEEE J. Sel. Areas Commun., vol. 39, no. 12, pp. 3640-3653, Dec. 2021.
  13. Y. Sun, W. Shi, X. Huang, S. Zhou and Z. Niu, “Edge learning with timeliness constraints: Challenges and solutions,” IEEE Commun. Mag., vol. 58, no. 12, pp. 27–33, Dec. 2020.
  14. D. Gündüz, D. B. Kurka, M. Jankowski, M. M. Amiri, E. Ozfatura, and S. Sreekumar, “Communicate to learn at the edge,” IEEE Commun. Mag., vol. 58, no. 12, pp. 14–19, Dec. 2020.
  15. Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-efficient edge AI: Algorithms and systems,” IEEE Commun. Surveys & Tutorials, vol. 22, no. 4, pp. 2167–2191, 2020.
  16. L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Client-edge-cloud hierarchical federated learning,” in Proc. IEEE Int. Conf. Commun. (ICC), Dublin, Ireland, Jun. 2020, pp. 1–6.
  17. M. S. H. Abad, E. Ozfatura, D. Gunduz, and O. Ercetin, “Hierarchical federated learning across heterogeneous cellular networks,” in IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, May 2020, pp. 8866–8870.
  18. G. Ayache, V. Dassari, and S. El Rouayheb, “Walk for learning: A random walk approach for federated learning from heterogeneous data,” IEEE J. Sel. Areas Commun., vol. 41, no. 4, pp. 929–940, Apr. 2023.
  19. W. Shi, S. Zhou, Z. Niu, M. Jiang, and L. Geng, “Joint device scheduling and resource allocation for latency constrained wireless federated learning,” IEEE Trans. Wireless Commun., vol. 20, no. 1, pp. 453–467, Jan. 2021.
  20. L. Wang, S. Xu, X. Wang, and Q. Zhu, “Addressing class imbalance in federated learning,” in Proc. AAAI Conf. Artif. Intell., Feb. 2021, vol. 35, pp. 10165–10173.
  21. C. Feng, H. H. Yang, D. Hu, Z. Zhao, T. Q. Quek, and G. Min, “Mobility-aware cluster federated learning in hierarchical wireless networks,” IEEE Trans. Wireless Commun., vol. 21, no. 10, pp. 8441–8458, Oct. 2022.
  22. R. C. Shah, S. Roy, S. Jain, and W. Brunette, “Data MULEs: Modeling and analysis of a three-tier architecture for sparse sensor networks,” Ad Hoc Networks, vol. 1, no. 2-3, pp. 215–233, Sep. 2003.
  23. X. Ma, J. Zhu, Z. Lin, S. Chen, and Y. Qin, “A state-of-the-art survey on solving non-iid data in federated learning,” Future Gener. Comput. Syst., vol. 135, pp. 244–258, Oct. 2022.
  24. H. Zhu, J. Xu, S. Liu, and Y. Jin, “Federated learning on non-iid data: A survey,” Neurocomputing, vol. 465, pp. 371–390, Nov. 2021.
  25. Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” arXiv preprint arXiv:1806.00582, 2018.
  26. E. Jeong, S. Oh, H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Communication-efficient on-device machine learning: Federated distillation and augmentation under non-iid private data,” in Neural Inform. Process. Syst. Workshop, Montréal, Canada, Dec. 2018.
  27. S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” in Int. Conf. on Machine Learning, Jul. 2020, pp. 5132–5143.
  28. T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” Proc. Mach. Learn. Syst. (MLSys), vol. 2, pp. 429–450, Mar. 2020.
  29. S. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečnỳ, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in Int. Conf. on Learning Representations (ICLR), May 2021.
  30. V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in Neural Inform. Process. Syst., Long Beach, CA, USA, Dec. 2017, pp. 4424–-4434.
  31. M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, “Federated learning with personalization layers,” arXiv preprint arXiv:1912.00818, 2019.
  32. A. Ghosh, J. Chung, D. Yin, and K. Ramchandran, “An efficient framework for clustered federated learning,” IEEE Trans. Inf. Theory, vol. 68, no. 12, pp. 8076-8091, Dec. 2022.
  33. D. Wang, N. Zhang, and M. Tao, “Adaptive clustering-based model aggregation for federated learning with imbalanced data,” in Int. Workshop on Signal Process. Advances in Wireless Commun. (SPAWC), Lucca, Italy, Nov. 2021, pp. 591–595.
  34. J. Zhang, Z. Li, B. Li, J. Xu, S. Wu, S. Ding, and C. Wu, “Federated learning with label distribution skew via logits calibration,” in Int. Conf. on Machine Learning, Baltimore, MD, USA, Jul. 2022, pp. 26311–26329.
  35. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  36. M. Duan, D. Liu, X. Chen, Y. Tan, J. Ren, L. Qiao, and L. Liang, “Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications,” in Int. Conf. on Comput. Design (ICCD), Washington DC, USA, Nov. 2019, pp. 246–254.
  37. M. Luo, F. Chen, D. Hu, Y. Zhang, J. Liang, and J. Feng, “No fear of heterogeneity: Classifier calibration for federated learning with non-iid data,” Advances in Neural Inform. Process. Syst., Dec. 2021, pp. 5972–5984.
  38. Y. Deng, F. Lyu, J. Ren, Y. Zhang, Y. Zhou, Y. Zhang, and Y. Yang, “Share: Shaping data distribution at edge for communication-efficient hierarchical federated learning,” in Int. Conf. on Distributed Comput. Syst. (ICDCS), Jul. 2021, pp. 24–34.
  39. S. Liu, G. Yu, X. Chen, and M. Bennis, “Joint user association and resource allocation for wireless hierarchical federated learning with iid and non-iid data,” IEEE Trans. Wireless Commun., vol. 21, no. 10, pp. 7852–7866, Apr. 2022.
  40. S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1205–1221, Jun. 2019.
  41. X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” Proc. of the Int. Conf. on Learning Representations, New Orleans, LA, USA, May 2019.
  42. J. Wang, S. Wang, R.-R. Chen, and M. Ji, “Demystifying why local aggregation helps: Convergence analysis of hierarchical sgd,” in Proc. of the AAAI Conf. on Artificial Intell., vol. 36, pp. 8548–8556, 2022.
  43. L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Hierarchical federated learning with quantization: Convergence analysis and system design,” IEEE Trans. Wireless Commun., vol. 22, no. 1, pp. 2-18, Jan. 2023.
  44. Y. Cai, W. Xi, Y. Shen, Y. Peng, S. Song, and J. Zhao, “High-efficient hierarchical federated learning on non-iid data with progressive collaboration,” Future Generation Comput. Systems, vol. 137, pp. 111–128, 2022.
  45. Q. Wu, X. Chen, T. Ouyang, Z. Zhou, X. Zhang, S. Yang, and J. Zhang, “Hiflash: Communication-efficient hierarchical federated learning with adaptive staleness control and heterogeneity-aware client-edge association,” IEEE Trans. Parallel Distrib. Syst., vol. 34, no. 5, pp. 1560-1579, May 2023.
  46. R. M. Gray et al., “Toeplitz and circulant matrices: A review,” Found. and Trends® in Comm. and Inform. Theory, vol. 2, no. 3, pp. 155–239, 2006.
  47. X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “Fedbn: Federated learning on non-iid features via local batch normalization,” in Int. Conf. on Learning Representations (ICLR), May 2021.
  48. J. S. Rosenthal, “Convergence rates for markov chains,” Siam Review, vol. 37, no. 3, pp. 387–405, 1995.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Tan Chen (17 papers)
  2. Jintao Yan (6 papers)
  3. Yuxuan Sun (79 papers)
  4. Sheng Zhou (186 papers)
  5. Zhisheng Niu (97 papers)
  6. Deniz Gündüz (144 papers)
Citations (3)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets