Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Communication Efficiency Optimization of Federated Learning for Computing and Network Convergence of 6G Networks (2311.16540v1)

Published 28 Nov 2023 in cs.LG, cs.DC, and cs.NI

Abstract: Federated learning effectively addresses issues such as data privacy by collaborating across participating devices to train global models. However, factors such as network topology and device computing power can affect its training or communication process in complex network environments. A new network architecture and paradigm with computing-measurable, perceptible, distributable, dispatchable, and manageable capabilities, computing and network convergence (CNC) of 6G networks can effectively support federated learning training and improve its communication efficiency. By guiding the participating devices' training in federated learning based on business requirements, resource load, network conditions, and arithmetic power of devices, CNC can reach this goal. In this paper, to improve the communication efficiency of federated learning in complex networks, we study the communication efficiency optimization of federated learning for computing and network convergence of 6G networks, methods that gives decisions on its training process for different network conditions and arithmetic power of participating devices in federated learning. The experiments address two architectures that exist for devices in federated learning and arrange devices to participate in training based on arithmetic power while achieving optimization of communication efficiency in the process of transferring model parameters. The results show that the method we proposed can (1) cope well with complex network situations (2) effectively balance the delay distribution of participating devices for local training (3) improve the communication efficiency during the transfer of model parameters (4) improve the resource utilization in the network.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (20)
  1. P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  2. O. A. Wahab, A. Mourad, H. Otrok, and T. Taleb, “Federated machine learning: Survey, multi-level classification, desirable criteria and future directions in communication and networking systems,” IEEE Communications Surveys & Tutorials, vol. 23, no. 2, pp. 1342–1397, 2021.
  3. T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020.
  4. J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.
  5. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics.   PMLR, 2017, pp. 1273–1282.
  6. Y. Fraboni, R. Vidal, L. Kameni, and M. Lorenzi, “Clustered sampling: Low-variance and improved representativity for clients selection in federated learning,” in International Conference on Machine Learning.   PMLR, 2021, pp. 3407–3416.
  7. W. Wu, L. He, W. Lin, R. Mao, C. Maple, and S. Jarvis, “Safa: A semi-asynchronous protocol for fast federated learning with low overhead,” IEEE Transactions on Computers, vol. 70, no. 5, pp. 655–668, 2020.
  8. H. H. Yang, Z. Liu, T. Q. Quek, and H. V. Poor, “Scheduling policies for federated learning in wireless networks,” IEEE transactions on communications, vol. 68, no. 1, pp. 317–333, 2019.
  9. J. So, B. Güler, and A. S. Avestimehr, “Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 1, pp. 479–489, 2021.
  10. L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Client-edge-cloud hierarchical federated learning,” in ICC 2020-2020 IEEE International Conference on Communications (ICC).   IEEE, 2020, pp. 1–6.
  11. Y. Deng, F. Lyu, J. Ren, Y. Zhang, Y. Zhou, Y. Zhang, and Y. Yang, “Share: Shaping data distribution at edge for communication-efficient hierarchical federated learning,” in 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS).   IEEE, 2021, pp. 24–34.
  12. T. Lin, L. Kong, S. U. Stich, and M. Jaggi, “Ensemble distillation for robust model fusion in federated learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 2351–2363, 2020.
  13. C. He, M. Annavaram, and S. Avestimehr, “Group knowledge transfer: Federated learning of large cnns at the edge,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 068–14 080, 2020.
  14. D. Li and J. Wang, “Fedmd: Heterogenous federated learning via model distillation,” arXiv preprint arXiv:1910.03581, 2019.
  15. C. T. Dinh, N. H. Tran, M. N. Nguyen, C. S. Hong, W. Bao, A. Y. Zomaya, and V. Gramoli, “Federated learning over wireless networks: Convergence analysis and resource allocation,” IEEE/ACM Transactions on Networking, vol. 29, no. 1, pp. 398–409, 2020.
  16. M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 269–283, 2020.
  17. Z. Qin, G. Y. Li, and H. Ye, “Federated learning and wireless communications,” IEEE Wireless Communications, vol. 28, no. 5, pp. 134–140, 2021.
  18. Q. Cao, X. Zhang, Y. Zhang, and Y. Zhu, “Layered model aggregation based federated learning in mobile edge networks,” in 2021 IEEE/CIC International Conference on Communications in China (ICCC).   IEEE, 2021, pp. 1–6.
  19. P. Pinyoanuntapong, P. Janakaraj, P. Wang, M. Lee, and C. Chen, “Fedair: Towards multi-hop federated learning over-the-air,” in 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).   IEEE, 2020, pp. 1–5.
  20. Y. Sun, J. Liu, H. Huang, X. Zhang, B. Lei, J. Peng, and W. Wang, “Computing power network: A survey,” to appear in China Communications, online in arXiv, 2022.

Summary

  • The paper demonstrates enhanced federated learning communication efficiency by optimizing model training based on device heterogeneity and network conditions.
  • It employs traditional and peer-to-peer architectures to significantly reduce transmission latency and energy consumption per training round.
  • The proposed approach leverages real-time resource allocation and data traffic path selection in 6G networks to speed up model convergence.

Federated learning (FL) is a distributed machine learning approach that allows for the training of models across multiple devices while maintaining data privacy. However, this approach faces significant challenges when operating in complex network environments, such as those anticipated in emerging 6G networks. The heterogeneity of device computing power, along with network topology variations, can lead to inefficiencies in both the learning and communication processes.

Addressing this, a paper investigates the communication efficiency of federated learning within the context of computing and network convergence (CNC) for 6G networks. The authors aim to enhance federated learning's communication efficiency by directing model training according to various factors such as business requirements, resource load, network conditions, and the participating devices' computing power.

To delve into the issue, researchers conducted experiments focusing on two architectures: traditional architecture, where devices individually train the model and then send it to a central server for aggregation, and peer-to-peer architecture, where there's no central server, and devices form a chain to pass on and train the model before reaching a consensus.

The experiments utilize a novel approach within 6G networks' CNC, which exploits enhanced resource allocation, data traffic path selections, and real-time information synchronization. The proposed optimization methods proved to improve the communication efficiency by balancing device heterogeneity in computational power and optimizing model parameter transfers.

In practical scenarios, this boosted communication efficiency brings about significant improvements. In traditional federated learning architectures, the system reduced transmission latency and energy consumption per training round significantly when compared to a baseline FedAvg algorithm. Similarly, in peer-to-peer architectures, the newly proposed system showed faster convergence times with similar or improved transmission performance under varying network conditions.

The significance of these findings lies in the potential for federated learning to benefit from 6G network capabilities. As networks evolve, they will provide more than just faster speeds—they will facilitate smarter and more efficient distributed learning frameworks, like the one proposed by the researchers. This speaks to the future of AI and machine learning where vast networks of devices can contribute to a collective intelligence while preserving user privacy and reducing communication demands.

To conclude, the paper presents a fresh perspective that paves the way for more effective federated learning within the advancing landscape of wireless communication. As 6G technology evolves, it could enable a seamless and efficient integration of CNC-supported federated learning, addressing some of the central challenges of machine learning model training across widespread and diverse devices.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets