Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inference Load-Aware Orchestration for Hierarchical Federated Learning (2407.16836v1)

Published 23 Jul 2024 in cs.DC

Abstract: Hierarchical federated learning (HFL) designs introduce intermediate aggregator nodes between clients and the global federated learning server in order to reduce communication costs and distribute server load. One side effect is that machine learning model replication at scale comes "for free" as part of the HFL process: model replicas are hosted at the client end, intermediate nodes, and the global server level and are readily available for serving inference requests. This creates opportunities for efficient model serving but simultaneously couples the training and serving processes and calls for their joint orchestration. This is particularly important for continual learning, where serving a model while (re)training it periodically, upon specific triggers, or continuously, takes place over shared infrastructure spanning the computing continuum. Consequently, training and inference workloads can interfere with detrimental effects on performance. To address this issue, we propose an inference load-aware HFL orchestration scheme, which makes informed decisions on HFL configuration, considering knowledge about inference workloads and the respective processing capacity. Applying our scheme to a continual learning use case in the transportation domain, we demonstrate that by optimizing aggregator node placement and device-aggregator association, significant inference latency savings can be achieved while communication costs are drastically reduced compared to flat centralized federated learning.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, “Edge intelligence: The confluence of edge computing and artificial intelligence,” IEEE Internet Things J., vol. 7, no. 8, pp. 7457–7469, 2020.
  2. W. Y. B. Lim, J. S. Ng, Z. Xiong, J. Jin, Y. Zhang, D. Niyato, C. Leung, and C. Miao, “Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning,” IEEE Trans. Parallel Distrib. Syst., vol. 33, no. 3, pp. 536–550, 2022.
  3. A. Furtuanpey, P. Raith, and S. Dustdar, “Frankensplit: Efficient neural feature compression with shallow variational bottleneck injection for mobile edge computing,” IEEE Trans. Mobile Comput., pp. 1–17, 2024.
  4. J. Wang, Y. Hong, J. Wang, J. Xu, Y. Tang, Q.-L. Han, and J. Kurths, “Cooperative and competitive multi-agent systems: From optimization to games,” IEEE/CAA J. Autom. Sinica, vol. 9, no. 5, pp. 763–783, 2022.
  5. K. Wong, M. Nguyen-Duc, K. Le-Huy, L. Ho-Tuan, C. Do-Danh, and D. L. Phuoc, “An empirical study of federated learning on iot-edge devices: Resource allocation and heterogeneity,” CoRR, vol. abs/2305.19831, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2305.19831
  6. L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Client-edge-cloud hierarchical federated learning,” in Proc. IEEE ICC, 2020.
  7. M. S. H. Abad, E. Ozfatura, D. Gündüz, and Ö. Erçetin, “Hierarchical federated learning ACROSS heterogeneous cellular networks,” in Proc. IEEE ICASSP, 2020.
  8. Z. Wang, H. Xu, J. Liu, H. Huang, C. Qiao, and Y. Zhao, “Resource-efficient federated learning with hierarchical aggregation in edge computing,” in Proc. IEEE INFOCOM, 2021.
  9. Z. Xu, D. Zhao, W. Liang, O. F. Rana, P. Zhou, M. Li, W. Xu, H. Li, and Q. Xia, “HierFedML: Aggregator placement and UE assignment for hierarchical federated learning in mobile edge computing,” IEEE Trans. Parallel Distributed Syst., vol. 34, no. 1, pp. 328–345, 2023.
  10. Q. Wu, X. Chen, T. Ouyang, Z. Zhou, X. Zhang, S. Yang, and J. Zhang, “HiFlash: Communication-efficient hierarchical federated learning with adaptive staleness control and heterogeneity-aware client-edge association,” IEEE Trans. Parallel Distributed Syst., vol. 34, no. 5, pp. 1560–1579, 2023.
  11. Y. Guo, F. Liu, T. Zhou, Z. Cai, and N. Xiao, “Privacy vs. efficiency: Achieving both through adaptive hierarchical federated learning,” IEEE Trans. Parallel Distributed Syst., vol. 34, no. 4, pp. 1331–1342, 2023.
  12. Y. Sai, X. Wu, J. Jiang, Y. Huang, Q. Yan, Z. Li, and H. Huang, “Optimizing hierarchical federated learning: A reinforcement learning approach,” IEEE Trans. Consum. Electron., 2024, early access.
  13. M. F. Pervej, R. Jin, and H. Dai, “Hierarchical federated learning in wireless networks: Pruning tackles bandwidth scarcity and system heterogeneity,” IEEE Trans. Wireless Commun., 2024, early access.
  14. S. Trindade and N. L. S. da Fonseca, “Client selection in hierarchical federated learning,” IEEE Internet Things J., 2024, early access.
  15. Y. Deng, F. Lyu, T. Xia, Y. Zhou, Y. Zhang, J. Ren, and Y. Yang, “A communication-efficient hierarchical federated learning framework via shaping data distribution at edge,” IEEE/ACM Trans. Netw., vol. 32, no. 3, pp. 2600–2615, 2024.
  16. D. Shenaj, M. Toldo, A. Rigon, and P. Zanuttigh, “Asynchronous federated continual learning,” in Proc IEEE/CVF CVPR Workshops, 2023.
  17. M. F. Criado, F. E. Casado, R. Iglesias, C. V. Regueiro, and S. Barro, “Non-iid data and continual learning processes in federated learning: A long road ahead,” Information Fusion, vol. 88, pp. 263–280, 2022.
  18. P. Han, S. Wang, Y. Jiao, and J. Huang, “Federated learning while providing model as a service: Joint training and inference optimization,” in Proc. IEEE INFOCOM, 2024.
  19. S. Thrun, “A lifelong learning perspective for mobile robot control,” in Intelligent robots and systems.   Elsevier, 1995, pp. 201–214.
  20. L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: Theory, method and application,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 8, pp. 5362–5383, 2024.
  21. M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” in Psychology of learning and motivation.   Elsevier, 1989, vol. 24, pp. 109–165.
  22. Z. Li and D. Hoiem, “Learning without forgetting,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 12, pp. 2935–2947, 2017.
  23. J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences, vol. 114, no. 13, pp. 3521–3526, 2017.
  24. J. Le, X. Lei, N. Mu, H. Zhang, K. Zeng, and X. Liao, “Federated continuous learning with broad network architecture,” IEEE Trans. Cybern., vol. 51, no. 8, pp. 3874–3888, 2021.
  25. X. Yang, H. Yu, X. Gao, H. Wang, J. Zhang, and T. Li, “Federated continual learning via knowledge fusion: A survey,” IEEE Trans. Knowl. Data Eng., vol. 36, no. 8, pp. 3832–3850, 2024.
  26. D. Wen, K.-J. Jeon, and K. Huang, “Federated dropout—a simple approach for enabling federated learning on resource constrained devices,” IEEE Wirel. Commun. Lett., vol. 11, no. 5, pp. 923–927, 2022.
  27. Z. Jiang, Y. Xu, H. Xu, Z. Wang, and C. Qian, “Heterogeneity-aware federated learning with adaptive client selection and gradient compression,” in Proc. IEEE INFOCOM, 2023.
  28. Q. Zhang, Y. Li, D. Zhang, I. Murturi, V. C. Pujol, S. Dustdar, and S. Wang, “Intelligence inference on IoT devices,” in Learning Techniques for the Internet of Things, P. K. Donta, A. Hazra, and L. Lovén, Eds.   Cham: Springer Nature Switzerland, 2024, pp. 171–195.
  29. X. Yuan, J. Chen, J. Yang, N. Zhang, T. Yang, T. Han, and A. Taherkordi, “FedSTN: graph representation driven federated learning for edge computing enabled urban traffic flow prediction,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 8, pp. 8738–8748, 2023.
  30. C. Zhang, L. Cui, S. Yu, and J. James, “A communication-efficient federated learning scheme for iot-based traffic forecasting,” IEEE Internet Things J., vol. 9, no. 14, pp. 11 918–11 931, 2021.
  31. Y. Liu, J. James, J. Kang, D. Niyato, and S. Zhang, “Privacy-preserving traffic flow prediction: A federated learning approach,” IEEE Internet Things J., vol. 7, no. 8, pp. 7751–7763, 2020.
  32. S. Hu, Y. Ye, Q. Hu, X. Liu, S. Cao, H. H. Yang, Y. Shen, P. Angeloudis, L. Parada, and C. Wu, “A federated learning-based framework for ride-sourcing traffic demand prediction,” IEEE Trans. Veh. Technol., vol. 72, no. 11, pp. 14 002–14 015, 2023.
  33. D. B. Shmoys, É. Tardos, and K. I. Aardal, “Approximation algorithms for facility location problems,” in Proc. ACM STOC, 1997.
  34. V. Arya, N. Garg, R. Khandekar, A. Meyerson, K. Munagala, and V. Pandit, “Local search heuristic for k-median and facility location problems,” in Proc. ACM STOC, 2001.
  35. Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” Proc. International Conference on Learning Representations (ICLR), 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Anna Lackinger (4 papers)
  2. Pantelis A. Frangoudis (4 papers)
  3. Ivan Čilić (3 papers)
  4. Alireza Furutanpey (9 papers)
  5. Ilir Murturi (11 papers)
  6. Ivana Podnar Žarko (8 papers)
  7. Schahram Dustdar (72 papers)
Citations (2)
View on Semantic Scholar