Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On-demand Quantization for Green Federated Generative Diffusion in Mobile Edge Networks (2403.04430v1)

Published 7 Mar 2024 in cs.LG, cs.DC, and cs.NI

Abstract: Generative Artificial Intelligence (GAI) shows remarkable productivity and creativity in Mobile Edge Networks, such as the metaverse and the Industrial Internet of Things. Federated learning is a promising technique for effectively training GAI models in mobile edge networks due to its data distribution. However, there is a notable issue with communication consumption when training large GAI models like generative diffusion models in mobile edge networks. Additionally, the substantial energy consumption associated with training diffusion-based models, along with the limited resources of edge devices and complexities of network environments, pose challenges for improving the training efficiency of GAI models. To address this challenge, we propose an on-demand quantized energy-efficient federated diffusion approach for mobile edge networks. Specifically, we first design a dynamic quantized federated diffusion training scheme considering various demands from the edge devices. Then, we study an energy efficiency problem based on specific quantization requirements. Numerical results show that our proposed method significantly reduces system energy consumption and transmitted model size compared to both baseline federated diffusion and fixed quantized federated diffusion methods while effectively maintaining reasonable quality and diversity of generated data.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. B. Mao, Y. Liu, J. Liu, and N. Kato, “AI-assisted edge caching for metaverse of connected and automated vehicles: Proposal, challenges, and future perspectives,” IEEE Vehicular Technology Magazine, vol. 18, no. 4, pp. 66–74, 2023.
  2. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  3. S. Choi, J. Kim, and H. Yeo, “Trajgail: Generating urban vehicle trajectories using generative adversarial imitation learning,” Transportation Research Part C: Emerging Technologies, vol. 128, p. 103091, 2021.
  4. G. Cooper, “Examining science education in chatgpt: An exploratory study of generative artificial intelligence,” Journal of Science Education and Technology, vol. 32, no. 3, pp. 444–452, 2023.
  5. J. Kang, J. Wen, D. Ye, B. Lai, T. Wu, Z. Xiong, J. Nie, D. Niyato, Y. Zhang, and S. Xie, “Blockchain-empowered federated learning for healthcare metaverses: User-centric incentive mechanism with optimal data freshness,” IEEE Transactions on Cognitive Communications and Networking, 2023.
  6. B. Mao, X. Zhou, J. Liu, and N. Kato, “Digital twin satellite networks towards 6G: Motivations, challenges, and future perspectives,” IEEE Network, pp. 1–1, 2023.
  7. J. Kang, J. He, H. Du, Z. Xiong, Z. Yang, X. Huang, and S. Xie, “Adversarial attacks and defenses for semantic communication in vehicular metaverses,” IEEE Wireless Communications, vol. 30, no. 4, pp. 48–55, 2023.
  8. X. Luo, J. Wen, J. Kang, J. Nie, Z. Xiong, Y. Zhang, Z. Yang, and S. Xie, “Privacy attacks and defenses for digital twin migrations in vehicular metaverses,” IEEE Network, 2023.
  9. F. V. S. Jothiraj and A. Mashhadi, “Phoenix: A federated generative diffusion model,” arXiv preprint arXiv:2306.04098, 2023.
  10. M. de Goede, “Training diffusion models with federated learning: A communication-efficient model for cross-silo federated image generation,” 2023.
  11. B. Mao, J. Liu, Y. Wu, and N. Kato, “Security and privacy on 6g network edge: A survey,” IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1095–1127, 2023.
  12. X. Huang, P. Li, H. Du, J. Kang, D. Niyato, D. I. Kim, and Y. Wu, “Federated learning-empowered AI-generated content in wireless networks,” arXiv preprint arXiv:2307.07146, 2023.
  13. J. Wen, J. Kang, M. Xu, H. Du, Z. Xiong, Y. Zhang, and D. Niyato, “Freshness-aware incentive mechanism for mobile AI-generated content (aigc) networks,” in 2023 IEEE/CIC International Conference on Communications in China (ICCC), pp. 1–6, 2023.
  14. Z. Yang, M. Chen, W. Saad, C. S. Hong, and M. Shikh-Bahaei, “Energy efficient federated learning over wireless communication networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp. 1935–1949, 2020.
  15. P. Li, G. Cheng, X. Huang, J. Kang, R. Yu, Y. Wu, M. Pan, and D. Niyato, “Snowball: Energy efficient and accurate federated learning with coarse-to-fine compression over heterogeneous wireless edge devices,” IEEE Transactions on Wireless Communications, 2023.
  16. X. Meng and Y. Kabashima, “Quantized compressed sensing with score-based generative models,” in The Eleventh International Conference on Learning Representations, 2022.
  17. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
  18. R. Chen, L. Li, K. Xue, C. Zhang, M. Pan, and Y. Fang, “Energy efficient federated learning over heterogeneous mobile devices via joint design of weight quantization and wireless transmission,” IEEE Transactions on Mobile Computing, 2022.
  19. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics, pp. 1273–1282, PMLR, 2017.
  20. H. G. Myung, J. Lim, and D. J. Goodman, “Single carrier fdma for uplink wireless transmission,” IEEE vehicular technology magazine, vol. 1, no. 3, pp. 30–38, 2006.
  21. S. P. Boyd and L. Vandenberghe, Convex optimization. Cambridge university press, 2004.
  22. A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009.
  23. S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark,” in International Joint Conference on Neural Networks, no. 1288, 2013.
  24. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Bingkun Lai (3 papers)
  2. Jiayi He (20 papers)
  3. Jiawen Kang (204 papers)
  4. Gaolei Li (29 papers)
  5. Minrui Xu (57 papers)
  6. Shengli Xie (36 papers)
  7. Tao Zhang (481 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now