Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Meta Federated Reinforcement Learning for Distributed Resource Allocation (2307.02900v2)

Published 6 Jul 2023 in eess.SP, cs.SY, and eess.SY

Abstract: In cellular networks, resource allocation is usually performed in a centralized way, which brings huge computation complexity to the base station (BS) and high transmission overhead. This paper explores a distributed resource allocation method that aims to maximize energy efficiency (EE) while ensuring the quality of service (QoS) for users. Specifically, in order to address wireless channel conditions, we propose a robust meta federated reinforcement learning (\textit{MFRL}) framework that allows local users to optimize transmit power and assign channels using locally trained neural network models, so as to offload computational burden from the cloud server to the local users, reducing transmission overhead associated with local channel state information. The BS performs the meta learning procedure to initialize a general global model, enabling rapid adaptation to different environments with improved EE performance. The federated learning technique, based on decentralized reinforcement learning, promotes collaboration and mutual benefits among users. Analysis and numerical results demonstrate that the proposed \textit{MFRL} framework accelerates the reinforcement learning process, decreases transmission overhead, and offloads computation, while outperforming the conventional decentralized reinforcement learning algorithm in terms of convergence speed and EE performance across various scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. Z. Ji and Z. Qin, “Federated learning for distributed energy-efficient resource allocation,” in Proc. IEEE Int. Conf. on Commun., Seoul, Republic of Korea, May 2022, pp. 1–6.
  2. M. Rasti, S. K. Taskou, H. Tabassum, and E. Hossain, “Evolution toward 6g multi-band wireless networks: A resource management perspective,” IEEE Wireless Commun., vol. 29, no. 4, pp. 118–125, Aug. 2022.
  3. L. Liang, G. Y. Li, and W. Xu, “Resource allocation for D2D-enabled vehicular communications,” IEEE Trans. on Commun., vol. 65, no. 7, pp. 3186–3197, Apr. 2017.
  4. J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, and M. Botvinick, “Learning to reinforcement learn,” arXiv preprint arXiv: 1611.05763, Jan. 2016.
  5. Y. Duan, J. Schulman, X. Chen, P. L. Bartlett, I. Sutskever, and P. Abbeel, “Rl22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT: Fast reinforcement learning via slow reinforcement learning,” arXiv preprint arXiv: 1611.02779, Nov. 2016.
  6. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” arXiv preprint arXiv: 1703.03400, Mar. 2017.
  7. J. Kang, Z. Xiong, D. Niyato, Y. Zou, Y. Zhang, and M. Guizani, “Reliable federated learning for mobile networks,” IEEE Wireless Commun., vol. 27, no. 2, pp. 72–80, Apr. 2020.
  8. G. Song and Y. Li, “Utility-based resource allocation and scheduling in OFDM-based wireless broadband networks,” IEEE Commun. Mag., vol. 43, no. 12, pp. 127–134, Dec. 2005.
  9. M. Robat Mili, P. Tehrani, and M. Bennis, “Energy-efficient power allocation in OFDMA D2D communication by multiobjective optimization,” IEEE Wireless. Commun. Lett., vol. 5, no. 6, pp. 668–671, Dec. 2016.
  10. F. Meshkati, H. V. Poor, and S. C. Schwartz, “Energy-efficient resource allocation in wireless networks,” IEEE Signal Processing Mag., vol. 24, no. 3, pp. 58–68, May 2007.
  11. L. Yang, D. Wu, C. Yue, Y. Zhang, and Y. Wu, “Pricing-based channel selection for D2D content sharing in dynamic environments,” IEEE Trans. Wireless Commun., vol. 20, no. 4, pp. 2175–2189, Dec. 2021.
  12. S. Dominic and L. Jacob, “Distributed resource allocation for D2D communications underlaying cellular networks in time-varying environment,” IEEE Commun. Lett., vol. 22, no. 2, pp. 388–391, Nov. 2018.
  13. L. Liang, H. Ye, and G. Y. Li, “Spectrum sharing in vehicular networks based on multi-agent reinforcement learning,” IEEE J. Sel. Areas Commun., vol. 37, no. 10, pp. 2282–2292, Aug. 2019.
  14. L. Wang, H. Ye, L. Liang, and G. Y. Li, “Learn to compress CSI and allocate resources in vehicular networks,” IEEE Trans. Commun., vol. 68, no. 6, pp. 3640–3653, Mar. 2020.
  15. Z. Ji, Z. Qin, and C. G. Parini, “Reconfigurable intelligent surface aided cellular networks with device-to-device users,” IEEE Trans. Commun., vol. 70, no. 3, pp. 1808–1819, Jan. 2022.
  16. H. Ye, G. Y. Li, and B.-H. F. Juang, “Deep reinforcement learning based resource allocation for V2V communications,” IEEE Trans. Veh. Technol., vol. 68, no. 4, pp. 3163–3173, Feb. 2019.
  17. Y. Yuan, G. Zheng, K.-K. Wong, and K. B. Letaief, “Meta-reinforcement learning based resource allocation for dynamic V2X communications,” IEEE Trans. Veh. Technol., vol. 70, no. 9, pp. 8964–8977, Jul. 2021.
  18. M. He, Y. Li, X. Wang, and Z. Liu, “NOMA resource allocation method in IoV based on prioritized DQN-DDPG network,” EURASIP Journal on Advances in Signal Processing, vol. 2021, no. 1, pp. 1–17, Dec. 2021.
  19. J. Foerster et al., “Stabilising experience replay for deep multi-agent reinforcement learning,” in Proc. Int. Conf. Mach. Learn., Jul. 2017, pp. 1146–1155.
  20. Z. Qin, G. Ye Li, and H. Ye, “Federated learning and wireless communications,” IEEE Wireless Commun., pp. 1–7, Sep. 2021.
  21. A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage, “Federated learning for mobile keyboard prediction,” arXiv preprint arXiv: 1811.03604, Feb. 2019.
  22. K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Konečný, S. Mazzocchi, H. B. McMahan, T. V. Overveldt, D. Petrou, D. Ramage, and J. Roselander, “Towards federated learning at scale: System design,” arXiv preprint arXiv: 1902.01046, Mar. 2019.
  23. H. H. Zhuo, W. Feng, Y. Lin, Q. Xu, and Q. Yang, “Federated deep reinforcement learning,” arXiv preprint arXiv: 1901.08277, Feb. 2020.
  24. L. Zhang, H. Yin, Z. Zhou, S. Roy, and Y. Sun, “Enhancing WiFi multiple access performance with federated deep reinforcement learning,” in Proc. IEEE Veh. Technol. Conf., Nov. 2020, pp. 1–6.
  25. R. Zhong, X. Liu, Y. Liu, Y. Chen, and Z. Han, “Mobile reconfigurable intelligent surfaces for NOMA networks: Federated learning approaches,” arXiv preprint arXiv:2105.09462, Mar. 2021.
  26. A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach,” in Proc. Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., Dec. 2020, pp. 3557–3568.
  27. H. W. Kuhn, “The hungarian method for the assignment problem,” Naval Research Logistics Quarterly, vol. 2, no. 1‐2, pp. 83–97, Mar. 1955.
  28. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv: 1707.06347, Jul. 2017.
  29. J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv: 1506.02438, Jun. 2015.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now