Fairness-Driven Optimization of RIS-Augmented 5G Networks for Seamless 3D UAV Connectivity Using DRL Algorithms (2312.09420v1)
Abstract: In this paper, we study the problem of joint active and passive beamforming for reconfigurable intelligent surface (RIS)-assisted massive multiple-input multiple-output systems towards the extension of the wireless cellular coverage in 3D, where multiple RISs, each equipped with an array of passive elements, are deployed to assist a base station (BS) to simultaneously serve multiple unmanned aerial vehicles (UAVs) in the same time-frequency resource of 5G wireless communications. With a focus on ensuring fairness among UAVs, our objective is to maximize the minimum signal-to-interference-plus-noise ratio (SINR) at UAVs by jointly optimizing the transmit beamforming parameters at the BS and phase shift parameters at RISs. We propose two novel algorithms to address this problem. The first algorithm aims to mitigate interference by calculating the BS beamforming matrix through matrix inverse operations once the phase shift parameters are determined. The second one is based on the principle that one RIS element only serves one UAV and the phase shift parameter of this RIS element is optimally designed to compensate the phase offset caused by the propagation and fading. To obtain the optimal parameters, we utilize one state-of-the-art reinforcement learning algorithm, deep deterministic policy gradient, to solve these two optimization problems. Simulation results are provided to illustrate the effectiveness of our proposed solution and some insightful remarks are observed.
- C. Zhang, W. Zhang, W. Wang, L. Yang, and W. Zhang, “Research challenges and opportunities of UAV millimeter-wave communications,” IEEE Wireless Commun., vol. 26, no. 1, pp. 58–62, 2019.
- Y. Tian, G. Pan, M. A. Kishk, and M.-S. Alouini, “Stochastic analysis of cooperative satellite-UAV communications,” IEEE Trans. Wireless Commun., vol. 21, no. 6, pp. 3570–3586, 2022.
- Z. Zhang, Y. Xiao, Z. Ma, M. Xiao, Z. Ding, X. Lei, G. K. Karagiannidis, and P. Fan, “6G wireless networks: Vision, requirements, architecture, and key technologies,” IEEE Veh. Technol. Mag., vol. 14, no. 3, pp. 28–41, 2019.
- J. He, A. Fakhreddine, A. S. de Sena, Y. Tian, and M. Debbah, “Unleashing 3D connectivity in beyond 5G networks with reconfigurable intelligent surfaces,” arXiv preprint arXiv:2305.04661, 2023.
- C. Huang, R. Mo, and C. Yuen, “Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning,” IEEE J. Sel. Areas Commun., vol. 38, no. 8, pp. 1839–1850, 2020.
- B. Saglam, D. Gurgunoglu, and S. S. Kozat, “Deep reinforcement learning based joint downlink beamforming and RIS configuration in RIS-Aided MU-MISO systems under hardware impairments and imperfect CSI,” in 2023 IEEE Int. Conf. Commun. Workshops (ICC Workshops), 2023, pp. 66–72.
- H. Chen, G. Yang, and Y.-C. Liang, “Joint active and passive beamforming for reconfigurable intelligent surface enhanced symbiotic radio system,” IEEE Wireless Commun. Lett., vol. 10, no. 5, pp. 1056–1060, 2021.
- Y. Xu, Z. Gao, Z. Wang, C. Huang, Z. Yang, and C. Yuen, “RIS-enhanced WPCNs: Joint radio resource allocation and passive beamforming optimization,” IEEE Trans. Veh. Technol., vol. 70, no. 8, pp. 7980–7991, 2021.
- K. Stylianopoulos, G. Alexandropoulos, C. Huang, C. Yuen, M. Bennis, and M. Debbah, “Deep contextual bandits for orchestrating multi-user MISO systems with multiple RISs,” in ICC 2022 - IEEE Int. Conf. Commun., 2022, pp. 1556–1561.
- X. Li, J. Fang, F. Gao, and H. Li, “Joint active and passive beamforming for intelligent reflecting surface-assisted massive mimo systems,” arXiv preprint arXiv:1912.00728, 2019.
- H. Lu, Y. Zeng, S. Jin, and R. Zhang, “Aerial intelligent reflecting surface: Joint placement and passive beamforming design with 3D beam flattening,” IEEE Trans. Wireless Commun., vol. 20, no. 7, pp. 4128–4143, 2021.
- H. Zhou, M. Erol-Kantarci, Y. Liu, and H. V. Poor, “A survey on model-based, heuristic, and machine learning optimization approaches in RIS-aided wireless networks,” arXiv preprint arXiv:2303.14320, 2023.
- T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Int. Conf. Mach. Learn. PMLR, 2018, pp. 1861–1870.
- B. Saglam, “RIS MISO deep reinforcement learning,” https://github.com/baturaysaglam/RIS-MISO-Deep-Reinforcement-Learning, 2021.