A Deep Reinforcement Learning Approach for Autonomous Reconfigurable Intelligent Surfaces
Abstract: A reconfigurable intelligent surface (RIS) is a prospective wireless technology that enhances wireless channel quality. An RIS is often equipped with passive array of elements and provides cost and power-efficient solutions for coverage extension of wireless communication systems. Without any radio frequency (RF) chains or computing resources, however, the RIS requires control information to be sent to it from an external unit, e.g., a base station (BS). The control information can be delivered by wired or wireless channels, and the BS must be aware of the RIS and the RIS-related channel conditions in order to effectively configure its behavior. Recent works have introduced hybrid RIS structures possessing a few active elements that can sense and digitally process received data. Here, we propose the operation of an entirely autonomous RIS that operates without a control link between the RIS and BS. Using a few sensing elements, the autonomous RIS employs a deep Q network (DQN) based on reinforcement learning in order to enhance the sum rate of the network. Our results illustrate the potential of deploying autonomous RISs in wireless networks with essentially no network overhead.
- E. Basar, M. Di Renzo, J. De Rosny, M. Debbah, M.-S. Alouini, and R. Zhang, “Wireless communications through reconfigurable intelligent surfaces,” IEEE Access, vol. 7, pp. 116 753–116 773, Sept. 2019.
- M. Di Renzo, A. Zappone, M. Debbah, M.-S. Alouini, C. Yuen, J. de Rosny, and S. Tretyakov, “Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead,” IEEE J. Select. Areas in Commun., vol. 38, no. 11, pp. 2450–2525, Nov. 2020.
- C. Pan, G. Zhou, K. Zhi, S. Hong, T. Wu, Y. Pan, H. Ren, M. D. Renzo, A. Swindlehurst, R. Zhang, and A. Y. Zhang, “An overview of signal processing techniques for RIS/IRS-aided wireless systems,” IEEE J. Select. Topics Signal Process., vol. 16, no. 5, pp. 883–917, Aug. 2022.
- S. Kim, H. Lee, J. Cha, S.-J. Kim, J. Park, and J. Choi, “Practical channel estimation and phase shift design for intelligent reflecting surface empowered mimo systems,” IEEE Trans. Wireless Commun., vol. 21, no. 8, pp. 6226–6241, 2022.
- G. C. Alexandropoulos, N. Shlezinger, I. Alamzadeh, M. F. Imani, H. Zhang, and Y. C. Eldar, “Hybrid reconfigurable intelligent metasurfaces: Enabling simultaneous tunable reflections and sensing for 6G wireless communications,” arxiv preprint 2104.04690, Apr. 2021.
- I. Alamzadeh, G. C. Alexandropoulos, N. Shlezinger, and M. F. Imani, “A reconfigurable intelligent surface with integrated sensing capability,” Sci. Rep., vol. 11, no. 1, Oct. 2021, Art. no. 20737.
- H. Zhang, N. Shlezinger, G. C. Alexandropoulos, A. Shultzman, I. Alamzadeh, M. F. Imani, and Y. C. Eldar, “Channel estimation with hybrid reconfigurable intelligent metasurfaces,” IEEE Trans. Commun., vol. 71, no. 4, pp. 2441–2456, Apr. 2023.
- L. V. Nguyen and A. Swindlehurst, “Decision-directed hybrid RIS channel estimation with minimal pilot overhead,” arxiv preprint 2309.11485, 2023.
- A. Taha, Y. Zhang, F. B. Mismar, and A. Alkhateeb, “Deep reinforcement learning for intelligent reflecting surfaces: Towards standalone operation,” in 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2020, pp. 1–5.
- W. Wang and W. Zhang, “Intelligent reflecting surface configurations for smart radio using deep reinforcement learning,” IEEE J. Select. Areas Commun., vol. 40, no. 8, pp. 2335–2346, Aug. 2022.
- V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arxiv preprint 1312.5602, 2013.
- F. B. Mismar, B. L. Evans, and A. Alkhateeb, “Deep reinforcement learning for 5G networks: Joint beamforming, power control, and interference coordination,” IEEE Trans. Commun., vol. 68, no. 3, pp. 1581–1592, 2020.
- S. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,” IEEE Tran. on Signal Process., vol. 41, no. 12, pp. 3397–3415, 1993.
- B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” arxiv preprint 1505.00853, 2015.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.