Distributed Multi-Agent Deep Q-Learning for Fast Roaming in IEEE 802.11ax Wi-Fi Systems

Abstract: The innovation of Wi-Fi 6, IEEE 802.11ax, was be approved as the next sixth-generation (6G) technology of wireless local area networks (WLANs) by improving the fundamental performance of latency, throughput, and so on. The main technical feature of orthogonal frequency division multiple access (OFDMA) supports multi-users to transmit respective data concurrently via the corresponding access points (APs). However, the conventional IEEE 802.11 protocol for Wi-Fi roaming selects the target AP only depending on received signal strength indication (RSSI) which is obtained by the received Response frame from the APs. In the long term, it may lead to congestion in a single channel under the scenarios of dense users further increasing the association delay and packet drop rate, even reducing the quality of service (QoS) of the overall system. In this paper, we propose a multi-agent deep Q-learning for fast roaming (MADAR) algorithm to effectively minimize the latency during the station roaming for Smart Warehouse in Wi-Fi 6 system. The MADAR algorithm considers not only RSSI but also channel state information (CSI), and through online neural network learning and weighting adjustments to maximize the reward of the action selected from Epsilon-Greedy. Compared to existing benchmark methods, the MADAR algorithm has been demonstrated for improved roaming latency by analyzing the simulation result and realistic dataset.