Optimizing Search and Rescue UAV Connectivity in Challenging Terrain through Multi Q-Learning (2405.10042v1)
Abstract: Using Unmanned Aerial Vehicles (UAVs) in Search and rescue operations (SAR) to navigate challenging terrain while maintaining reliable communication with the cellular network is a promising approach. This paper suggests a novel technique employing a reinforcement learning multi Q-learning algorithm to optimize UAV connectivity in such scenarios. We introduce a Strategic Planning Agent for efficient path planning and collision awareness and a Real-time Adaptive Agent to maintain optimal connection with the cellular base station. The agents trained in a simulated environment using multi Q-learning, encouraging them to learn from experience and adjust their decision-making to diverse terrain complexities and communication scenarios. Evaluation results reveal the significance of the approach, highlighting successful navigation in environments with varying obstacle densities and the ability to perform optimal connectivity using different frequency bands. This work paves the way for enhanced UAV autonomy and enhanced communication reliability in search and rescue operations.
- V. San Juan, M. Santos, J. M. Andújar et al., “Intelligent uav map generation and discrete path planning for search and rescue operations,” Complexity, vol. 2018, 2018.
- M. M. Qazzaz, S. A. Zaidi, D. McLernon, A. M. Hayajneh, A. Salama, and S. A. Aldalahmeh, “Non-terrestrial uav clients for beyond 5g networks: A comprehensive survey,” Ad Hoc Networks, p. 103440, 2024.
- A. Salama, S. A. Zaidi, D. McLernon, and M. M. Qazzaz, “Flcc: Efficient distributed federated learning on iomt over csma/ca,” in 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring). IEEE, 2023, pp. 1–6.
- D. Oh and J. Han, “Smart search system of autonomous flight uavs for disaster rescue,” Sensors, vol. 21, no. 20, p. 6810, 2021.
- C. Zhang, W. Zhou, W. Qin, and W. Tang, “A novel uav path planning approach: Heuristic crossing search and rescue optimization algorithm,” Expert Systems with Applications, vol. 215, p. 119243, 2023.
- G. Kumar, A. Anwar, A. Dikshit, A. Poddar, U. Soni, and W. K. Song, “Obstacle avoidance for a swarm of unmanned aerial vehicles operating on particle swarm optimization: A swarm intelligence approach for search and rescue missions,” Journal of the Brazilian Society of Mechanical Sciences and Engineering, vol. 44, no. 2, p. 56, 2022.
- Y. Du et al., “Multi-uav search and rescue with enhanced a algorithm path planning in 3d environment,” International Journal of Aerospace Engineering, vol. 2023, 2023.
- M. M. Qazzaz, S. A. Zaidi, D. McLernon, A. Salama, and A. A. Al-Hameed, “Low complexity online rl enabled uav trajectory planning considering connectivity and obstacle avoidance constraints,” in 2023 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom). IEEE, 2023, pp. 82–89.
- S. Hayat, E. Yanmaz, C. Bettstetter, and T. X. Brown, “Multi-objective drone path planning for search and rescue with quality-of-service requirements,” Autonomous Robots, vol. 44, no. 7, pp. 1183–1198, 2020.
- S. Lins, K. V. Cardoso, C. B. Both, L. Mendes, J. F. De Rezende, A. Silveira, N. Linder, and A. Klautau, “Artificial intelligence for enhanced mobility and 5g connectivity in uav-based critical missions,” IEEE Access, vol. 9, pp. 111 792–111 801, 2021.
- R. Zahínos, H. Abaunza, J. I. Murillo, M. A. Trujillo, and A. Viguria, “Cooperative multi-uav system for surveillance and search&rescue operations over a mobile 5g node,” in 2022 International Conference on Unmanned Aircraft Systems (ICUAS), 2022, pp. 1016–1024.
- M. M. Qazzaz, Ł. Kułacz, A. Kliks, S. A. Zaidi, M. Dryjanski, and D. McLernon, “Machine learning-based xapp for dynamic resource allocation in o-ran networks,” arXiv preprint arXiv:2401.07643, 2024.
- A. A. Meera, M. Popović, A. Millane, and R. Siegwart, “Obstacle-aware adaptive informative path planning for uav-based target search,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 718–724.
- J. P. Queralta, J. Taipalmaa, B. Can Pullinen, V. K. Sarker, T. Nguyen Gia, H. Tenhunen, M. Gabbouj, J. Raitoharju, and T. Westerlund, “Collaborative multi-robot search and rescue: Planning, coordination, perception, and active vision,” IEEE Access, vol. 8, pp. 191 617–191 643, 2020.
- G. Muñoz, C. Barrado, E. Çetin, and E. Salami, “Deep reinforcement learning for drone delivery,” Drones, vol. 3, no. 3, p. 72, 2019.
- C. Dalela, M. Prasad, P. Dalela et al., “Tuning of cost-231 hata model for radio wave propagation predictions,” Academy & Industry Research Collaboration Center, 2012.
- Mohammed M. H. Qazzaz (4 papers)
- Syed A. R. Zaidi (2 papers)
- Desmond C. McLernon (1 paper)
- Abdelaziz Salama (4 papers)
- Aubida A. Al-Hameed (7 papers)