Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Deep Q-Learning based, Base-Station Connectivity-Aware, Decentralized Pheromone Mobility Model for Autonomous UAV Networks (2311.16409v1)

Published 28 Nov 2023 in cs.NI and eess.SP

Abstract: UAV networks consisting of low SWaP (size, weight, and power), fixed-wing UAVs are used in many applications, including area monitoring, search and rescue, surveillance, and tracking. Performing these operations efficiently requires a scalable, decentralized, autonomous UAV network architecture with high network connectivity. Whereas fast area coverage is needed for quickly sensing the area, strong node degree and base station (BS) connectivity are needed for UAV control and coordination and for transmitting sensed information to the BS in real time. However, the area coverage and connectivity exhibit a fundamental trade-off: maintaining connectivity restricts the UAVs' ability to explore. In this paper, we first present a node degree and BS connectivity-aware distributed pheromone (BS-CAP) mobility model to autonomously coordinate the UAV movements in a decentralized UAV network. This model maintains a desired connectivity among 1-hop neighbors and to the BS while achieving fast area coverage. Next, we propose a deep Q-learning policy based BS-CAP model (BSCAP-DQN) to further tune and improve the coverage and connectivity trade-off. Since it is not practical to know the complete topology of such a network in real time, the proposed mobility models work online, are fully distributed, and rely on neighborhood information. Our simulations demonstrate that both proposed models achieve efficient area coverage and desired node degree and BS connectivity, improving significantly over existing schemes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. A. L. Alfeo, M. G.C.A. Cimino, N. De Francesco, M. Lega, G. Vaglini, “Design and simulation of the emergent behavior of small drones swarming for distributed target localization,” J. Computational Sci., vol. 29, pp. 19-33, 2018.
  2. I. Martinez-Alpiste, G. Golcarenarenji, Q. Wang and J. M. Alcaraz-Calero, “Search and rescue operation using UAVs: A case study,” Expert Systems with Applications, vol 178, p.114937, 2021.
  3. E. Y. Adam, “Leveraging connectivity for coverage in drone networks for target detection.” Balkan J. Electrical and Computer Engineering, vol. 7, pp. 218-225, 2019.
  4. D. Hambling, “The US Navy wants swarms of thousands of small drones.” https://www.technologyreview.com/2022/10/24/1062039/us-navy-swarms-of-thousands-of-small-drones/ (accessed Oct. 15, 2023).
  5. “AeroVironment Switchblade,” Wikipedia, https://en.wikipedia.org/wiki/AeroVironment_Switchblade (accessed Oct. 15, 2023).
  6. “Raytheon Coyote,” Wikipedia, https://en.wikipedia.org/wiki/Raytheon_Coyote (accessed Oct. 15, 2023).
  7. “Zala Lancet,” Wikipedia, https://en.wikipedia.org/wiki/ZALA_Lancet (accessed Oct. 15, 2023).
  8. E. Y. Adam, “Connectivity considerations for mission planning of a search and rescue drone team,” Turkish J. Electrical Engineering and Computer Sciences, 28(4), pp. 2228-2243, 2020.
  9. S. Garg, A. Ihler, E. S. Bentley and S. Kumar, “A Cross-Layer, Mobility, and Congestion-Aware Routing Protocol for UAV Networks,” IEEE Trans. Aerospace and Electronic Systems, vol. 59, pp. 3778-3796, Aug. 2023.
  10. R. Rajan, M. Otte and D. Sofge, “Novel physicomimetic bio-inspired algorithm for search and rescue applications,” IEEE Symp. Series Comput. Intell., pp. 1-8, 2017.
  11. E. Yanmaz, “Joint or decoupled optimization: Multi-UAV path planning for search and rescue” Ad Hoc Networks, vol. 138, p. 103018, 2023.
  12. H. J. Na and S. J. Yoo, “PSO-Based Dynamic UAV Positioning Algorithm for Sensing Information Acquisition in Wireless Sensor Networks,” in IEEE Access, vol. 7, pp. 77499-77513, 2019.
  13. A. I. Hentati, and L. C. Fourati, “Comprehensive survey of UAVs communication networks,” Computer Standards & Interfaces, 72, p. 103451, 2020.
  14. Y. Shao, Z. Zhao, R. Li, and Y. Zhou, “Target detection for multi-UAVs via digital pheromones and navigation algorithm in unknown environments,” Frontiers of Inform. Tech. & Electronic Eng. 21, pp. 796-808, 2020.
  15. H. Zhao, H. Wang, W. Wu, and J. Wei, “Deployment Algorithms for UAV Airborne Networks Toward On-Demand Coverage,” IEEE J. Selected Areas in Commun., vol. 36, pp. 2015-2031, Sept. 2018.
  16. M. Messous, H. Sedjelmaci, and S. Senouci, “Implementing an emerging mobility model for a fleet of UAVs based on a fuzzy logic inference system,” Pervasive and Mobile Computing, 42, pp. 393-410, 2017.
  17. S. Devaraju, A. Ihler, and S. Kumar, “A Connectivity-Aware Pheromone Mobility Model for Autonomous UAV Networks,” IEEE Consumer Communications & Networking Conf. (RoboCom’23 Workshop), Las Vegas, NV, Jan. 2023.
  18. S. Devaraju, A. Ihler, and S. Kumar, “A Deep Q-Learning Connectivity-Aware Pheromone Mobility Model for Autonomous UAV Networks,” IEEE Int. Conf. Computing, Networking & Comm., Honolulu, Hawaii, Feb. 2023.
  19. H. El Hammouti, M. Benjillali, B. Shihada and M. -S. Alouini, “Learn-As-You-Fly: A Distributed Algorithm for Joint 3D Placement and User Association in Multi-UAVs Networks,” IEEE Trans. Wireless Commun., vol. 18, pp. 5831-5844, Dec. 2019.
  20. H. A. Poonawala, and M. W. Spong, “Decentralized estimation of the algebraic connectivity for strongly connected networks,” IEEE American Control Conf., pp. 4068–4073, 2015.
  21. J. Scherer, and B. Rinner, “Short and full horizon motion planning for persistent multi-UAV surveillance with energy and communication constraints,” Proc. IEEE/RSJ Intl. Conf. Intelligent Robots and Systems (IROS), pp. 230–235, 2017.
  22. S. Rezwan and W. Choi, “A survey on applications of reinforcement learning in flying ad-hoc networks,” Electronics, vol. 10, pp. 449, 2021.
  23. J. K. Gupta, M. Egorov, and M. J. Kochenderfer, “Cooperative multi-agent control using deep reinforcement learning,” Proc. Int. Conf. Auton. Agents Multiagent Syst., pp. 66-83, May 2017.
  24. W. Yue, X. Guan, and L. Wang. “A Novel Searching Method Using Reinforcement Learning Scheme for Multi-UAVs in Unknown Environments,” Applied Sciences, vol. 9, pp. 4964, 2019.
  25. C. H. Liu, X. Ma, X. Gao and J. Tang, “Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning,” IEEE Trans. Mobile Computing, vol. 19, pp. 1274-1285, 1 June 2020.
  26. L. Yu, F. Wu, Z. Xu, Z. Xie and D. Yang, “UAV path design with connectivity constraint based on deep reinforcement learning,” Physical Communication, 52, p. 101582, 2022.
  27. S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, May 2020.
  28. “Multi-sensor payloads: Intelligence, Surveillance, and Reconnaissance (ISR) micro-gimbals,” AeroVironment, Inc., https://www.avinc.com/uas/payloads (accessed Oct. 16, 2023).
  29. M. Rosalie, M. R. Brust, G. Danoy, S. Chaumette, and P. Bouvry, “Coverage Optimization with Connectivity Preservation for UAV Swarms Applying Chaotic Dynamics,” IEEE Int. Conf. Autonomic Computing (ICAC), Columbus, OH, USA, 2017, pp. 113-118.
  30. R. K. Jain, D. M. Chiu, and W. R. Hawe, “A quantitative measure of fairness and discrimination for resource allocation in shared computer systems,” Eastern Research Laboratory, Digital Equipment Corporation, 1984.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now