2000 character limit reached
Learn Once Plan Arbitrarily (LOPA): Attention-Enhanced Deep Reinforcement Learning Method for Global Path Planning (2401.04145v1)
Published 8 Jan 2024 in cs.LG, cs.AI, and cs.RO
Abstract: Deep reinforcement learning (DRL) methods have recently shown promise in path planning tasks. However, when dealing with global planning tasks, these methods face serious challenges such as poor convergence and generalization. To this end, we propose an attention-enhanced DRL method called LOPA (Learn Once Plan Arbitrarily) in this paper. Firstly, we analyze the reasons of these problems from the perspective of DRL's observation, revealing that the traditional design causes DRL to be interfered by irrelevant map information. Secondly, we develop the LOPA which utilizes a novel attention-enhanced mechanism to attain an improved attention capability towards the key information of the observation. Such a mechanism is realized by two steps: (1) an attention model is built to transform the DRL's observation into two dynamic views: local and global, significantly guiding the LOPA to focus on the key information on the given maps; (2) a dual-channel network is constructed to process these two views and integrate them to attain an improved reasoning capability. The LOPA is validated via multi-objective global path planning experiments. The result suggests the LOPA has improved convergence and generalization performance as well as great path planning efficiency.
- A. Kenshi, X. Zijian, S. Issei, and S. Masashi, “Solving np-hard problems on graphs by reinforcement learning without domain knowledge,” ArXiv, vol. abs/1905.11623, 2019.
- A. OpenAI: Marcin, B. Bowen, C. Maciek, J. Rafal, and M. Bob, “Learning dexterous in-hand manipulation,” The International Journal of Robotics Research, vol. 39, no. 1, pp. 3–20, 2020.
- N. Juan, H. Zhenhua, C. Jiujun, and G. Shangce, “An effective recommendation model based on deep representation learning,” Information Sciences, vol. 542, pp. 324–342, 2021.
- S. Miki, D. Yamamoto, and H. Ebara, “Applying deep learning and reinforcement learning to traveling salesman problem,” 2018 International Conference on Computing, Electronics Communications Engineering (iCCECE), pp. 65–70, 2018.
- G. Tong, J. Nan, L. Biyue, Z. Xi, Y. Wang, and W. Du, “Uav navigation in high dynamic environments: A deep reinforcement learning approach,” Chinese Journal of Aeronautics, vol. 34, no. 2, pp. 479–489, 2021.
- Y. Zhao, X. Wang, R. Wang, Y. Yang, and F. Lv, “Path planning for mobile robots based on tpr-ddpg,” pp. 1–8, 2021.
- J. Gao, W. Ye, J. Guo, and Z. Li, “Deep reinforcement learning for indoor mobile robot path planning,” Sensors, vol. 20, no. 19, 2020.
- J. Hu, H. Niu, J. Carrasco, B. Lennox, and F. Arvin, “Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 12, pp. 14 413–14 423, 2020.
- A. A. Ravankar, A. Ravankar, T. Emaru, and Y. Kobayashi, “Hpprm: Hybrid potential based probabilistic roadmap algorithm for improved dynamic path planning of mobile robots,” IEEE Access, vol. 8, pp. 221 743–221 766, 2020.
- Y. Wang, M. Li, Q. Zhao, and Y. Xie, “Improved rrt algorithm for field environment tends to smooth path,” pp. 6657–6661, 2021.
- X. Bo, Z. Feng, and M. G. Antonio, “Multi-objective particle swarm optimization algorithm for the minimum constraint removal problem,” International Journal of Computational Intelligence Systems, vol. 13, pp. 291–299, 2020.
- G. Xinghai, J. Mingjun, Z. Ziwei, W. Dusu, and Z. Weidan, “Global path planning and multi-objective path control for unmanned surface vehicle based on modified particle swarm optimization (pso) algorithm,” Ocean Engineering, vol. 216, 2020.
- M. Norouzi, J. V. Miro, and G. Dissanayake, “Planning high-visibility stable paths for reconfigurable robots on uneven terrain,” pp. 2844–2849, 2012.
- C. I. Ammar Adel, Bennaceur Hachemi, “Relaxed dijkstra and a* with linear complexity for robot path planning problems in large-scale grid environments,” Soft Comput, vol. 20, no. 10, pp. 4146–4171, 2016.
- Y.-N. Ma, Y.-J. Gong, C.-F. Xiao, Y. Gao, and J. Zhang, “Path planning for autonomous underwater vehicles: An ant colony algorithm incorporating alarm pheromone,” IEEE Transactions on Vehicular Technology, vol. 68, no. 1, pp. 141–154, 2019.
- G. Huang, X. Yuan, K. Shi, Z. Liu, and X. Wu, “A 3-d multi-object path planning method for electric vehicle considering the energy consumption and distance,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–13, 2021.
- L. Jiang, H. Huang, and Z. Ding, “Path planning for intelligent robots based on deep q-learning with experience replay and heuristic knowledge,” IEEE/CAA Journal of Automatica Sinica, vol. 7, no. 4, pp. 1179–1189, 2020.
- R. Cimurs, I. H. Suh, and J. H. Lee, “Goal-driven autonomous exploration through deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 730–737, 2022.
- C. Wang, J. Wang, Y. Shen, and X. Zhang, “Autonomous navigation of uavs in large-scale complex environments: A deep reinforcement learning approach,” IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 2124–2136, 2019.
- J. Jiang, J. Xu, J. Zhang, and S. Chen, “Deep reinforcement learning with new-field exploration for navigation in detour environment,” pp. 13–18, 2021.
- F. Zeng, C. Wang, and S. S. Ge, “Tutor-guided interior navigation with deep reinforcement learning,” IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 4, pp. 934–944, 2021.
- S. Guo, X. Zhang, Y. Zheng, and Y. Du, “An autonomous path planning model for unmanned ships based on deep reinforcement learning,” Sensors, vol. 20, no. 2, 2020.
- M. Piotr, K. G. Matthew, M. Mateusz, M. H. Karl, and A. Keith, “Learning to navigate in cities without a map,” CoRR, vol. abs/1804.00168, 2018. [Online]. Available: http://arxiv.org/abs/1804.00168
- R. Xie, Z. Meng, L. Wang, H. Li, K. Wang, and Z. Wu, “Unmanned aerial vehicle path planning algorithm based on deep reinforcement learning in large-scale and dynamic environments,” IEEE Access, vol. 9, pp. 24 884–24 900, 2021.
- M. Sutoh, M. Otsuki, S. Wakabayashi, T. Hoshino, and T. Hashimoto, “The right path: Comprehensive path planning for lunar exploration rovers,” IEEE Robotics Automation Magazine, vol. 22, no. 1, pp. 22–33, March 2015.
- M. Azizi and E. Tarshizi, “Autonomous control and navigation of a lab-scale underground mining haul truck using lidar sensor and triangulation - feasibility study,” pp. 1–6, 2016.
- Y. Ji, Y. Tanaka, Y. Tamura, M. Kimura, A. Umemura, and Y. Kaneshima, “Adaptive motion planning based on vehicle characteristics and regulations for off-road ugvs,” IEEE Transactions on Industrial Informatics, vol. 15, no. 1, pp. 599–611, 2019.
- W. Ziyu, F. Nando de, and L. Marc, “Dueling network architectures for deep reinforcement learning,” CoRR, vol. abs/1511.06581, 2015. [Online]. Available: http://arxiv.org/abs/1511.06581
- D. Mansoor, P. Fatemeh, M. Ali, and N. H. Seyed, “Clear and smooth path planning,” Applied Soft Computing, vol. 32, pp. 568–579, 2015.
- S. Tom, Q. John, A. Ioannis, and S. David, “Prioritized experience replay,” International Conference on Learning Representations(ICLR 2016), 2016.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” arXiv, 2017.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.