Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigation (2403.10083v1)

Published 15 Mar 2024 in cs.RO

Abstract: Crowd navigation has received significant research attention in recent years, especially DRL-based methods. While single-robot crowd scenarios have dominated research, they offer limited applicability to real-world complexities. The heterogeneity of interaction among multiple agent categories, like in decentralized multi-robot pedestrian scenarios, are frequently disregarded. This "interaction blind spot" hinders generalizability and restricts progress towards robust navigation algorithms. In this paper, we propose a heterogeneous relational deep reinforcement learning(HeR-DRL), based on customised heterogeneous GNN, in order to improve navigation strategies in decentralized multi-robot crowd navigation. Firstly, we devised a method for constructing robot-crowd heterogenous relation graph that effectively simulates the heterogeneous pair-wise interaction relationships. We proposed a new heterogeneous graph neural network for transferring and aggregating the heterogeneous state information. Finally, we incorporate the encoded information into deep reinforcement learning to explore the optimal policy. HeR-DRL are rigorously evaluated through comparing it to state-of-the-art algorithms in both single-robot and multi-robot circle crowssing scenario. The experimental results demonstrate that HeR-DRL surpasses the state-of-the-art approaches in overall performance, particularly excelling in safety and comfort metrics. This underscores the significance of interaction heterogeneity for crowd navigation. The source code will be publicly released in https://github.com/Zhouxy-Debugging-Den/HeR-DRL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. L. Huber, J.-J. Slotine, and A. Billard, “Avoiding dense and dynamic obstacles in enclosed spaces: Application to moving in crowds,” IEEE Transactions on Robotics, vol. 38, no. 5, pp. 3113–3132, 2022.
  2. K. Kodur and M. Kyrarini,“Patient–robot co-navigation of crowded hospital environments,” Applied Sciences, vol. 13, no. 7, p. 4576, 2023.
  3. T. Fan, X. Cheng, J. Pan, P. Long, W. Liu, R. Yang, and D. Manocha, “Getting robots unfrozen and unlost in dense pedestrian crowds,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1178–1185, 2019.
  4. T. Kruse, A. K. Pandey, R. Alami, and A. Kirsch, “Human-aware robot navigation: A survey,” Robotics and Autonomous Systems, vol. 61, no. 12, pp. 1726–1743, 2013.
  5. J. Van den Berg, M. Lin, and D. Manocha, “Reciprocal velocity obstacles for real-time multi-agent navigation,” in 2008 IEEE international conference on robotics and automation.   Ieee, 2008, pp. 1928–1935.
  6. D. Helbing and P. Molnar, “Social force model for pedestrian dynamics,”Physical review E, vol. 51, no. 5, p. 4282, 1995.
  7. P. Trautman and A. Krause, “Unfreezing the robot: Navigation in dense, interacting crowds,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.   IEEE,2010, pp. 797–803.
  8. F. Feurtey, “Simulating the collision avoidance behavior of pedestrians,”The University of Tokyo, School of Engineering. Department of Electronic Engineering, 2000.
  9. G. S. Aoude, B. D. Luders, J. M. Joseph, N. Roy, and J. P. How, “Probabilistically safe motion planning to avoid dynamic obstacles with uncertain motion patterns,” Autonomous Robots, vol. 35, pp. 51–76, 2013.
  10. H. Kretzschmar, M. Spies, C. Sprunk, and W. Burgard, “Socially compliant mobile robot navigation via inverse reinforcement learning,” The International Journal of Robotics Research, vol. 35, no. 11, pp. 1289–1307, 2016.
  11. P. Trautman, J. Ma, R. M. Murray, and A. Krause, “Robot navigation in dense human crowds: the case for cooperation,” in 2013 IEEE international conference on robotics and automation.   IEEE, 2013, pp. 2153–2160.
  12. C. Chen, S. Hu, P. Nikdel, G. Mori, and M. Savva, “Relational graph learning for crowd navigation,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 10 007–10 013.
  13. C. Cao, P. Trautman, and S. Iba, “Dynamic channel: A planning framework for crowd navigation,” in 2019 international conference on robotics and automation (ICRA).   IEEE, 2019, pp.5551–5557.
  14. K. Driggs-Campbell, V. Govindarajan, and R. Bajcsy, “Integrating intuitive driver models in autonomous planning for interactive maneuvers,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 12, pp.3461–3472, 2017.
  15. K. Driggs-Campbell, R. Dong, and R. Bajcsy, “Robust, informative human-in-the-loop predictions via empirical reachable sets,” IEEE Transactions on Intelligent Vehicles, vol. 3, no. 3, pp. 300–309, 2018.
  16. Y. F. Chen, M. Liu, M. Everett, and J. P. How, “Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning,” in 2017 IEEE international conference on robotics and automation (ICRA).   IEEE, 2017, pp. 285–292.
  17. C. Chen, Y. Liu, S. Kreiss, and A. Alahi, “Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning,” in 2019 international conference on robotics and automation(ICRA).   IEEE, 2019, pp. 6015–6022.
  18. W. Shi, Y. Zhou, X. Zeng, S. Li, and M. Bennewitz, “Enhanced spatial attention graph for motion planning in crowded, partially observable environments,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 4750–4756.
  19. X. Zhang, W. Xi, X. Guo, Y. Fang, B. Wang, W. Liu, and J. Hao, “Relational navigation learning in continuous action space among crowds,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 3175–3181.
  20. Y. Yang, J. Jiang, J. Zhang, J. Huang, and M. Gao, “St22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT: Spatial-temporal state transformer for crowd-aware autonomous navigation,” IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 912–919, 2023.
  21. J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha, “Reciprocal n-body collision avoidance,” in Robotics Research: The 14th International Symposium ISRR.   Springer, 2011, pp.3–19.
  22. M. Everett, Y. F. Chen, and J. P. How, “Motion planning among dynamic,decision-making agents with deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2018, pp. 3052–3059.
  23. W. Wang, L. Mao, R. Wang, and B.-C. Min, “Multi-robot cooperative socially-aware navigation using multi-agent reinforcement learning,”arXiv preprint arXiv:2309.15234.
  24. R. Chandra, V. Zinage, E. Bakolas, J. Biswas, and P. Stone, “Decentralized multi-robot social navigation in constrained environments via game-theoretic control barrier functions,” arXiv preprint arXiv:2308.10966, 2023.
  25. Y. Chen, C. Liu, B. E. Shi, and M. Liu, “Robot navigation in crowds by graph convolutional networks with attention learned from human gaze,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2754–2761, 2020.
  26. Z. Zhou, P. Zhu, Z. Zeng, J. Xiao, H. Lu, and Z. Zhou, “Robot navigation in a crowd by integrating deep reinforcement learning and online planning,” Applied Intelligence, vol. 52, no. 13, pp. 15 600–15 616, 2022.
  27. S. Liu, P. Chang, W. Liang, N. Chakraborty, and K. Driggs-Campbell, “Decentralized structural-rnn for robot crowd navigation with deep reinforcement learning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 3517–3524.
  28. S. Liu, P. Chang, Z. Huang, N. Chakraborty, K. Hong, W. Liang, D. L. McPherson, J. Geng, and K. Driggs-Campbell, “Intention aware robot crowd navigation with attention-based interaction graph,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 12 015–12 021.
  29. K. Zhu, B. Li, W. Zhe, and T. Zhang, “Collision avoidance among dense heterogeneous agents using deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 8, no. 1, pp. 57–64, 2022.
  30. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  31. W. L. Hamilton, R. Ying, and J. Leskovec, “Representation learning on graphs: Methods and applications,” arXiv preprint arXiv:1709.05584, 2017.
  32. M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15.   Springer, 2018, pp. 593–607.
  33. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  34. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  35. C. Shi, Y. Li, J. Zhang, Y. Sun, and S. Y. Philip, “A survey of heterogeneous information network analysis,” IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 1, pp. 17–37, 2016.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now