Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CQLite: Communication-Efficient Multi-Robot Exploration Using Coverage-biased Distributed Q-Learning (2307.00500v2)

Published 2 Jul 2023 in cs.RO and cs.MA

Abstract: Frontier exploration and reinforcement learning have historically been used to solve the problem of enabling many mobile robots to autonomously and cooperatively explore complex surroundings. These methods need to keep an internal global map for navigation, but they do not take into consideration the high costs of communication and information sharing between robots. This study offers CQLite, a novel distributed Q-learning technique designed to minimize data communication overhead between robots while achieving rapid convergence and thorough coverage in multi-robot exploration. The proposed CQLite method uses ad hoc map merging, and selectively shares updated Q-values at recently identified frontiers to significantly reduce communication costs. The theoretical analysis of CQLite's convergence and efficiency, together with extensive numerical verification on simulated indoor maps utilizing several robots, demonstrates the method's novelty. With over 2x reductions in computation and communication alongside improved mapping performance, CQLite outperformed cutting-edge multi-robot exploration techniques like Rapidly Exploring Random Trees and Deep Reinforcement Learning. Related codes are open-sourced at \url{https://github.com/herolab-uga/cqlite}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, “Coordinated multi-robot exploration,” IEEE Transactions on robotics, vol. 21, no. 3, pp. 376–386, 2005.
  2. R. Parasuraman, S. Caccamo, F. Båberg, P. Ögren, and M. Neerincx, “A new ugv teleoperation interface for improved awareness of network connectivity and physical surroundings,” Journal of Human-Robot Interaction, vol. 6, no. 3, pp. 48–70, 2017.
  3. B. Fang, J. Ding, and Z. Wang, “Autonomous robotic exploration based on frontier point optimization and multistep path planning,” IEEE Access, vol. 7, pp. 46 104–46 113, 2019.
  4. A. Dai, S. Papatheodorou, N. Funk, D. Tzoumanikas, and S. Leutenegger, “Fast frontier-based information-driven autonomous exploration with an mav,” in 2020 IEEE international conference on robotics and automation (ICRA).   IEEE, 2020, pp. 9570–9576.
  5. E. Latif, W. Song, and R. Parasuraman, “Communication-efficient reinforcement learning in swarm robotic networks for maze exploration,” in IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2023, pp. 1–6.
  6. R. Shrestha, F.-P. Tian, W. Feng, P. Tan, and R. Vaughan, “Learned map prediction for enhanced mobile robot exploration,” in 2019 International Conference on Robotics and Automation (ICRA).   IEEE, 2019, pp. 1197–1204.
  7. Z. Zhang, X. Wang, Q. Zhang, and T. Hu, “Multi-robot cooperative pursuit via potential field-enhanced reinforcement learning,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 8808–8814.
  8. E. Tolstaya, J. Paulos, V. Kumar, and A. Ribeiro, “Multi-robot coverage and exploration using spatial graph neural networks,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 8944–8950.
  9. Q. Yang, Z. Luo, W. Song, and R. Parasuraman, “Self-reactive planning of multi-robots with dynamic task assignments,” in 2019 International Symposium on Multi-Robot and Multi-Agent Systems (MRS).   IEEE, 2019, pp. 89–91.
  10. A. K. Sadhu and A. Konar, “An efficient computing of correlated equilibrium for cooperative q𝑞qitalic_q-learning-based multi-robot planning,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 50, no. 8, pp. 2779–2794, 2018.
  11. L. Zhang, Z. Lin, J. Wang, and B. He, “Rapidly-exploring random trees multi-robot map exploration under optimization framework,” Robotics and Autonomous Systems, vol. 131, p. 103565, 2020.
  12. J. Hu, H. Niu, J. Carrasco, B. Lennox, and F. Arvin, “Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 12, pp. 14 413–14 423, 2020.
  13. W. Gao, M. Booker, A. Adiwahono, M. Yuan, J. Wang, and Y. W. Yun, “An improved frontier-based approach for autonomous exploration,” in 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV).   IEEE, 2018, pp. 292–297.
  14. A. Bouman, M. F. Ginting, N. Alatur, M. Palieri, D. D. Fan, T. Touma, T. Pailevanian, S.-K. Kim, K. Otsu, J. Burdick et al., “Autonomous spot: Long-range autonomous exploration of extreme environments with legged locomotion,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 2518–2525.
  15. E. Latif and R. Parasuraman, “Seal: Simultaneous exploration and localization for multi-robot systems,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 5358–5365.
  16. Z. Zhang, J. Yu, J. Tang, Y. Xu, and Y. Wang, “Mr-topomap: Multi-robot exploration based on topological map in communication restricted environment,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 794–10 801, 2022.
  17. K. Masaba and A. Q. Li, “Gvgexp: Communication-constrained multi-robot exploration system based on generalized voronoi graphs,” in 2021 International Symposium on Multi-Robot and Multi-Agent Systems (MRS).   IEEE, 2021, pp. 146–154.
  18. M. Corah, C. O’Meadhra, K. Goel, and N. Michael, “Communication-efficient planning and mapping for multi-robot exploration in large environments,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1715–1721, 2019.
  19. Y. Gao, Y. Wang, X. Zhong, T. Yang, M. Wang, Z. Xu, Y. Wang, Y. Lin, C. Xu, and F. Gao, “Meeting-merging-mission: A multi-robot coordinate framework for large-scale communication-limited exploration,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2022, pp. 13 700–13 707.
  20. A. Serra-Gómez, B. Brito, H. Zhu, J. J. Chung, and J. Alonso-Mora, “With whom to communicate: learning efficient communication for multi-robot collision avoidance,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 11 770–11 776.
  21. R. Han, S. Chen, and Q. Hao, “Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 448–454.
  22. X. Yu, Y. Wu, and X.-M. Sun, “A navigation scheme for a random maze using reinforcement learning with quadrotor vision,” in 2019 18th European Control Conference (ECC).   IEEE, 2019, pp. 518–523.
  23. J. Liu, K. Chen, R. Liu, Y. Yang, Z. Wang, and J. Zhang, “Robust and accurate multi-agent slam with efficient communication for smart mobiles,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 2782–2788.
  24. L. Bernreiter, S. Khattak, L. Ott, R. Siegwart, M. Hutter, and C. Cadena, “Collaborative robot mapping using spectral graph analysis,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 3662–3668.
  25. M. Zhao, X. Guo, L. Song, B. Qin, X. Shi, G. H. Lee, and G. Sun, “A general framework for lifelong localization and mapping in changing environment,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 3305–3312.
  26. Y. Jia, H. Luo, F. Zhao, G. Jiang, Y. Li, J. Yan, Z. Jiang, and Z. Wang, “Lvio-fusion: A self-adaptive multi-sensor fusion slam framework using actor-critic method,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2021, pp. 286–293.
  27. J. G. Mangelson, D. Dominic, R. M. Eustice, and R. Vasudevan, “Pairwise consistent measurement set maximization for robust multi-robot map merging,” in 2018 IEEE international conference on robotics and automation (ICRA).   IEEE, 2018, pp. 2916–2923.
  28. M. Keidar and G. A. Kaminka, “Efficient frontier detection for robot exploration,” The International Journal of Robotics Research, vol. 33, no. 2, pp. 215–236, 2014.
  29. Z. Uykan, “On the working principle of the hopfield neural networks and its equivalence to the gadia in optimization,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 9, pp. 3294–3304, 2019.
  30. J. Li, “Faster parallel algorithm for approximate shortest path,” in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, 2020, pp. 308–321.
  31. S. S. OV, R. Parasuraman, and R. Pidaparti, “Impact of heterogeneity in multi-robot systems on collective behaviors studied using a search and rescue problem,” in 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).   IEEE, 2020, pp. 290–297.
  32. P. Duda, M. Jaworski, L. Pietruczuk, and L. Rutkowski, “A novel application of hoeffding’s inequality to decision trees construction for data streams,” in 2014 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2014, pp. 3324–3330.
  33. R. Konda, H. M. La, and J. Zhang, “Decentralized function approximated q-learning in multi-robot systems for predator avoidance,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6342–6349, 2020.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now