Embodied AI in Mobile Robots: Coverage Path Planning with Large Language Models
Abstract: In recent years, LLMs have demonstrated remarkable capabilities in understanding and solving mathematical problems, leading to advancements in various fields. We propose an LLM-embodied path planning framework for mobile agents, focusing on solving high-level coverage path planning issues and low-level control. Our proposed multi-layer architecture uses prompted LLMs in the path planning phase and integrates them with the mobile agents' low-level actuators. To evaluate the performance of various LLMs, we propose a coverage-weighted path planning metric to assess the performance of the embodied models. Our experiments show that the proposed framework improves LLMs' spatial inference abilities. We demonstrate that the proposed multi-layer framework significantly enhances the efficiency and accuracy of these tasks by leveraging the natural language understanding and generative capabilities of LLMs. Our experiments show that this framework can improve LLMs' 2D plane reasoning abilities and complete coverage path planning tasks. We also tested three LLM kernels: gpt-4o, gemini-1.5-flash, and claude-3.5-sonnet. The experimental results show that claude-3.5 can complete the coverage planning task in different scenarios, and its indicators are better than those of the other models.
- M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. B. Shaikh, N. Akhtar, J. Wu, S. Mirjalili et al., “A survey on large language models: Applications, challenges, limitations, and practical usage,” Authorea Preprints, 2023.
- R. Chrisley, “Embodied artificial intelligence,” Artificial intelligence, vol. 149, no. 1, pp. 131–150, 2003.
- V. S. Dorbala, S. Chowdhury, and D. Manocha, “Can llms generate human-like wayfinding instructions? towards platform-agnostic embodied instruction synthesis,” arXiv preprint arXiv:2403.11487, 2024.
- L. Cao, “Ai robots and humanoid ai: Review, perspectives and directions,” arXiv preprint arXiv:2405.15775, 2024.
- S. Gu, “Llms as potential brainstorming partners for math and science problems–case studies and analysis.”
- H. Hewawasam, M. Y. Ibrahim, and G. K. Appuhamillage, “Past, present and future of path-planning algorithms for mobile robot navigation in dynamic environments,” IEEE Open Journal of the Industrial Electronics Society, vol. 3, pp. 353–365, 2022.
- E. Galceran and M. Carreras, “Efficient seabed coverage path planning for asvs and auvs,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012, pp. 88–93.
- M. Torres, D. A. Pelta, J. L. Verdegay, and J. C. Torres, “Coverage path planning with unmanned aerial vehicles for 3d terrain reconstruction,” Expert Systems with Applications, vol. 55, pp. 441–451, 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0957417416300306
- S. Hazem, M. Mostafa, E. Mohamed, M. Hesham, A. Mohamed, E. Lotfy, A. Mahmoud, and M. Yacoub, “Design and path planning of autonomous solar lawn mower,” in International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, vol. 85369. American Society of Mechanical Engineers, 2021, p. V001T01A016.
- C. W. Warren, “Fast path planning using modified a* method,” in [1993] Proceedings IEEE International Conference on Robotics and Automation. IEEE, 1993, pp. 662–667.
- D. Ferguson and A. Stentz, “The field d* algorithm for improved path planning and replanning in uniform and non-uniform cost environments,” Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-05-19, 2005.
- J. Barraquand, B. Langlois, and J.-C. Latombe, “Numerical potential field techniques for robot path planning,” IEEE transactions on systems, man, and cybernetics, vol. 22, no. 2, pp. 224–241, 1992.
- R. Zhang, D. Jiang, Y. Zhang, H. Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K.-W. Chang, P. Gao et al., “Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?” arXiv preprint arXiv:2403.14624, 2024.
- J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” 2023.
- Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-level reward design via coding large language models,” 2024.
- D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence, “Palm-e: An embodied multimodal language model,” in arXiv preprint arXiv:2303.03378, 2023.
- J. Su, C. Jiang, X. Jin, Y. Qiao, T. Xiao, H. Ma, R. Wei, Z. Jing, J. Xu, and J. Lin, “Large language models for forecasting and anomaly detection: A systematic literature review,” 2024.
- R. Schumann, W. Zhu, W. Feng, T.-J. Fu, S. Riezler, and W. Y. Wang, “Velma: Verbalization embodiment of llm agents for vision and language navigation in street view,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 17, pp. 18 924–18 933, Mar. 2024. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/29858
- P. Sharma, B. Sundaralingam, V. Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting robot plans with natural language feedback,” 2022.
- E. Latif, “3p-llm: Probabilistic path planning using large language model for autonomous robot navigation,” 2024.
- I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” 2022.
- Y.-L. Kuo, B. Katz, and A. Barbu, “Deep compositional robotic planners that follow natural language commands,” 2020.
- S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022.
- M. N. Ab Wahab, A. Nazir, A. Khalil, W. J. Ho, M. F. Akbar, M. H. M. Noor, and A. S. A. Mohamed, “Improved genetic algorithm for mobile robot path planning in static environments,” Expert Systems with Applications, vol. 249, p. 123762, 2024.
- O. Castillo, L. Trujillo, and P. Melin, “Multiple objective genetic algorithms for path-planning optimization in autonomous mobile robots,” Soft Computing, vol. 11, pp. 269–279, 2007.
- H. S. Dewang, P. K. Mohanty, and S. Kundu, “A robust path planning for mobile robot using smart particle swarm optimization,” Procedia computer science, vol. 133, pp. 290–297, 2018.
- A. I. Panov, K. S. Yakovlev, and R. Suvorov, “Grid path planning with deep reinforcement learning: Preliminary results,” Procedia computer science, vol. 123, pp. 347–353, 2018.
- C. Di Franco and G. Buttazzo, “Coverage path planning for uavs photogrammetry with energy and resolution constraints,” Journal of Intelligent & Robotic Systems, vol. 83, pp. 445–462, 2016.
- C. Sagües Blazquiz and E. Montijano Muñoz, “Multi-robot persistent coverage in complex environments.”
- S. Petitjean, “A survey of methods for recovering quadrics in triangle meshes,” ACM Computing Surveys (CSUR), vol. 34, no. 2, pp. 211–262, 2002.
- D. R. Smith, “The design of divide and conquer algorithms,” Science of Computer Programming, vol. 5, pp. 37–58, 1985.
- K. L. Hoffman, M. Padberg, G. Rinaldi et al., “Traveling salesman problem,” Encyclopedia of operations research and management science, vol. 1, pp. 1573–1578, 2013.
- J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
- P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V. Koltun, J. Kosecka, J. Malik, R. Mottaghi, M. Savva, and A. R. Zamir, “On evaluation of embodied navigation agents.” [Online]. Available: http://arxiv.org/abs/1807.06757
- M. Zhao, P. Anderson, V. Jain, S. Wang, A. Ku, J. Baldridge, and E. Ie, “On the evaluation of vision-and-language navigation instructions,” 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.