Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

OceanPlan: Hierarchical Planning and Replanning for Natural Language AUV Piloting in Large-scale Unexplored Ocean Environments (2403.15369v1)

Published 22 Mar 2024 in cs.RO

Abstract: We develop a hierarchical LLM-task-motion planning and replanning framework to efficiently ground an abstracted human command into tangible Autonomous Underwater Vehicle (AUV) control through enhanced representations of the world. We also incorporate a holistic replanner to provide real-world feedback with all planners for robust AUV operation. While there has been extensive research in bridging the gap between LLMs and robotic missions, they are unable to guarantee success of AUV applications in the vast and unknown ocean environment. To tackle specific challenges in marine robotics, we design a hierarchical planner to compose executable motion plans, which achieves planning efficiency and solution quality by decomposing long-horizon missions into sub-tasks. At the same time, real-time data stream is obtained by a replanner to address environmental uncertainties during plan execution. Experiments validate that our proposed framework delivers successful AUV performance of long-duration missions through natural language piloting.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. F. Zhang, D. M. Fratantoni, D. A. Paley, J. M. Lund, and N. E. Leonard, “Control of coordinated patterns for ocean sampling,” International Journal of Control, vol. 80, no. 7, pp. 1186–1199, 2007.
  2. D. A. Paley, F. Zhang, and N. E. Leonard, “Cooperative control for ocean sampling: The glider coordinated control system,” IEEE Transactions on Control Systems Technology, vol. 16, no. 4, pp. 735–744, 2008.
  3. S. Tellex, N. Gopalan, H. Kress-Gazit, and C. Matuszek, “Robots that use language,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, pp. 25–55, 2020.
  4. W. Huang, P. Abbeel, D. Pathak, and I. Mordatch, “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,” in International Conference on Machine Learning.   PMLR, 2022, pp. 9118–9147.
  5. J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 9493–9500.
  6. T. Lin, C. Yue, Z. Liu, and X. Cao, “Generalized multi-level replanning tamp framework for dynamic environment,” arXiv preprint arXiv:2310.14816, 2023.
  7. I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: program generation for situated robot task planning using large language models,” Autonomous Robots, pp. 1–14, 2023.
  8. W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar et al., “Inner monologue: Embodied reasoning through planning with language models,” in Conference on Robot Learning.   PMLR, 2023, pp. 1769–1782.
  9. W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei, “Voxposer: Composable 3d value maps for robotic manipulation with language models,” in Conference on Robot Learning.   PMLR, 2023, pp. 540–562.
  10. D. Shah, M. R. Equi, B. Osiński, F. Xia, S. Levine et al., “Navigation with large language models: Semantic guesswork as a heuristic for planning,” in 7th Annual Conference on Robot Learning, 2023.
  11. M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022.
  12. Z. Dai, A. Asgharivaskasi, T. Duong, S. Lin, M.-E. Tzes, G. Pappas, and N. Atanasov, “Optimal scene graph planning with large language model guidance,” arXiv preprint arXiv:2309.09182, 2023.
  13. B. Yu, H. Kasaei, and M. Cao, “L3mvn: Leveraging large language models for visual target navigation,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 3554–3560.
  14. B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone, “Llm+ p: Empowering large language models with optimal planning proficiency,” arXiv preprint arXiv:2304.11477, 2023.
  15. K. Valmeekam, A. Olmo, S. Sreedharan, and S. Kambhampati, “Large language models still can’t plan (a benchmark for llms on planning and reasoning about change),” in NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022.
  16. A. Rajvanshi, K. Sikka, X. Lin, B. Lee, H.-P. Chiu, and A. Velasquez, “Saynav: Grounding large language models for dynamic planning to navigation in new environments,” arXiv preprint arXiv:2309.04077, 2023.
  17. C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Pérez, “Integrated task and motion planning,” Annual review of control, robotics, and autonomous systems, vol. 4, pp. 265–293, 2021.
  18. M. Gualtieri and R. Platt, “Robotic pick-and-place with uncertain object instance segmentation and shape completion,” IEEE robotics and automation letters, vol. 6, no. 2, pp. 1753–1760, 2021.
  19. H. Zhang, S.-H. Chan, J. Zhong, J. Li, P. Kolapo, S. Koenig, Z. Agioutantis, S. Schafrik, and S. Nikolaidis, “Multi-robot geometric task-and-motion planning for collaborative manipulation tasks,” Autonomous Robots, pp. 1–22, 2023.
  20. M. Burke, K. Lu, D. Angelov, A. Straižys, C. Innes, K. Subr, and S. Ramamoorthy, “Learning rewards from exploratory demonstrations using probabilistic temporal ranking,” Autonomous Robots, vol. 47, no. 6, pp. 733–751, 2023.
  21. Y. Ding, X. Zhang, C. Paxton, and S. Zhang, “Task and motion planning with large language models for object rearrangement,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 2086–2092.
  22. T. Lozano-Pérez, M. T. Mason, and R. H. Taylor, “Automatic synthesis of fine-motion strategies for robots,” The International Journal of Robotics Research, vol. 3, no. 1, pp. 3–24, 1984.
  23. A. Curtis, X. Fang, L. P. Kaelbling, T. Lozano-Pérez, and C. R. Garrett, “Long-horizon manipulation of unknown objects via task and motion planning with estimated affordances,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 1940–1946.
  24. M. Hou, Y. Li, F. Zhang, S. Sundaram, and S. Mou, “An interleaved algorithm for integration of robotic task and motion planning,” in 2023 American Control Conference (ACC).   IEEE, 2023, pp. 539–544.
  25. C. R. Garrett, C. Paxton, T. Lozano-Pérez, L. P. Kaelbling, and D. Fox, “Online replanning in belief space for partially observable task and motion problems,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 5678–5684.
  26. P. Sharma, B. Sundaralingam, V. Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting Robot Plans with Natural Language Feedback,” in Proceedings of Robotics: Science and Systems, New York City, NY, USA, June 2022.
  27. D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence, “Palm-e: An embodied multimodal language model,” 2023.
  28. R. Munos et al., “From bandits to monte-carlo tree search: The optimistic principle applied to optimization and planning,” Foundations and Trends® in Machine Learning, vol. 7, no. 1, pp. 1–129, 2014.
  29. L. Smith, J. C. Kew, X. B. Peng, S. Ha, J. Tan, and S. Levine, “Legged robots that keep on learning: Fine-tuning locomotion policies in the real world,” in 2022 International Conference on Robotics and Automation (ICRA).   IEEE, 2022, pp. 1593–1599.
  30. Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi, “Target-driven visual navigation in indoor scenes using deep reinforcement learning,” in 2017 IEEE international conference on robotics and automation (ICRA).   IEEE, 2017, pp. 3357–3364.
  31. A. Devo, G. Mezzetti, G. Costante, M. L. Fravolini, and P. Valigi, “Towards generalization in target-driven visual navigation by using deep reinforcement learning,” IEEE Transactions on Robotics, vol. 36, no. 5, pp. 1546–1561, 2020.
  32. D. Nau, Y. Cao, A. Lotem, and H. Munoz-Avila, “Shop: Simple hierarchical ordered planner,” in Proceedings of the 16th international joint conference on Artificial intelligence-Volume 2, 1999, pp. 968–973.
  33. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.
  34. E. Potokar, S. Ashford, M. Kaess, and J. Mangelson, “HoloOcean: An underwater robotics simulator,” in Proc. IEEE Intl. Conf. on Robotics and Automation, ICRA, Philadelphia, PA, USA, May 2022.
  35. C. Wang, F. Zhang, and D. Schaefer, “Dynamic modeling of an autonomous underwater vehicle,” Journal of Marine Science and Technology, vol. 20, pp. 199–212, 2015.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now