Papers
Topics
Authors
Recent
Search
2000 character limit reached

Self-Improving Autonomous Underwater Manipulation

Published 24 Oct 2024 in cs.RO | (2410.18969v1)

Abstract: Underwater robotic manipulation faces significant challenges due to complex fluid dynamics and unstructured environments, causing most manipulation systems to rely heavily on human teleoperation. In this paper, we introduce AquaBot, a fully autonomous manipulation system that combines behavior cloning from human demonstrations with self-learning optimization to improve beyond human teleoperation performance. With extensive real-world experiments, we demonstrate AquaBot's versatility across diverse manipulation tasks, including object grasping, trash sorting, and rescue retrieval. Our real-world experiments show that AquaBot's self-optimized policy outperforms a human operator by 41% in speed. AquaBot represents a promising step towards autonomous and self-improving underwater manipulation systems. We open-source both hardware and software implementation details.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. O. Khatib, X. Yeh, G. Brantner, B. Soe, B. Kim, S. Ganguly, H. Stuart, S. Wang, M. Cutkosky, A. Edsinger, P. Mullins, M. Barham, C. R. Voolstra, K. N. Salama, M. L’Hour, and V. Creuze, “Ocean one: A robotic avatar for oceanic discovery,” IEEE Robotics and Automation Magazine, vol. 23, no. 4, pp. 20–29, 2016.
  2. H. Stuart, S. Wang, O. Khatib, and M. R. Cutkosky, “The ocean one hands: An adaptive design for robust marine manipulation,” The International Journal of Robotics Research, vol. 36, no. 2, pp. 150–166, 2017.
  3. G. Brantner and O. Khatib, “Controlling ocean one: Human–robot collaboration for deep-sea manipulation,” Journal of Field Robotics, vol. 38, no. 1, pp. 28–51, 2021.
  4. H. Huang, Q. Tang, J. Li, W. Zhang, X. Bao, H. Zhu, and G. Wang, “A review on underwater autonomous environmental perception and target grasp, the challenge of robotic organism capture,” Ocean Engineering, vol. 195, p. 106644, 2020.
  5. S. Aldhaheri, G. De Masi, È. Pairet, and P. Ardón, “Underwater robot manipulation: Advances, challenges and prospective ventures,” in OCEANS 2022-Chennai.   IEEE, 2022, pp. 1–7.
  6. K. Leabourne and S. Rock, “Model development of an underwater manipulator for coordinated arm-vehicle control,” in IEEE Oceanic Engineering Society. OCEANS’98. Conference Proceedings (Cat. No.98CH36259), vol. 2, 1998, pp. 941–946 vol.2.
  7. S. T. Wilson, A. P. Sudheer, and S. Mohan, “Dynamic modelling, simulation and spatial control of an underwater robot equipped with a planar manipulator,” in 2011 International Conference on Process Automation, Control and Computing, 2011, pp. 1–6.
  8. L. Gümüşel and N. G. Özmen, “Modelling and control of manipulators with flexible links working on land and underwater environments,” Robotica, vol. 29, no. 3, pp. 461–470, 2011.
  9. C. Barbălată, M. W. Dunnigan, and Y. Pétillot, “Dynamic coupling and control issues for a lightweight underwater vehicle manipulator system,” in 2014 Oceans - St. John’s, 2014, pp. 1–6.
  10. E. Kelasidi, K. Y. Pettersen, J. T. Gravdahl, and P. Liljebäck, “Modeling of underwater snake robots,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014, pp. 4540–4547.
  11. J. Sverdrup-Thygeson, E. Kelasidi, K. Pettersen, and J. Gravdahl, “Modeling of underwater swimming manipulators**this research was partly funded by the research council of norway through the centres of excellence funding scheme, project no. 223254 ntnu amos, and partly funded by vista, a basic research program in collaboration between the norwegian academy of science and letters, and statoil.” IFAC-PapersOnLine, vol. 49, no. 23, pp. 81–88, 2016, 10th IFAC Conference on Control Applications in Marine SystemsCAMS 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405896316319127
  12. J. Sverdrup-Thygeson, E. Kelasidi, K. Y. Pettersen, and J. T. Gravdahl, “The underwater swimming manipulator—a bioinspired solution for subsea operations,” IEEE Journal of Oceanic Engineering, vol. 43, no. 2, pp. 402–417, 2018.
  13. P. Fairchild, Y. Mei, and X. Tan, “Physics-informed online estimation of stiffness and shape of soft robotic manipulators,” IEEE Control Systems Letters, 2023.
  14. C. Barbalata, M. W. Dunnigan, and Y. Petillot, “Position/force operational space control for underwater manipulation,” Robotics and Autonomous Systems, vol. 100, pp. 150–159, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S092188901730386X
  15. C. Barbălată, M. W. Dunnigan, and Y. Pétillot, “Reduction of the dynamic coupling in an underwater vehicle-manipulator system using an inverse dynamic model approach,” IFAC-PapersOnLine, vol. 48, no. 2, pp. 44–49, 2015, 4th IFAC Workshop onNavigation, Guidance and Controlof Underwater VehiclesNGCUV 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405896315002475
  16. B. O. A. Haugaløkken, E. K. Jørgensen, and I. Schjølberg, “Experimental validation of end-effector stabilization for underwater vehicle-manipulator systems in subsea operations,” Robotics and Autonomous Systems, vol. 109, pp. 1–12, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889018300952
  17. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.
  18. Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, D. d. L. Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq, et al., “Deepmind control suite,” arXiv preprint arXiv:1801.00690, 2018.
  19. Y. R. Jadhav and Y. S. Moon, “Rov manipulation from observation and exploration using deep reinforcement learning,” Journal of Advanced Research in Ocean Engineering, vol. 3, no. 3, pp. 136–148, 2017.
  20. I. Carlucho, M. De Paula, C. Barbalata, and G. G. Acosta, “A reinforcement learning control approach for underwater manipulation under position and torque constraints,” in Global Oceans 2020: Singapore – U.S. Gulf Coast.   IEEE, Oct. 2020. [Online]. Available: http://dx.doi.org/10.1109/IEEECONF38699.2020.9389378
  21. H. Yang, J. Liu, X. Fang, X. Chen, Z. Gong, S. Wang, S. Kong, J. Yu, and L. Wen, “Prediction model-based learning adaptive control for underwater grasping of a soft manipulator,” International Journal of Intelligent Robotics and Applications, vol. 5, pp. 337–353, 2021.
  22. A. Nikou, C. K. Verginis, and D. V. Dimarogonas, “A tube-based mpc scheme for interaction control of underwater vehicle manipulator systems,” in 2018 IEEE/OES Autonomous Underwater Vehicle Workshop (AUV), 2018, pp. 1–6.
  23. T. Salloom, X. Yu, W. He, and O. Kaynak, “Adaptive neural network control of underwater robotic manipulators tuned by a genetic algorithm,” Journal of Intelligent & Robotic Systems, vol. 97, pp. 657–672, 2020.
  24. I. Carlucho, D. W. Stephens, and C. Barbalata, “An adaptive data-driven controller for underwater manipulators with variable payload,” Applied Ocean Research, vol. 113, p. 102726, 2021.
  25. J. P. Oubre, I. Carlucho, and C. Barbalata, “Data-driven controllers and the need for perception systems in underwater manipulation,” 2021. [Online]. Available: https://arxiv.org/abs/2109.10327
  26. K. Y. Chee and M. A. Hsieh, “LEARNEST: LEARNing Enhanced Model-based State ESTimation for robots using knowledge-based neural ordinary differential equations,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 11 590–11 596.
  27. S. Rahman, A. Quattrini Li, and I. Rekleitis, “Svin2: A multi-sensor fusion-based underwater slam system,” The International Journal of Robotics Research, vol. 41, no. 11-12, pp. 1022–1042, 2022.
  28. A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “Monoslam: Real-time single camera slam,” IEEE transactions on pattern analysis and machine intelligence, vol. 29, no. 6, pp. 1052–1067, 2007.
  29. H. Matsuki, R. Murai, P. H. Kelly, and A. J. Davison, “Gaussian splatting slam,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 039–18 048.
  30. B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering.” ACM Trans. Graph., vol. 42, no. 4, pp. 139–1, 2023.
  31. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
  32. R. Liu, R. Wu, B. Van Hoorick, P. Tokmakov, S. Zakharov, and C. Vondrick, “Zero-1-to-3: Zero-shot one image to 3d object,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 9298–9309.
  33. S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud, “Dust3r: Geometric 3d vision made easy,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 697–20 709.
  34. Y. Tang, C. Zhu, R. Wan, C. Xu, and B. Shi, “Neural underwater scene representation,” 2024.
  35. T. Zhang and M. Johnson-Roberson, “Beyond nerf underwater: Learning neural reflectance fields for true color correction of marine imagery,” IEEE Robotics and Automation Letters, 2023.
  36. M. Kalakrishnan, L. Righetti, P. Pastor, and S. Schaal, “Learning force control policies for compliant manipulation,” in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011, pp. 4639–4644.
  37. S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” Journal of Machine Learning Research, vol. 17, no. 39, pp. 1–40, 2016.
  38. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  39. T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al., “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905, 2018.
  40. I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, et al., “Solving rubik’s cube with a robot hand,” arXiv preprint arXiv:1910.07113, 2019.
  41. D. Ha and J. Schmidhuber, “World models,” arXiv preprint arXiv:1803.10122, 2018.
  42. D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in International conference on machine learning.   PMLR, 2019, pp. 2555–2565.
  43. D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” arXiv preprint arXiv:1912.01603, 2019.
  44. P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Daydreamer: World models for physical robot learning,” in Conference on robot learning.   PMLR, 2023, pp. 2226–2240.
  45. J. Liang, R. Liu, E. Ozguroglu, S. Sudhakar, A. Dave, P. Tokmakov, S. Song, and C. Vondrick, “Dreamitate: Real-world visuomotor policy learning via video generation,” arXiv preprint arXiv:2406.16862, 2024.
  46. A. Zeng, S. Song, J. Lee, A. Rodriguez, and T. Funkhouser, “Tossingbot: Learning to throw arbitrary objects with residual physics,” IEEE Transactions on Robotics, vol. 36, no. 4, pp. 1307–1319, 2020.
  47. Z. Xu, C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song, “Dextairity: Deformable manipulation can be a breeze,” arXiv preprint arXiv:2203.01197, 2022.
  48. H. Ha and S. Song, “Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding,” in Conference on Robot Learning.   PMLR, 2022, pp. 24–33.
  49. C. Chi, B. Burchfiel, E. Cousineau, S. Feng, and S. Song, “Iterative residual policy: for goal-conditioned dynamic manipulation of deformable objects,” The International Journal of Robotics Research, vol. 43, no. 4, pp. 389–404, 2024.
  50. R. Liu, J. Liang, S. Sudhakar, H. Ha, C. Chi, S. Song, and C. Vondrick, “Paperbot: Learning to design real-world tools using paper,” arXiv preprint arXiv:2403.09566, 2024.
  51. C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137, 2023.
  52. T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” 2023. [Online]. Available: https://arxiv.org/abs/2304.13705
  53. S. Lee, Y. Wang, H. Etukuru, H. J. Kim, N. M. M. Shafiullah, and L. Pinto, “Behavior generation with latent actions,” arXiv preprint arXiv:2403.03181, 2024.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free