Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation (2402.19432v1)

Published 29 Feb 2024 in cs.RO

Abstract: Recent years in robotics and imitation learning have shown remarkable progress in training large-scale foundation models by leveraging data across a multitude of embodiments. The success of such policies might lead us to wonder: just how diverse can the robots in the training set be while still facilitating positive transfer? In this work, we study this question in the context of heterogeneous embodiments, examining how even seemingly very different domains, such as robotic navigation and manipulation, can provide benefits when included in the training data for the same model. We train a single goal-conditioned policy that is capable of controlling robotic arms, quadcopters, quadrupeds, and mobile bases. We then investigate the extent to which transfer can occur across navigation and manipulation on these embodiments by framing them as a single goal-reaching task. We find that co-training with navigation data can enhance robustness and performance in goal-conditioned manipulation with a wrist-mounted camera. We then deploy our policy trained only from navigation-only and static manipulation-only data on a mobile manipulator, showing that it can control a novel embodiment in a zero-shot manner. These results provide evidence that large-scale robotic policies can benefit from data collected across various embodiments. Further information and robot videos can be found on our project website http://extreme-cross-embodiment.github.io.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. R. Bommasani et al., “On the opportunities and risks of foundation models,” 2022.
  2. S. Dasari, F. Ebert, S. Tian, S. Nair, B. Bucher, K. Schmeckpeper, S. Singh, S. Levine, and C. Finn, “Robonet: Large-scale multi-robot learning,” in Annual Conference on Robot Learning (CoRL), 2019.
  3. H.-S. Fang, H. Fang, Z. Tang, J. Liu, C. Wang, J. Wang, H. Zhu, and C. Lu, “Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot,” 2023.
  4. Open X-Embodiment Collaboration et al., “Open x-embodiment: Robotic learning datasets and rt-x models,” arXiv preprint arXiv:2310.08864, 2023.
  5. D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine, “GNM: A General Navigation Model to Drive Any Robot,” in International Conference on Robotics and Automation (ICRA), 2023.
  6. Y.-H. H. Tsai, V. Dhar, J. Li, B. Zhang, and J. Zhang, “Multimodal large language model for visual navigation,” 2023.
  7. T.-H. Wang, A. Maalouf, W. Xiao, Y. Ban, A. Amini, G. Rosman, S. Karaman, and D. Rus, “Drive anywhere: Generalizable end-to-end autonomous driving with multi-modal foundation models,” arXiv preprint arXiv:2310.17642, 2023.
  8. J. Lin, A. Zeng, S. Lu, Y. Cai, R. Zhang, H. Wang, and L. Zhang, “Motion-x: A large-scale 3d expressive whole-body human motion dataset,” Advances in Neural Information Processing Systems, 2023.
  9. W. Yu, J. Tan, C. K. Liu, and G. Turk, “Preparing for the unknown: Learning a universal policy with online system identification,” in Robotics: Science and Systems (RSS), 2017.
  10. T. Chen, A. Murali, and A. Gupta, “Hardware conditioned policies for multi-robot transfer learning,” 2019.
  11. H. You, T. Yang, Y. Zheng, J. Hao, and E. Taylor, Matthew, “Cross-domain adaptive transfer reinforcement learning based on state-action correspondence,” in Uncertainty in Artificial Intelligence, 2022.
  12. E. S. Hu, K. Huang, O. Rybkin, and D. Jayaraman, “Know thyself: Transferable visual control policies through robot-awareness,” in International Conference on Learning Representations (ICLR), 2022.
  13. G. Salhotra, xI Chun Arthur Liu, and G. Sukhatme, “Bridging action space mismatch in learning from demonstrations,” arXiv preprint arXiv:2304.03833, 2023.
  14. J. H. Yang, D. Sadigh, and C. Finn, “Polybot: Training one policy across robots while embracing variability,” in Annual Conference on Robot Learning (CoRL), 2023.
  15. P. F. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, “Transfer from simulation to real world through learning deep inverse dynamics model,” ArXiv preprint arXiv:1610.03518, 2016.
  16. F. Sadeghi, A. Toshev, E. Jang, and S. Levine, “Sim2real view invariant visual servoing by recurrent control,” in International Conference on Robotics and Automation (ICRA), 2017.
  17. X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization,” in International Conference on Robotics and Automation (ICRA).   IEEE, 2018.
  18. Q. Zhang, T. Xiao, A. A. Efros, L. Pinto, and X. Wang, “Learning cross-domain correspondence for control with dynamics cycle-consistency,” in International Conference on Learning Representations (ICLR), 2021.
  19. C. Devin, A. Gupta, T. Darrell, P. Abbeel, and S. Levine, “Learning modular neural network policies for multi-task and multi-robot transfer,” in International Conference on Robotics and Automation (ICRA).
  20. T. Wang, R. Liao, and S. F. Jimmy Ba, “Nervenet: Learning structured policy with graph neural networks,” in International Conference on Learning Representations (ICLR), 2018.
  21. W. Huang, I. Mordatch, and D. Pathak, “One policy to control them all: Shared modular policies for agent-agnostic control,” in International Conference on Machine Learning (ICML), 2020.
  22. A. Ghadirzadeh, X. Chen, P. Poklukar, C. Finn, M. Björkman, and D. Kragic, “Bayesian meta-learning for few-shot policy adaptation across robotic platforms,” in International Conference on Intelligent Robots and Systems (IROS), 2021.
  23. N. Hirose, D. Shah, A. Sridhar, and S. Levine, “Exaug: Robot-conditioned navigation policies via geometric experience augmentation,” in International Conference on Robotics and Automation (ICRA), 2023.
  24. M. Attarian, M. A. Asif, J. Liu, R. Hari, A. Garg, I. Gilitschenski, and J. Tompson, “Geometry matching for multi-embodiment grasping,” 2023.
  25. A. Loquercio, A. I. Maqueda, C. R. D. Blanco, and D. Scaramuzza, “Dronet: Learning to fly by driving,” IEEE Robotics and Automation Letters, 2018.
  26. R. Martín-Martín, M. A. Lee, R. Gardner, S. Savarese, J. Bohg, and A. Garg, “Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks,” 2019.
  27. M. Chang, A. Gupta, and S. Gupta, “Semantic visual navigation by watching youtube videos,” in Neural Information Processing Systems (NeurIPS), 2020.
  28. L. Shao, F. Ferreira, M. Jorda, V. Nambiar, J. Luo, E. Solowjow, J. A. Ojea, O. Khatib, and J. Bohg, “Unigrasp: Learning a unified model to grasp with multifingered robotic hands,” IEEE Robotics and Automation Letters, 2020.
  29. K. Kang, G. Kahn, and S. Levine, “Hierarchically integrated models: Learning to navigate from heterogeneous robots,” in Annual Conference on Robot Learning (CoRL), 2021.
  30. S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak, “Affordances from human videos as a versatile representation for robotics,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 01–13.
  31. D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine, “ViNT: A foundation model for visual navigation,” in Annual Conference on Robot Learning (CoRL), 2023.
  32. Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” 2016.
  33. K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” 2017.
  34. A. Gupta, C. Devin, Y. Liu, P. Abbeel, and S. Levine, “Learning invariant feature spaces to transfer skills with reinforcement learning,” 2017.
  35. K. Fang, Y. Bai, S. Hinterstoisser, S. Savarese, and M. Kalakrishnan, “Multi-task domain adaptation for deep learning of instance grasping from simulation,” in International Conference on Robotics and Automation (ICRA), 2018, pp. 3516–3523.
  36. N. H. Kim, Z. Xie, and M. van de Panne, “Learning to correspond dynamical systems,” 2020.
  37. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in International Conference on Computer Vision (ICCV), 2017, pp. 2242–2251.
  38. H. You, T. Yang, Y. Zheng, J. Hao, and E. Taylor, Matthew, “Cross-domain adaptive transfer reinforcement learning based on state-action correspondence,” in Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, ser. Proceedings of Machine Learning Research, J. Cussens and K. Zhang, Eds., vol. 180.   PMLR, 01–05 Aug 2022, pp. 2299–2309.
  39. P. Sharma, L. Mohan, L. Pinto, and A. Gupta, “Multiple interactions made easy (mime): Large scale demonstrations data for imitation,” 2018.
  40. A. Mandlekar, Y. Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, S. Savarese, and L. Fei-Fei, “Roboturk: A crowdsourcing platform for robotic skill learning through imitation,” in Annual Conference on Robot Learning (CoRL), 2018.
  41. S. Young, D. Gandhi, S. Tulsiani, A. Gupta, P. Abbeel, and L. Pinto, “Visual imitation made easy,” 2020.
  42. E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” in Conference on Robot Learning, 2021.
  43. F. Ebert, Y. Yang, K. Schmeckpeper, B. Bucher, G. Georgakis, K. Daniilidis, C. Finn, and S. Levine, “Bridge data: Boosting generalization of robotic skills with cross-domain datasets,” in Robotics: Science and Systems, 2022.
  44. T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” 2023.
  45. N. Hirose, F. Xia, R. Martín-Martín, A. Sadeghian, and S. Savarese, “Deep visual mpc-policy learning for navigation,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3184–3191, 2019.
  46. H. Karnan et al., “Socially CompliAnt Navigation Dataset (SCAND): A Large-Scale Dataset Of Demonstrations For Social Navigation,” IEEE Robotics and Automation Letters, 2022.
  47. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 3213–3223.
  48. A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 3354–3361.
  49. J. Geyer et al., “A2d2: Audi autonomous driving dataset,” 2020.
  50. X. Huang, P. Wang, X. Cheng, D. Zhou, Q. Geng, and R. Yang, “The apolloscape open dataset for autonomous driving and its application,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
  51. F. Yu, H. Chen, X. Wang, W. Xian, Y. Chen, F. Liu, V. Madhavan, and T. Darrell, “Bdd100k: A diverse driving dataset for heterogeneous multitask learning,” 2020.
  52. S. Ettinger et al., “Large scale interactive motion forecasting for autonomous driving: The waymo open motion dataset,” in International Conference on Computer Vision (ICCV), 2021.
  53. S. Reed et al., “A generalist agent,” 2022.
  54. K. Bousmalis et al., “Robocat: A self-improving foundation agent for robotic manipulation,” ArXiv, 2023.
  55. Octo Model Team et al., “Octo: An open-source generalist robot policy,” https://octo-models.github.io, 2023.
  56. A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado, “Gaia-1: A generative world model for autonomous driving,” 2023.
  57. W. Goodwin, S. Vaze, I. Havoutis, and I. Posner, “Zero-shot category-level object pose estimation,” in Proceedings of the European Conference on Computer Vision (ECCV), 2022.
  58. Y. Zhu, A. Joshi, P. Stone, and Y. Zhu, “Viola: Imitation learning for vision-based manipulation with object proposal priors,” 2023.
  59. Y. Zhu, Z. Jiang, P. Stone, and Y. Zhu, “Learning generalizable manipulation policies with object-centric 3d representations,” in 7th Annual Conference on Robot Learning, 2023.
  60. R. Bonatti, S. Vemprala, S. Ma, F. Frujeri, S. Chen, and A. Kapoor, “Pact: Perception-action causal transformer for autoregressive robotics pre-training,” 2022.
  61. S. Karamcheti, S. Nair, A. S. Chen, T. Kollar, C. Finn, D. Sadigh, and P. Liang, “Language-driven representation learning for robotics,” in Robotics: Science and Systems (RSS), 2023.
  62. Y. Du, M. Yang, B. Dai, H. Dai, O. Nachum, J. B. Tenenbaum, D. Schuurmans, and P. Abbeel, “Learning universal policies via text-guided video generation,” arXiv e-prints, pp. arXiv–2302, 2023.
  63. Z. Xian, T. Gervet, Z. Xu, Y.-L. Qiao, T.-H. Wang, and Y. Wang, “Towards generalist robots: A promising paradigm via generative simulation,” 2023.
  64. M. Yang, Y. Du, K. Ghasemipour, J. Tompson, L. Kaelbling, D. Schuurmans, and P. Abbeel, “Learning interactive real-world simulators,” 2024.
  65. A. Brohan et al., “Rt-1: Robotics transformer for real-world control at scale,” 2023.
  66. ——, “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” 2023.
  67. E. Rosete-Beas, O. Mees, G. Kalweit, J. Boedecker, and W. Burgard, “Latent plans for task agnostic offline reinforcement learning,” in Proceedings of the 6th Conference on Robot Learning (CoRL), 2022.
  68. S. Dass, J. Yapeter, J. Zhang, J. Zhang, K. Pertsch, S. Nikolaidis, and J. J. Lim, “Clvr jaco play dataset,” 2023. [Online]. Available: https://github.com/clvrai/clvr_jaco_play_dataset
  69. A. Mandlekar, J. Booher, M. Spero, A. Tung, A. Gupta, Y. Zhu, A. Garg, S. Savarese, and L. Fei-Fei, “Scaling robot supervision to hundreds of hours with roboturk: Robotic manipulation dataset through human reasoning and dexterity,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 1048–1055.
  70. J. Pari, M. Shafiullah, S. Arunachalam, and L. Pinto, “Visual imitation through nearest neighbors (vinn) implementation,” 2021.
  71. Y. Zhu, A. Joshi, P. Stone, and Y. Zhu, “Viola: Imitation learning for vision-based manipulation with object proposal priors,” 6th Annual Conference on Robot Learning (CoRL), 2022.
  72. L. Y. Chen, S. Adebola, and K. Goldberg, “Berkeley UR5 demonstration dataset,” https://sites.google.com/view/berkeley-ur5/home.
  73. G. Zhou, V. Dean, M. K. Srirama, A. Rajeswaran, J. Pari, K. Hatch, A. Jain, T. Yu, P. Abbeel, L. Pinto, C. Finn, and A. Gupta, “Train offline, test online: A real robot learning benchmark,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023.
  74. D. Shah, B. Eysenbach, N. Rhinehart, and S. Levine, “Rapid exploration for open-world navigation with latent goal models,” in Annual Conference on Robot Learning (CoRL), 2022.
  75. G. Kahn, A. Villaflor, B. Ding, P. Abbeel, and S. Levine, “Self-Supervised Deep RL with Generalized Computation Graphs for Robot Navigation,” in International Conference on Robotics and Automation (ICRA), 2018.
  76. A. Shaban, X. Meng, J. Lee, B. Boots, and D. Fox, “Semantic terrain classification for off-road autonomous driving,” in Conference on Robot Learning (CoRL), 2022.
  77. S. Triest et al., “TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models,” in International Conference on Robotics and Automation (ICRA), 2022.
  78. N. Hirose, D. Shah, A. Sridhar, and S. Levine, “Sacson: Scalable autonomous control for social navigation,” IEEE Robotics and Automation Letters, 2024.
  79. S. Ramos, S. Girgin, L. Hussenot, D. Vincent, H. Yakubovich, D. Toyama, A. Gergely, P. Stanczyk, R. Marinier, J. Harmsen, O. Pietquin, and N. Momchev, “Rlds: an ecosystem to generate, share and use datasets in reinforcement learning,” 2021.
  80. M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97.   PMLR, 09–15 Jun 2019, pp. 6105–6114.
  81. C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proceedings of Robotics: Science and Systems (RSS), 2023.
  82. A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martín-Martín, “What matters in learning from offline human demonstrations for robot manipulation,” in arXiv preprint arXiv:2108.03298, 2021.
  83. A. Sridhar, D. Shah, C. Glossop, and S. Levine, “Nomad: Goal masked diffusion policies for navigation and exploration,” 2023.
  84. Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” 2024.
  85. K. Hsu, M. J. Kim, R. Rafailov, J. Wu, and C. Finn, “Vision-based manipulators need to also see from their hands,” 2022.
  86. A. Xie, L. Lee, T. Xiao, and C. Finn, “Decomposing the generalization gap in imitation learning for visual robotic manipulation,” 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Jonathan Yang (9 papers)
  2. Catherine Glossop (4 papers)
  3. Arjun Bhorkar (5 papers)
  4. Dhruv Shah (48 papers)
  5. Quan Vuong (41 papers)
  6. Chelsea Finn (264 papers)
  7. Dorsa Sadigh (162 papers)
  8. Sergey Levine (531 papers)
Citations (25)
View on Semantic Scholar

Summary

Analysis of Heterogeneous Cross-Embodiment Learning for Manipulation and Navigation

The paper "Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation" investigates the potential for cross-embodiment learning in robotics, focusing on manipulation and navigation tasks. This research explores the extent to which heterogeneous datasets can facilitate knowledge transfer across different robot embodiments, thus enabling a single policy to control a wide variety of robots. The work contributes empirical findings to the field by demonstrating the benefits and challenges of utilizing foundation models to unify diverse embodiments, showcasing the application to drones, wheeled robots, quadrupeds, and more.

Summary and Numerical Results

The paper empirically evaluates whether co-training with navigation data can enhance manipulation tasks and vice versa. The salient result is a remarkable improvement in task success rates: the co-trained policies achieve about a 20% higher success rate in manipulation tasks compared to policies trained exclusively on manipulation data. Also noteworthy is the 5-7% performance boost observed in navigation across different robotic platforms, driven by incorporating manipulation datasets. These results confirm the hypothesis that data from significantly diverse systems contain transferable insights that can enhance performance across tasks traditionally viewed as distinct.

Implications and Speculative Insights on Future AI Developments

Theoretical Implications: From a theoretical standpoint, these findings suggest a profound implication: the underlying learning mechanisms of different robotic tasks, such as navigation and manipulation, may share common sensory-motor principles that can be exploited through large-scale data-driven learning. The research challenges the traditional approach of embodiment-specific datasets and encourages a paradigm where diverse data sources are viewed as potential assets for enhancing generalization.

Practical Implications: Practically, this research hints at the potential for developing 'generalist' robotic models that could operate a range of devices without task-specific tuning. Such advancements could drastically reduce the costs and time associated with collecting and training on separate datasets for new embodiments, thus accelerating the deployment of robotic applications in various real-world scenarios, from dynamic environments to hazardous settings not directly encountered during initial learning.

Future Developments in AI and Robotics

A key direction for future research involves refining the architecture and training methods to harness the benefits observed in this paper. Specifically, extending the incorporation of more sophisticated task modalities, such as language-based instructions, could enhance usability for non-expert users. Additionally, pursuing scalability tests could further confirm the robustness of these findings across unprecedentedly large datasets and novel tasks. Optimizing architectures to seamlessly integrate varying degrees of freedom, especially for complex robots like multi-fingered hands, will also advance this field substantially.

Conclusion

The results of this paper mark a significant step toward realizing robots that can generalize across multiple tasks and embodiments, bridging navigation and manipulation more closely. This convergence demands further exploration of scalable architectures and diverse data integration strategies. The empirical evidence presented suggests that with continued research, truly universal robot foundation models could become viable, offering flexibility and adaptability at levels previously thought unattainable. As the field progresses, such foundational work will underpin broader applications and facilitate breakthroughs in autonomous systems.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube