Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Visual Whole-Body Control for Legged Loco-Manipulation (2403.16967v5)

Published 25 Mar 2024 in cs.RO, cs.CV, and cs.LG

Abstract: We study the problem of mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by conducting whole-body control. That is, the robot can control the legs and the arm at the same time to extend its workspace. We propose a framework that can conduct the whole-body control autonomously with visual observations. Our approach, namely Visual Whole-Body Control(VBC), is composed of a low-level policy using all degrees of freedom to track the body velocities along with the end-effector position, and a high-level policy proposing the velocities and end-effector position based on visual inputs. We train both levels of policies in simulation and perform Sim2Real transfer for real robot deployment. We perform extensive experiments and show significant improvements over baselines in picking up diverse objects in different configurations (heights, locations, orientations) and environments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Legged locomotion in challenging terrains using egocentric vision. In Conference on Robot Learning, pages 403–415. PMLR, 2023.
  2. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022.
  3. Deep kernels for optimizing locomotion controllers. In Conference on Robot Learning, pages 47–56. PMLR, 2017.
  4. Bayesian multi-task learning mpc for robotic mobile manipulation. IEEE Robotics and Automation Letters, 2023.
  5. Alma-articulated locomotion and manipulation for a torque-controllable robot. In 2019 International conference on robotics and automation (ICRA), pages 8477–8483. IEEE, 2019.
  6. Estimating terrain elevation maps from sparse and uncertain multi-sensor data. In 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 715–722. IEEE, 2012.
  7. Mit cheetah 3: Design and control of a robust, dynamic quadruped robot. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2245–2252. IEEE, 2018.
  8. Robust visual inertial odometry using a direct ekf-based approach. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 298–304. IEEE, 2015.
  9. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022.
  10. Learning inertial odometry for dynamic legged robot state estimation. In Conference on robot learning, pages 1575–1584. PMLR, 2022.
  11. Yale-cmu-berkeley dataset for robotic manipulation research. The International Journal of Robotics Research, 36(3):261–268, 2017.
  12. Legs as manipulator: Pushing quadrupedal agility beyond locomotion. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023a.
  13. Extreme parkour with legged robots. arXiv preprint arXiv:2309.14341, 2023b.
  14. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2021.
  15. Learning vision-based bipedal locomotion for challenging terrain. arXiv preprint arXiv:2309.14594, 2023.
  16. Adversarial motion priors make good substitutes for complex reward functions. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 25–32. IEEE, 2022.
  17. Robot-centric elevation mapping with uncertainty estimates. In Mobile Service Robotics, pages 433–440. World Scientific, 2014.
  18. Probabilistic terrain mapping for mobile robots with uncertain localization. IEEE Robotics and Automation Letters, 3(4):3019–3026, 2018.
  19. Optimization based full body control for the atlas robot. In 2014 IEEE-RAS International Conference on Humanoid Robots, pages 120–127. IEEE, 2014.
  20. Coupling vision and proprioception for navigation of legged robots. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17273–17283, 2022.
  21. Deep whole-body control: learning a unified policy for manipulation and locomotion. In Conference on Robot Learning, pages 138–149. PMLR, 2023.
  22. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02117, 2024.
  23. Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5092–5098. IEEE, 2023.
  24. Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning, 2020.
  25. Practice makes perfect: An optimization-based approach to controlling agile motions for a quadruped robot. IEEE Robotics & Automation Magazine, 23(1):34–43, 2016.
  26. Multi-skill mobile manipulation for object rearrangement. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Z3IClM_bzvP.
  27. Robot learning in homes: Improving generalization and reducing dataset bias. Advances in neural information processing systems, 31, 2018.
  28. Mastering diverse domains through world models, 2023.
  29. Td-mpc2: Scalable, robust world models for continuous control, 2023.
  30. Reinforcement learning with automated auxiliary loss search. Advances in Neural Information Processing Systems, 35:1820–1834, 2022.
  31. Anymal-a highly mobile and dynamic quadrupedal robot. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 38–44. IEEE, 2016.
  32. Learning whole-body manipulation for quadrupedal robot. IEEE Robotics and Automation Letters, 9(1):699–706, 2023.
  33. Human motion control of quadrupedal robots using deep reinforcement learning. In Proceedings of Robotics: Science and Systems, New York, USA, June 2022.
  34. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
  35. Real-time localization and elevation mapping within urban search and rescue scenarios. Journal of Field Robotics, 24(8-9):723–745, 2007.
  36. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021.
  37. Adapting rapid motor adaptation for bipedal robots. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1161–1168. IEEE, 2022.
  38. Cascaded compositional residual learning for complex interactive behaviors. IEEE Robotics and Automation Letters, 8(8):4601–4608, 2023a. doi: 10.1109/LRA.2023.3286171.
  39. Words into action: Learning diverse humanoid robot behaviors using language guided iterative motion refinement, 2023b.
  40. High-resolution terrain map from multiple sensor data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):278–292, 1992.
  41. Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47):eabc5986, 2020.
  42. Robotic table wiping via reinforcement learning and whole-body trajectory optimization, 2022.
  43. Learning agile skills via adversarial imitation of rough partial demonstrations. In Conference on Robot Learning, pages 342–352. PMLR, 2023.
  44. Reinforcement learning for robust parameterized locomotion control of bipedal robots. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 2811–2817. IEEE, 2021.
  45. Energy-based imitation learning. arXiv preprint arXiv:2004.09395, 2020.
  46. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021.
  47. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning, pages 22–31. PMLR, 2023.
  48. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62):eabk2822, 2022.
  49. Dynamic walk of a biped. The International Journal of Robotics Research, 3(2):60–74, 1984.
  50. Continuous jumping for legged robots on stepping stones via trajectory optimization and model predictive control. In 2022 IEEE 61st Conference on Decision and Control (CDC), pages 93–99. IEEE, 2022.
  51. Dynamic walking on randomly-varying discrete terrain with one-step preview. In Robotics: Science and Systems, volume 2, pages 384–99, 2017.
  52. Optimized jumping on the mit cheetah 3 robot. In 2019 International Conference on Robotics and Automation (ICRA), pages 7448–7454. IEEE, 2019.
  53. Learning agile robotic locomotion skills by imitating animals. arXiv preprint arXiv:2004.00784, 2020.
  54. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
  55. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011.
  56. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pages 91–100. PMLR, 2022.
  57. On bringing robots home. arXiv preprint arXiv:2311.16098, 2023.
  58. A compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on mabel. The International Journal of Robotics Research, 30(9):1170–1193, 2011.
  59. Combined task and motion planning through an extensible planner-independent interface layer. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 639–646, 2014. doi: 10.1109/ICRA.2014.6906922.
  60. Fully autonomous real-world reinforcement learning with applications to mobile manipulation. In Conference on Robot Learning, pages 308–319. PMLR, 2022.
  61. Amp in the wild: Learning robust, agile, natural legged locomotion skills. arXiv preprint arXiv:2304.10888, 2023.
  62. Robust legged robot state estimation using factor graph optimization. IEEE Robotics and Automation Letters, 4(4):4507–4514, 2019.
  63. Vilens: Visual, inertial, lidar, and leg odometry for all-terrain legged robots. IEEE Transactions on Robotics, 39(1):309–326, 2022.
  64. Error-aware imitation learning from teleoperation data for mobile manipulation. In Conference on Robot Learning, pages 1367–1378. PMLR, 2022.
  65. Relmogen: Leveraging motion generation in reinforcement learning for mobile manipulation, 2021.
  66. Drm: Mastering visual reinforcement learning through dormant ratio minimization, 2023.
  67. Generalized animal imitator: Agile locomotion with versatile motion prior. arXiv preprint arXiv:2310.01408, 2023a.
  68. Harmonic mobile manipulation. arXiv preprint arXiv:2312.06639, 2023b.
  69. Neural volumetric memory for visual locomotion control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1430–1440, 2023c.
  70. State estimation for legged robots using contact-centric leg odometry. arXiv preprint arXiv:1911.05176, 2019.
  71. Moma-force: Visual-force imitation for real-world mobile manipulation. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6847–6852. IEEE, 2023d.
  72. Associating objects with transformers for video object segmentation. Advances in Neural Information Processing Systems, 34:2491–2502, 2021.
  73. Mastering visual continuous control: Improved data-augmented reinforcement learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=_SJ-_yyes8.
  74. Asc: Adaptive skill coordination for robotic mobile manipulation, 2023a.
  75. Adaptive skill coordination for robotic mobile manipulation. arXiv preprint arXiv:2304.00410, 2023b.
  76. Visual-locomotion: Learning to walk on complex terrains with vision. In 5th Annual Conference on Robot Learning, 2021.
  77. Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion. arXiv preprint arXiv:2309.15459, 2023.
  78. Robot parkour learning. arXiv preprint arXiv:2309.05665, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Minghuan Liu (29 papers)
  2. Zixuan Chen (50 papers)
  3. Xuxin Cheng (42 papers)
  4. Yandong Ji (8 papers)
  5. Ruihan Yang (43 papers)
  6. Xiaolong Wang (243 papers)
  7. Ri-Zhao Qiu (9 papers)
Citations (24)
View on Semantic Scholar

Summary

Visual Whole-Body Control for Legged Loco-Manipulation: Expanding the Horizons of Mobile Robots

Introduction to Visual Whole-Body Control (VBC)

In the pursuit of enhancing robotic mobility and manipulation capabilities beyond conventional limitations, Visual Whole-Body Control (VBC) emerges as a comprehensive framework developed by researchers from UC San Diego, Shanghai Jiao Tong University, and Fudan University. VBC is engineered to address the intricacies of mobile manipulation using legged robots equipped with arms, enabling these machines to handle objects across diverse environments autonomously, guided solely by visual inputs.

The Framework

VBC introduces a two-tiered architecture consisting of a low-level policy that manages all degrees of freedom for locomotion and manipulation and a high-level policy that proposes end-effector positions based on segmented depth images. Notably, this design allows for the seamless execution of actions in various settings without the need for task-specific fine-tuning, showcasing the framework's adaptability and efficiency.

Low-Level Policy

The low-level policy within VBC is meticulously engineered to track end-effector poses and robot body velocities, enabling the robot to accomplish whole-body behaviors across different terrains. This policy utilizes reinforcement learning (RL) techniques to adapt to given goals, thereby allowing the robot to interact with its environment robustly.

  • Command and Observation: The policy receives specific commands, including end-effector position, orientation, and desired velocities. Observations fed into the policy encapsulate the robot's proprioceptive states and are enriched with environment extrinsics, fostering a dynamic response mechanism.
  • Adaptation and Control: A significant innovation in the low-level policy is the incorporation of Regularized Online Adaptation (ROA), which facilitates sim-to-real transfer by accounting for environmental discrepancies such as terrain type and robot mass variation.

High-Level Task-Planning Policy

The high-level policy operates on visual observations to navigate and manipulate objects with precision. Given the challenges associated with high-frequency control through direct RL from raw visual data, the approach instead distills guidance from a privileged teacher policy, which possesses access to object shape and pose details, into a visuomotor student policy.

  • Privileged Teacher Policy: The privileged teacher policy, trained via RL, leverages object shape features to refine the manipulation strategy, demonstrating the importance of detailed object understanding in achieving successful interaction.
  • Visual-Motor Coordination: The visuomotor student policy, distilled from the teacher policy, derives commands from segmented depth images, illustrating the critical role of visual data in adapting to real-world conditions.

Real-World Deployment and Implications

VBC's practical application is manifest in its deployment on a Unitree B1 quadruped robot, equipped with a Unitree Z1 robotic arm. The system demonstrates remarkable proficiency in picking up a variety of objects across different heights and surfaces, heralding a new era of adaptability and autonomy in robotic systems.

  • Sim-To-Real Transfer: Detailed measures, including domain randomization and depth image processing, ensure the seamless transition of the VBC framework from simulated environments to real-world scenarios, underscoring the effectiveness of the training methodologies.
  • System Design and Challenges: Adjustments to the hardware and software architecture accommodate various operational frequencies, highlighting the consideration for systemic harmony in ensuring efficient task execution.

Conclusion and Future Directions

VBC presents a holistic strategy for enhancing the capabilities of legged robots in navigating and manipulating objects in complex environments, driven by visual inputs. The framework's success in sim-to-real transfer, combined with its hierarchical design, sets a strong foundation for further research focused on improving hardware components and simplification processes to overcome current limitations. Future developments may concentrate on refining visual input quality, enhancing gripper functionality, and streamlining system operations, promising an expansive trajectory for the evolution of mobile robotic manipulation.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. Visual Whole-Body Loco-Manipulation