Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Expressive Whole-Body Control for Humanoid Robots (2402.16796v2)

Published 26 Feb 2024 in cs.RO and cs.LG

Abstract: Can we enable humanoid robots to generate rich, diverse, and expressive motions in the real world? We propose to learn a whole-body control policy on a human-sized robot to mimic human motions as realistic as possible. To train such a policy, we leverage the large-scale human motion capture data from the graphics community in a Reinforcement Learning framework. However, directly performing imitation learning with the motion capture dataset would not work on the real humanoid robot, given the large gap in degrees of freedom and physical capabilities. Our method Expressive Whole-Body Control (Exbody) tackles this problem by encouraging the upper humanoid body to imitate a reference motion, while relaxing the imitation constraint on its two legs and only requiring them to follow a given velocity robustly. With training in simulation and Sim2Real transfer, our policy can control a humanoid robot to walk in different styles, shake hands with humans, and even dance with a human in the real world. We conduct extensive studies and comparisons on diverse motions in both simulation and the real world to show the effectiveness of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Legged locomotion in challenging terrains using egocentric vision. In Conference on Robot Learning, pages 403–415. PMLR, 2023.
  2. Legs as manipulator: Pushing quadrupedal agility beyond locomotion. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023a.
  3. Extreme parkour with legged robots. arXiv preprint arXiv:2309.14341, 2023b.
  4. The mit humanoid robot: Design, motion planning, and control for acrobatic behaviors. In 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), pages 1–8. IEEE, 2021.
  5. Sim-to-real learning for humanoid box loco-manipulation. arXiv preprint arXiv:2310.03191, 2023.
  6. Whole body humanoid control from human motion descriptors. In 2008 IEEE International Conference on Robotics and Automation, pages 2677–2684. IEEE, 2008.
  7. Learning vision-based bipedal locomotion for challenging terrain. arXiv preprint arXiv:2309.14594, 2023.
  8. Adversarial motion priors make good substitutes for complex reward functions. 2022 ieee. In International Conference on Intelligent Robots and Systems (IROS), volume 2, 2022a.
  9. Adversarial motion priors make good substitutes for complex reward functions. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 25–32. IEEE, 2022b.
  10. Learning deep sensorimotor policies for vision-based autonomous drone racing, 2022.
  11. Minimizing energy consumption leads to the emergence of gaits in legged robots. Conference on Robot Learning (CoRL), 2021.
  12. Deep whole-body control: learning a unified policy for manipulation and locomotion. In Conference on Robot Learning, pages 138–149. PMLR, 2023.
  13. Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5092–5098. IEEE, 2023.
  14. Mabel, a new robotic bipedal walker and runner. In 2009 American Control Conference, pages 2030–2036. IEEE, 2009.
  15. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, June 2022.
  16. Synthesizing physical character-scene interactions. 2023. doi: 10.1145/3588432.3591525. URL https://doi.org/10.1145/3588432.3591525.
  17. The development of honda humanoid robot. In Proceedings. 1998 IEEE international conference on robotics and automation (Cat. No. 98CH36146), volume 2, pages 1321–1326. IEEE, 1998.
  18. Anymal-a highly mobile and dynamic quadrupedal robot. In IROS, 2016.
  19. Efficient multitask learning with an embodied predictive model for door opening and entry with whole-body control. Science Robotics, 7(65):eaax8177, 2022.
  20. Learning whole-body manipulation for quadrupedal robot. arXiv preprint arXiv:2308.16820, 2023.
  21. Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1479–1486. IEEE, 2022.
  22. Dribblebot: Dynamic legged manipulation in the wild. arXiv preprint arXiv:2304.01159, 2023.
  23. The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation. In Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180), volume 1, pages 239–246. IEEE, 2001.
  24. Ichiro Kato. Development of wabot 1. Biomechanism, 2:173–214, 1973.
  25. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034, 2021.
  26. Reinforcement learning for robust parameterized locomotion control of bipedal robots. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 2811–2817. IEEE, 2021.
  27. Robust and versatile bipedal jumping control through multi-task reinforcement learning. arXiv preprint arXiv:2302.09450, 2023.
  28. Physics-based character animation and human motor control. Physics of Life Reviews, 2023.
  29. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:1–248:16, October 2015.
  30. Universal humanoid motion representations for physics-based control. arXiv preprint arXiv:2310.04582, 2023a.
  31. Perpetual humanoid control for real-time simulated avatars. In International Conference on Computer Vision (ICCV), 2023b.
  32. Amass: Archive of motion capture as surface shapes. In The IEEE International Conference on Computer Vision (ICCV), Oct 2019. URL https://amass.is.tue.mpg.de.
  33. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021.
  34. Learning to jump from pixels. arXiv preprint arXiv:2110.15344, 2021.
  35. Rapid locomotion via reinforcement learning. arXiv preprint arXiv:2205.02824, 2022.
  36. Dynamic walk of a biped. IJRR, 1984.
  37. Whole-body control of humanoid robots. Humanoid Robotics: A reference, Springer, Dordrecht, 2019.
  38. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph., 37(4):143:1–143:14, July 2018. ISSN 0730-0301. doi: 10.1145/3197517.3201311. URL http://doi.acm.org/10.1145/3197517.3201311.
  39. Learning agile robotic locomotion skills by imitating animals. April 2020a.
  40. Learning agile robotic locomotion skills by imitating animals. In Robotics: Science and Systems, 07 2020b. doi: 10.15607/RSS.2020.XVI.064.
  41. Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (ToG), 40(4):1–20, 2021.
  42. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Trans. Graph., 41(4), July 2022.
  43. Real-world humanoid locomotion with reinforcement learning. arXiv:2303.03381, 2023.
  44. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pages 91–100. PMLR, 2022.
  45. Curiosity-driven learning of joint locomotion and manipulation tasks. In Proceedings of The 7th Conference on Robot Learning, volume 229, pages 2594–2610. PMLR, 2023.
  46. Deep imitation learning for humanoid loco-manipulation through human teleoperation. In 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), pages 1–8. IEEE, 2023.
  47. Blind bipedal stair traversal via sim-to-real reinforcement learning. arXiv preprint arXiv:2105.08328, 2021.
  48. A compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on mabel. IJRR, 2011.
  49. Humanmimic: Learning natural locomotion and transitions for humanoid robot via wasserstein adversarial imitation. arXiv preprint arXiv:2309.14225, 2023.
  50. Calm: Conditional adversarial latent models for directable virtual characters. In ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH ’23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701597. doi: 10.1145/3588432.3591541. URL https://doi.org/10.1145/3588432.3591541.
  51. Human motion diffusion model. arXiv preprint arXiv:2209.14916, 2022.
  52. Unicon: Universal neural controller for physics-based character motion, 2020.
  53. Amp in the wild: Learning robust, agile, natural legged locomotion skills. arXiv preprint arXiv:2304.10888, 2023.
  54. Hybrid zero dynamics of planar biped walkers. IEEE transactions on automatic control, 48(1):42–56, 2003.
  55. A scalable approach to control diverse behaviors for physically simulated characters. ACM Trans. Graph., 39(4), 2020. URL https://doi.org/10.1145/3386569.3392381.
  56. Generalized animal imitator: Agile locomotion with versatile motion prior. arXiv preprint arXiv:2310.01408, 2023a.
  57. Neural volumetric memory for visual locomotion control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1430–1440, 2023b.
  58. Simbicon: Simple biped locomotion control. ACM Transactions on Graphics, 2007.
  59. Learning physically simulated tennis skills from broadcast videos. ACM Trans. Graph., 42(4), jul 2023. ISSN 0730-0301. doi: 10.1145/3592408. URL https://doi.org/10.1145/3592408.
  60. Robot parkour learning. In Conference on Robot Learning (CoRL), 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Xuxin Cheng (42 papers)
  2. Yandong Ji (8 papers)
  3. Junming Chen (7 papers)
  4. Ruihan Yang (43 papers)
  5. Ge Yang (49 papers)
  6. Xiaolong Wang (243 papers)
Citations (40)
View on Semantic Scholar

Summary

  • The paper presents ExBody, a novel RL-based control policy trained on extensive human motion capture data to enable expressive robot movements.
  • It employs motion retargeting to adapt human motion data to robotic hardware, focusing on human-like upper body gestures.
  • Results demonstrate effective sim-to-real transfer, allowing humanoid robots to perform diverse tasks such as dancing and interactive movements.

Expressive Whole-Body Control for Humanoid Robots

Introduction

Recent advances in humanoid robotics have sought to bridge the gap between the mechanical movements traditionally associated with robots and the expressive, nuanced motions observed in living beings. In the presented work, a notable contribution towards this aim has been made through the development of an expressive whole-body control policy, referred to as ExBody, for humanoid robots. This initiative not only aims to enhance the robots' ability to perform a wider range of movements but is also designed to imbue them with a level of expressiveness that is closely akin to human motion.

Methodology

Learning from Human Motion Capture Data

The foundation of ExBody's approach lies in utilizing large-scale human motion capture data from the graphics community, integrating this with deep Reinforcement Learning (RL) in a simulated environment. This methodology enables the derivation of a control policy that can subsequently be transferred to a real humanoid robot, facilitating movements that are rich in expressivity.

Motion Retargeting

A key challenge in this process is the inherent difference in degrees of freedom (DoF) and physical capacity between the human models used in motion capture data and the actual robotic hardware. ExBody addresses this by employing motion retargeting techniques, adapting human motion data to fit the robotic model utilized in this paper, the Unitree H1 robot.

Reinforcement Learning Framework

The RL framework constitutes the core of the ExBody approach, where the humanoid robot is trained to follow command-conditioned locomotion alongside achieving specified expressive motion goals. These goals are derived from the human motion capture data, focusing on the upper body to ensure a practical level of expressivity without overly constraining the robot's movement capabilities. The policy is trained in highly randomized environments to ensure robustness and feasibility of sim-to-real transfer.

Results and Discussion

Policy Effectiveness

The paper demonstrates the effectiveness of ExBody in several key areas. The robot is capable of executing diverse movements with significant expressiveness, such as dancing and walking with varied gestures. The behavior of the robot in these instances showcases an unprecedented level of nuance in robotic movement that is both diverse and human-like.

Sim-to-Real Transfer

Crucial to the success of ExBody is its robust sim-to-real transfer capability. The control policy, once trained in simulation, exhibits high compliance and effectiveness when deployed in the real world, enabling the humanoid robot to interact dynamically with human partners and navigate various terrains with expressive motions.

Potential Implications and Future Directions

The implications of this research are significant for the future development of humanoid robots. By enabling robots to perform more expressive and varied movements, ExBody opens up new possibilities for human-robot interaction, entertainment, and assistive technologies. The approach also presents a scalable framework that could potentially be adapted for other robotic form factors.

However, the paper also recognizes the limitations inherent in the current implementation, such as the challenges related to motion retargeting and the risks associated with robot falls. Future work will need to address these challenges, further refining the control policy and exploring protective measures for the robots.

Conclusion

The presented work on ExBody represents a significant step forward in the quest for more expressive and human-like movement in humanoid robots. By leveraging extensive human motion capture data and employing a refined RL framework, the paper not only enhances the robots' movement capabilities but also enriches their ability to interact with the surrounding environment in a way that is both meaningful and engaging.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown