ExBody2: Advanced Expressive Humanoid Whole-Body Control (2412.13196v2)

Abstract: This paper tackles the challenge of enabling real-world humanoid robots to perform expressive and dynamic whole-body motions while maintaining overall stability and robustness. We propose Advanced Expressive Whole-Body Control (Exbody2), a method for producing whole-body tracking controllers that are trained on both human motion capture and simulated data and then transferred to the real world. We introduce a technique for decoupling the velocity tracking of the entire body from tracking body landmarks. We use a teacher policy to produce intermediate data that better conforms to the robot's kinematics and to automatically filter away infeasible whole-body motions. This two-step approach enabled us to produce a student policy that can be deployed on the robot that can walk, crouch, and dance. We also provide insight into the trade-off between versatility and the tracking performance on specific motions. We observed significant improvement of tracking performance after fine-tuning on a small amount of data, at the expense of the others.

• The paper introduces a novel RL-based framework that employs a two-stage teacher-student model for robust humanoid whole-body control.
• It leverages diverse motion datasets and converts global keypoint tracking to local coordinates to enhance stability and accuracy.
• The research integrates a CVAE for long-range motion synthesis, achieving expressive and continuous humanoid movements in real robots.

Advanced Expressive Humanoid Whole-Body Control

The paper entitled "Advanced Expressive Humanoid Whole-Body Control" introduces a novel framework aimed at enhancing the expressiveness and stability of humanoid robots while performing complex whole-body movements. This work contributes to the field of robotics by proposing an innovative method that seeks to replicate human-like movements on humanoid robots by leveraging reinforcement learning (RL) techniques and employing a robust Sim2Real pipeline.

Overview of the Methodology

The framework developed in this research addresses the challenge of humanoid robot control by introducing Advanced Expressive Whole-Body Control (ExBody2), a generalized whole-body tracking system. The approach is built upon the principles of tracking diverse human motions captured in large-scale datasets such as AMASS, and transferring this expertise to real-world humanoid robots using a simulation-to-reality (Sim2Real) transfer. Here are the prominent features of the proposed method:

1. Dataset Curation: The authors emphasize the importance of careful dataset selection, focusing on the diversity of upper body movements for stability, while ensuring the feasibility of lower body motions for effective training. They identify that maintaining a balance between challenging and feasible actions within the dataset is crucial for optimizing training effectiveness.
2. Two-Stage Training: The methodology employs a two-stage teacher-student framework. Initially, a teacher policy is developed using RL techniques in a simulated environment where it can access privileged information, essentially serving as an oracle for learning imitation skills. Subsequently, a deployable student policy is trained via DAgger-style distillation from the teacher, utilizing historical observation sequences in place of privileged data. This setup fosters the development of robust and transferable control policies.
3. Local Keypoint Tracking: Traditional approaches often rely on global keypoint tracking, which can lead to tracking failures in non-stationary environments. This paper innovatively converts keypoints into the local frame and decouples keypoint tracking from velocity control, thus enhancing robustness and stability in whole-body motion tracking.
4. Long-Range Motion Synthesis: To support continuous and uninterrupted motion execution, the researchers employ a Conditional Variational Autoencoder (CVAE) that generates future motion sequences. The CVAE model predicts motions using a learned history context, allowing the robot to perform extended sequences of expressive movements during real-world deployment.

Experimental Evaluation and Results

The proposed framework was comprehensively evaluated across two humanoid robot platforms, the Unitree G1 and H1. In these experiments, ExBody2 outperformed existing methods, demonstrating lower tracking errors across various metrics, including full-body tracking accuracy, upper and lower body keypoints, and velocity tracking. These results underscore the efficacy of the dataset curation and the incorporated training strategies. The experiments also illustrate that a moderately diverse dataset yields superior generalization compared to simplistic or overly noisy datasets.

Contributions and Implications

The contributions of this work lie in its comprehensive approach integrating dataset curation, dual-stage training, decoupled tracking, and motion synthesis. By addressing both upper and lower body movement challenges, this research paves the way for more adaptive and expressive humanoid robots capable of performing human-like behaviors in dynamic environments. The implications extend to various domains such as assistive robotics, entertainment, and human-robot interaction, where high-fidelity motion replication is crucial.

Future Directions

The research opens avenues for future work in several dimensions: automating the selection process for high-quality datasets, further optimizing Sim2Real transfer techniques, and exploring the integration of other sensory inputs to enhance robustness and adaptability. Additionally, leveraging more advanced reinforcement learning algorithms could further improve policy refinement and deployment across diverse robotic platforms.

This paper reflects an advancement in tackling the intricate problem of humanoid robot control, offering valuable insights and methodologies that could be fundamental in the trajectory of robotics research and development.