ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills (2502.01143v3)

Abstract: Humanoid robots hold the potential for unparalleled versatility in performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a significant challenge due to the dynamics mismatch between simulation and the real world. Existing approaches, such as system identification (SysID) and domain randomization (DR) methods, often rely on labor-intensive parameter tuning or result in overly conservative policies that sacrifice agility. In this paper, we present ASAP (Aligning Simulation and Real-World Physics), a two-stage framework designed to tackle the dynamics mismatch and enable agile humanoid whole-body skills. In the first stage, we pre-train motion tracking policies in simulation using retargeted human motion data. In the second stage, we deploy the policies in the real world and collect real-world data to train a delta (residual) action model that compensates for the dynamics mismatch. Then, ASAP fine-tunes pre-trained policies with the delta action model integrated into the simulator to align effectively with real-world dynamics. We evaluate ASAP across three transfer scenarios: IsaacGym to IsaacSim, IsaacGym to Genesis, and IsaacGym to the real-world Unitree G1 humanoid robot. Our approach significantly improves agility and whole-body coordination across various dynamic motions, reducing tracking error compared to SysID, DR, and delta dynamics learning baselines. ASAP enables highly agile motions that were previously difficult to achieve, demonstrating the potential of delta action learning in bridging simulation and real-world dynamics. These results suggest a promising sim-to-real direction for developing more expressive and agile humanoids.

• The paper introduces ASAP, a two-phase framework that aligns simulation and real-world physics for agile humanoid robots by using a delta action model to learn corrective actions.
• Evaluation shows ASAP outperforms existing methods like SysID and DR, significantly reducing motion tracking errors and enabling agile maneuvers on a real humanoid robot.
• ASAP's ability to compensate dynamics mismatch facilitates practical humanoid applications by enabling greater dexterity and expressiveness without extensive tuning.

Overview of the ASAP Framework: Aligning Simulation and Real-World Physics

The paper presents a comprehensive framework named ASAP, which addresses the persistent challenge of aligning simulation dynamics with real-world physics to enable humanoid robots to perform agile, whole-body tasks. This work is situated within the broader context of robotics, where achieving precise control over humanoid robots is quintessential for replicating human-like agility. Traditional methods like System Identification (SysID) and Domain Randomization (DR) have struggled with either time-consuming parameter tuning or producing overly conservative control policies. ASAP seeks to overcome these limitations through a novel two-phase approach.

Methodology and Key Innovations

The ASAP framework unfolds in two distinct stages:

1. Simulation Pre-training Phase:
   • Initial training of motion tracking policies is conducted using retargeted data from human motion videos. By reconstructing human motion via TRAM and translating these into robust robotic movements within the simulator, the framework secures a solid starting point for policy development.
   • The training integrates a phase-conditioned reinforcement learning approach inspired by the DeepMimic methodology, emphasizing motion tracking fidelity.
2. Real-World Deployment and Fine-Tuning:
   • Once trained, these policies are initially deployed in the real-world environment where discrepancies in robotic motion reveal the dynamics mismatch.
   • The core innovation lies in the deployment of a delta action model, which compensates for these discrepancies by learning corrective actions. The result is a fine-tuned policy that bridges the gap between simulated and real-world dynamics, enhancing motion precision.

The ASAP framework's design allows for testing across multiple simulation-to-simulation and simulation-to-real scenarios, demonstrating improvements in agility and whole-body coordination for dynamic humanoid operations.

Evaluation and Results

ASAP has been rigorously evaluated against existing techniques using metrics such as global mean joint position error, velocity, and acceleration consistency. Strong numerical results show ASAP's superiority in reducing motion tracking errors over SysID, DR, and baseline delta dynamic models. The adaptability across different test environments, including IsaacGym, IsaacSim, and the Genesis simulator, substantiates the versatility and efficiency of the delta action learning approach.

Simulation tests pave the way for more controlled, repeatable experimental conditions, while real-world deployments on the Unitree G1 humanoid robot corroborate the practicality and robustness of ASAP in field applications. These tests reveal a marked enhancement in performing agile maneuvers—such as jumps and complex athletic celebrations—a testament to ASAP's potential in advancing humanoid robotics.

Implications and Future Directions

ASAP's contribution to sim-to-real transfer in robotics lies in its ability to actively compensate dynamics mismatches without the exhaustive tuning traditional methods often require. This benefits practical humanoid applications—envisioning humanoid robots in spaces like manufacturing, caregiving, and complex human-robot interaction environments is becoming increasingly feasible as these robots achieve greater dexterity and expressiveness.

Looking forward, extending ASAP to accommodate more diverse robots and task environments could usher in significant developments. Future research could probe more into the delta action model's adaptability to varied limbs and sensors, efficient learning from diverse, fewer real-world samples, and integrating advanced sensor input for even more nuanced control policies.

ASAP marks a significant step in humanoid robotics, bridging simulation and reality to bring us closer to fully autonomous, agile humanoid applications. The framework sets the groundwork for expansive research and potential applications in real-world human-like robotics.