ExBody2: Advanced Expressive Whole-Body Control

• The paper presents a novel sim-to-real control framework that integrates teacher-student policy learning and decoupled reward shaping to achieve expressive humanoid motion imitation.
• The approach employs a transformer-based CVAE for continual motion synthesis and utilizes careful dataset curation to balance expressivity and precise trajectory tracking.
• The system demonstrates superior performance on Unitree platforms, achieving lower velocity and joint error metrics compared to competing baselines.

Advanced Expressive Whole-Body Control (ExBody2) defines a Sim-to-Real control and learning framework for robust, expressive humanoid whole-body motion imitation, capable of tracking arbitrary human movements on bipedal robots. The ExBody2 approach synthesizes carefully curated motion corpora, a teacher-student policy architecture with privileged RL and DAgger-style distillation, and decoupled reward shaping to enable a single policy to drive a real robot through varied expressive behaviors—including walking, crouching, and dance—while maintaining stability and generalization. Distinctively, ExBody2 addresses the trade-off between versatility and trajectory fidelity via dataset design and evaluation, and demonstrates stable deployment on commercial hardware without per-task tuning (Ji et al., 2024).

1. System Architecture and Learning Pipeline

ExBody2 is structured around four primary modules: dataset curation, teacher-student policy learning, continual motion synthesis, and real-world deployment.

1. Dataset Curation targets a "just-right" corpus—typically 250–500 filtered AMASS motions—favoring lower-body feasibility and upper-body expressiveness. Motions violating robot joint constraints or physical limits are excluded or modified, ensuring the teacher is only tasked with learnable trajectories.
2. Teacher-Student Policy Learning follows a two-stage approach:
   • The teacher policy $\hat\pi$ is trained via PPO in IsaacGym with access to privileged features: ground-truth root velocity, link positions, physical parameters ($s_t = \{p_t,o_t,g_t\}$).
   • The student policy $\pi$ is distilled through DAgger-style imitation, replacing privileged state with a history (H=25) of proprioceptive observations, and with direct supervision via $\ell_t = \|a_t - \hat a_t\|^2$.
3. Continual Motion Synthesis leverages a transformer-based conditional variational autoencoder (CVAE), ingesting past M frames and autoregressively predicting H future states for long-horizon planning.
4. Real-World Deployment runs the student policy at 50 Hz on Jetson Orin NX, with 500 Hz PD set-points executed on Unitree G1/H1 via LCM. Regularizations for stability are incorporated at runtime.

2. Decoupling Velocity and Landmark Tracking

ExBody2 introduces explicit separation between local body-landmark ("expression goal") and global velocity ("movement goal") tracking. Rather than global keypoint-based imitation (typical in previous methods), ExBody2 defines:

$$G^e_t = w_q \exp\left(-\alpha_q \|\mathbf{q}^{\mathrm{ref}}_t - \mathbf{q}_t\|\right) + w_k \exp\left(-\alpha_p \|\mathbf{p}^{\mathrm{ref}}_t - \mathbf{p}_t\|\right)$$

$$G^m_t = w_v \exp\left(-\alpha_v \|\mathbf{v}^{\mathrm{ref}}_t - \mathbf{v}_t\|\right) + w_d \exp\left(-\alpha_d \cos(\mathbf{v}^{\mathrm{ref}}_t, \mathbf{v}_t)\right) + w_r \exp\left(-\alpha_r \|\boldsymbol\Omega^{\phi\theta,\mathrm{ref}}_t - \boldsymbol\Omega^{\phi\theta}_t\|\right) + w_y \exp\left(-\alpha_y |\Delta y_t|\right)$$

The combined reward aligns the robot's local joint and keypoint configuration while separately targeting root velocity and orientation. This design prevents drift and cumulative error prevalent in pure global keypoint tracking and enables temporary global drift, with periodic keypoint-frame resets as a form of data augmentation.

3. Teacher Policy Optimization and Regularization

The teacher $\hat\pi$ is trained under a privileged MDP, maximizing cumulative reward that combines $G^e$, $G^m$, and several stability-oriented regularizers:

• Core Simulation Rewards: $G^e$, $s_t = \{p_t,o_t,g_t\}$0
• Physical Regularization: Penalties on violation of joint limits, excessive torque and power, high accelerations, and foot slip ($s_t = \{p_t,o_t,g_t\}$1, $s_t = \{p_t,o_t,g_t\}$2, $s_t = \{p_t,o_t,g_t\}$3, etc.).
• Feasibility Filtering: The dataset curation stage ensures reference motions are within the physical, kinematic, and dynamic admissibility of the target hardware, such that the teacher never attempts unattainable trajectories.

This design produces intermediate label policies that are finely tuned to the robot's true embodiment, facilitating distillation.

4. Distillation and Student Policy Deployment

Student training replaces privileged feedback with observable proprioception. The three-layer MLP ([1024, 1024, 512]) accepts concatenated proprioceptive histories and the same tracking goal. Distillation follows a DAgger loop: execute the current $s_t = \{p_t,o_t,g_t\}$4 in simulation, retrieve teacher actions $s_t = \{p_t,o_t,g_t\}$5 at student-visited states, and iteratively minimize the supervised loss.

Training hyperparameters include Adam optimizer with learning rate 1e-4, batch size 4096, $s_t = \{p_t,o_t,g_t\}$6, PPO clip 0.2, entropy coefficient 0.005, and five epochs per iteration. Empirically, $s_t = \{p_t,o_t,g_t\}$7 frames of history is optimal; further increases yield diminishing improvement. Policy deployment involves 50 Hz inference (student policy), with 500 Hz PD-level actuation. End-to-end delay is controlled in the 18–30 ms range; safety and stability are enhanced by regular foot contact and stumble penalization, as well as frequent global keypoint-frame resets.

5. Quantitative and Qualitative Evaluation

Experiments are conducted on Unitree G1 and H1 platforms, evaluating both trajectory tracking and motion expressivity. Main metrics include RMSE of root velocity ($s_t = \{p_t,o_t,g_t\}$8), mean per-keypoint position error (MPKPE, $s_t = \{p_t,o_t,g_t\}$9), and mean per-joint position error (MPJPE, $\pi$0; error decomposed for upper/lower body).

Method	$\pi$1 (G1)	$\pi$2 (G1, m)
ExBody2	0.1891	0.0854
OmniH2O*	0.2429	0.0959
ExBody†	0.2950	0.0914
ExBody	0.3083	0.1134

ExBody2 exhibits lower error across all axes. Similar gains are observed on H1 (e.g., $\pi$3 of 0.3016 vs. 0.4254/0.5964/0.6377 for competing baselines). Qualitatively, ExBody2 tracks diverse and high-amplitude human motions (walking, dancing, crouching, and complex gestures) stably for extended durations (e.g., a 43 s Cha-Cha sequence).

Ablation reveals that removing DAgger or global resets deteriorates error by 30–50%. Moderate motion-dataset size and diversity ($\pi$4) optimizes generalization and tracking; too small ($\pi$5) fails out-of-distribution, while large noisy sets ($\pi$6) dilute learning.

6. Versatility–Fidelity Trade-Off and Ablation Analysis

Expressivity and generalization fundamentally trade off against exact tracking fidelity. An over-constrained dataset ($\pi$7) achieves strong in-distribution tracking but generalizes poorly (e.g., $\pi$8 m on $\pi$9-test). Excessive data ($\ell_t = \|a_t - \hat a_t\|^2$0) introduces noise due to infeasible or ambiguous reference motions, leading the policy to allocate capacity inefficiently. The moderate ($\ell_t = \|a_t - \hat a_t\|^2$1) set achieves lowest aggregate error on both train/test splits.

Fine-tuning on a specific subset further improves target-motion fidelity but at the expense of versatility, indicating a policy capacity–diversity bottleneck typical in deep policy learning frameworks.

7. Limitations and Research Directions

Open challenges include:

• Automated Dataset Selection: Currently, corpus curation balances feasibility and expressiveness manually. Automating the identification of robot-achievable human motions remains unsolved.
• Extreme Dynamics: Motions with high-impact dynamics, e.g., jumping or somersaults, remain intractable without explicit model-based recovery mechanisms.
• Perception Integration: ExBody2 presumes perfect motion-capture; real-world robustness to sensor noise or onboard vision remains future work.

Key innovations of ExBody2 include modular motion corpus design, decoupled reward structuring, privileged-to-unprivileged policy distillation, and CVAE-based long-horizon synthesis. The resulting system is the first to support arbitrary expressive full-body human imitation on real humanoid platforms without per-task adaptation (Ji et al., 2024).