Human-to-Humanoid: Universal Control & Design

• Human-to-Humanoid (H2H) frameworks are systems that transfer and optimize whole-body human motion onto customizable humanoids using universal control policies.
• They employ a two-stage process where a universal controller is trained on large-scale motion capture data and then refined via motion-dependent design optimization.
• Empirical results demonstrate enhanced motion fidelity and emergent morphological adaptations, with significant improvements in success rates and tracking errors.

Human-to-Humanoid (H2H) frameworks enable the transfer, synthesis, and optimization of whole-body human motion and embodiment onto humanoid robots or virtual agents. These frameworks address two core challenges: (1) learning universal control policies that generalize across body morphologies and motion types, and (2) optimizing physical attributes of humanoid bodies to maximize motion imitation fidelity. H2H systems underpin a range of applications in robotics, computer graphics, and automatic character design, and are foundational to large-scale, data-driven humanoid skill learning.

1. Architectural Overview and Pipeline Structure

The canonical H2H framework, as formalized by "From Universal Humanoid Control to Automatic Physically Valid Character Creation" (Luo et al., 2022), consists of two central stages:

1. Universal Humanoid Controller (UHC) Training
   • Input: Large-scale human motion-capture data (AMASS), including motion clips $\widehat Q$ and shape parameters $\beta$.
   • Output: A single PPO-trained policy $\pi^C$ capable of controlling diverse SMPL-derived humanoids (varied $\beta$, design parameters $D$) to imitate arbitrary motion sequences.
2. Motion-Dependent Design & Control Optimization
   • Input: Target human motion sequence(s) $\widehat Q^*$.
   • Output: Optimized body‐design parameters $D^*$ (e.g., limb lengths, masses, joint limits, SMPL shape $\beta$), identified via a design policy $\pi^D$ that samples $D$ at each episode, rolls out $\pi^C$, accumulates rewards, and updates $\pi^D$ via PPO.
   • Objective:

   $$D^* = \arg\max_{D} \; E_{D\sim\pi^D, a\sim\pi^C} \bigg[\sum_{t} \gamma^{t-1} r_t \bigg]$$

This modular architecture enables both universal human motion imitation and automatic, motion-conditioned humanoid body creation.

2. Universal Humanoid Controller: MDP Formulation and Policy Design

The Universal Humanoid Controller treats motion imitation as an MDP $$M = (\mathcal{S}, \mathcal{A}, \mathcal{T}, R, \gamma)$$:

• State $s_t$:

$s_t = (q_t, \dot{q}_t)$, where $q_t$ includes joint positions $q_t^p \in \mathbb{R}^{J \times 3}$ and root orientation $q_t^r \in \mathbb{R}^{J \times 3}$, both in the world frame. $\dot{q}_t$ gives linear and angular velocities.

• Action $a_t$:

$a_t = (p_t^d, k_t, e_t)$ with target joint angles $p_t^d \in \mathbb{R}^K$, meta-PD control gains $k_t = (k_t^p, k_t^d)$, and residual contact forces $e_t$ for the foot geoms.

• Dynamics $\mathcal{T}$:

Simulated in MuJoCo at $\Delta t = 1/60$ s; contact forces are resolved per MuJoCo’s built-in penalty solver.

• Reward $r_t$:

$r_t = w_p r_p + w_e r_e + w_v r_v + w_{vf} r_{vf}$

where $r_p$, $r_e$, $r_v$, and $r_{vf}$ are exponentially weighted tracking errors on root orientation, joint positions, velocities, and contact forces, respectively.

• Policy $\pi^C_\theta(a_t | s_t, \widehat{q}_t, D)$:

A normal distribution centered at $\mu_\theta(\phi(s_t, \widehat{q}_t, D))$, where the feature extractor $\phi$ computes root-relative tracking errors. Design parameters $D$ are inputs to $\phi$, enabling morphology-conditioned control.

• Torque Generation:

$$\tau^i = k_t^{p,i} \odot (p^d_{t,i} - p_{t,i}) - k_t^{d,i} \odot \dot{p}_{t,i}$$

• Training:

PPO maximizes $E[\sum_t \gamma^{t-1} r_t]$, with hard-negative mining to sample challenging motion clips in proportion to their historical episode success. Early termination is triggered by excessive tracking errors.

3. Motion-Dependent Body Design Optimization

The second H2H stage optimizes humanoid design parameters for specialized motion reproduction:

• Parameterization $D$:

$D = (\beta, w, h, b, f, m, g)$ denote SMPL body-shape, mass/height scalars, per-joint friction, damping, bone sizes/densities, and actuator gear ratios.

• Design Policy $\pi^D_\phi(D|s_0, \widehat{q}_0)$:

Samples a design $D$ at $t=0$; $\pi^C$ is rolled out for $t=1...T$. Rewards accumulate over the episode, and the value function is conditioned on $D$:

$$V(s_t, D) = E \bigg[ \sum_{k=t}^{T} \gamma^{k-t} r_k \mid s_t, D, \pi^C \bigg]$$

• Algorithmic Protocol:

1. At each episode, $D_0 \sim \pi^D_\phi$ is sampled.
2. Simulator initialized with $D_0$.
3. For $t=1$ to $T$, actions $a_t$ are produced via the fixed $\pi^C$; simulator computes $r_t$.
4. Rollouts are accumulated; $\pi^D_\phi$ is updated with PPO.

4. Physics Simulation and Stability Metrics

• Simulation Details:

  • MuJoCo with geometries derived from SMPL skinning weights; convex hull per bone.
  • Contact: Only residual foot forces injected when foot geoms are in ground contact.
  • Stability: “Success rate” is episode survival without root translation error exceeding threshold or character fall (head/root crash before $T$).
• Metric Definitions:

Episodes are classified as “fail” under two conditions: root-to-reference translation error exceeds threshold or robot falls before allocated frames.

5. Quantitative Results and Emergent Design Patterns

• Universal Controller Performance (AMASS splits):

| Setting | Train Succ. (%) | Test Succ. (%) | $E_{mpjpe-g}$ (mm) Train | $E_{mpjpe-g}$ (mm) Test | |--------------------|-----------------|----------------|--------------------------|------------------------| | No-RFC | 89.7 | 65.5 | 50.7 | 156 | | RFC (root-only) | 94.7 | 80.7 | | | | RFC (foot, Ours) | 95.6 | 91.4 | 36.5 | 60.1 | | RFC (Oracle) | 100 | ~100 | | |

• Specialized Design Discovery (Single Sequence):

For sequences like Cartwheel-1: Success jumps from 0% to 100%; $E_{mpjpe}$ drops 160.9 mm to 37.4 mm; $E_{mpjpe-g}$ drops 284.8 mm to 66.2 mm. Similar improvements for Parkour-1, Belly-Dance-1, Karate-1.

• Category-Level Design:

For Dance-200: Success up from 57% to 72%, $E_{mpjpe}$ down from 84.1 mm to 58.0 mm.

• Robustness to Unseen Motions:

Specialized bodies retain high success on full AMASS test (≈90% success, $E_{mpjpe-g}$ ≈55 mm), evidencing specialization without loss of generality.

• Emergent Morphological Adaptations:
  • Parkour: Wider hips/thighs, stronger gears, lower center.
  • Cartwheeler: Enlarged hands/wrists.
  • Karate: Lower center of mass, robust legs.
  • Belly-dancer: Slender compliant limbs, high foot compliance.

6. Implementation Protocols and Extensibility

• Simulator-Kinematic Conversion:

Automatic conversion of SMPL parameterization to MuJoCo convex hulls.

• Network Structure and PPO Setup:

All hyperparameters and training schedules are supplied in the original paper, including the detailed configuration for PPO, feature extraction, entropy bonuses, and hard-negative mining.

• Code Blueprint (Design & Control Loop, Alg. 1):

# Pseudocode summarization from the original paper Input: pretrained π^C_θ, reference motions \widehat Q Loop until π^D converges: M ← empty replay while M not full: sample batch of target clips \widehat Q¹…\widehat Q^B for each clip: set s₀ from reference \widehat q₀ t=0: sample D₀∼π^D_ϕ(D|s₀,\widehat q₀), r₀=0, store (s₀,D₀,r₀) initialize sim with design D₀ for t=1…T: a_t∼π^C_θ(a|s_t,\widehat q_t,D₀) s_{t+1}←T(s_t,a_t) compute r_t via tracking reward store transition update π^D_ϕ by PPO on collected M

7. Relation to Broader Human-to-Humanoid Research

The H2H paradigm defined above offers:

• Universality of control (single policy generalizing to broad morphology and motion classes).
• Automated, data-driven humanoid body design conditioned on arbitrary motion criteria.
• Physical plausibility via a joint design–control optimization in simulation.

Empirical results demonstrate high-fidelity motion imitation, emergent adaptive morphologies, strong generalization to unseen tasks, and resilience to domain shifts. This approach serves as the backbone for advanced character creation in graphics, simulation, and rapidly deployable humanoid skill learning (Luo et al., 2022).