To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video.

Create a Video View Paper

HumanX: Teaching Humanoid Robots from Human Videos

This presentation introduces HumanX, a groundbreaking framework that enables humanoid robots to learn complex interaction skills directly from ordinary video footage of humans. By combining scalable data synthesis with unified imitation learning, HumanX achieves remarkable zero-shot generalization across 10 diverse skills—from basketball maneuvers to reactive fighting—using only single video demonstrations. The system eliminates traditional reward engineering while outperforming previous methods by over eight times in real-world deployment on the Unitree G1 humanoid robot.
Script
What if a robot could learn to catch a basketball, kick a football, or even engage in reactive combat simply by watching a single video of a human performing these tasks? HumanX makes this vision a reality, eliminating the need for expensive motion capture systems or hand-crafted reward functions.
Traditional robot learning faces a critical bottleneck: acquiring diverse interaction skills requires massive datasets and task-specific engineering.
Building on this challenge, the authors introduce a two-component solution. XGen transforms human video into diverse robot training data through physics-driven augmentation, while XMimic distills this into deployable policies that work with minimal sensing.
Let's examine how HumanX bridges the gap from pixels to physical robot skills.
The system starts by estimating human pose from video and retargeting it to humanoid morphology. Contact-aware segmentation then identifies when the human touches objects, using force-closure optimization to ensure physical plausibility. Non-contact phases leverage physics simulation to sample realistic object dynamics, creating a rich distribution of training scenarios from that single input video.
XMimic employs a clever two-stage approach. Teachers first learn with privileged information—seeing everything in simulation. Students then distill this knowledge while restricted to realistic sensing: either pure proprioception or limited motion capture data. This distillation enables direct real-world deployment without the sensor infrastructure teachers relied upon.
The results are striking. From just one demonstration video, policies generalize to object positions, trajectories, and targets never seen during training. This broad coverage emerges from XGen's augmentation strategies combined with online domain randomization during learning, preventing the common pitfall of overfitting to narrow demonstration data.
Deployed on the Unitree G1 humanoid, HumanX policies demonstrate remarkable robustness. The system exhibits emergent behaviors never explicitly programmed: autonomous recovery from dropped objects, discrimination between feints and actual attacks, and real-time adaptation to forceful disturbances. These capabilities suggest policy-level reasoning rather than simple trajectory replay.
However, the approach has important constraints. Sim-to-real success critically depends on training with realistic disturbances—omitting force perturbations or perception loss can cause catastrophic real-world failures. Current implementations focus on single-agent scenarios, leaving multi-agent collaboration and compositional task learning as open challenges.
HumanX fundamentally transforms how humanoid robots acquire interaction skills, proving that a single video can encode the foundation for generalizable, adaptive behavior. Visit EmergentMind.com to explore the full paper and discover how video-driven learning is reshaping the future of autonomous humanoid robots.