ActiveMimic: Egocentric Video Pretraining with Active Perception

Abstract: Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.

• The paper presents a novel method that redefines egocentric camera motion as active perception for joint pretraining of vision and manipulation.
• It introduces a unified 27D action vector encoding head and wrist movements, significantly enhancing robotic task success rates.
• Empirical evaluations show state-of-the-art performance and robust transfer capabilities, even under visual perturbations.

ActiveMimic: Egocentric Video Pretraining with Active Perception — A Technical Analysis

Introduction and Motivation

ActiveMimic (2606.06194) addresses the longstanding performance gap between vision-language-action (VLA) models pretrained on egocentric human video and those using scarce, expensive robot data. Traditional methods leveraging egocentric video have primarily supervised hand-object or object-centric actions, neglecting the active perception aspect—namely, the continuous camera (viewpoint) motion intrinsic to human manipulation. By reinterpreting egocentric camera movement as intentional viewpoint actions rather than noise, ActiveMimic offers a novel paradigm: joint pretraining of perception and manipulation without the need for specialized capture hardware.

Figure 1: ActiveMimic acquires active perception from in-the-wild egocentric human video and transfers it to real-world humanoid robots, using a unified 27D action representation for joint learning and transfer.

Methodology

Egocentric Video Processing and Action Representation

ActiveMimic introduces a procedure to recover temporally aligned head (camera) and bimanual wrist trajectories from single-RGB egocentric videos, discarding the need for motion capture, IMUs, or multi-camera calibration. Pose estimation is achieved using off-the-shelf models: SAM-3D-Body for wrists and VGGT for scale-aligned camera trajectories (with UniDepth for metric normalization). All poses are re-expressed in a temporally-local, chunk-centered reference, fully decoupling wrist and camera motion.

A joint, 27D action vector encodes each time chunk comprising: camera position and 6D rotation (9D), left wrist (9D), and right wrist (9D). This unified action space supports co-training of manipulation and viewpoint repositioning, tightly coupling active perception with physical action.

Figure 2: ActiveMimic workflow — from egocentric video, synchronized camera and wrist trajectories are derived, decoupled, and encoded as unified 27D actions for model pretraining and downstream adaptation.

Model Architecture and Training Protocol

ActiveMimic utilizes a mixture-of-transformers backbone, comprising a vision-language prefix for multimodal context encoding and a transformer-based action expert. Pretraining is performed via a conditional flow-matching objective on large-scale, filtered egocentric video, followed by adaptation to target robots using task-specific demonstrations.

Pretraining stages:

• Stage 1: Egocentric video pretraining on the unified 27D action, with VLM initialization and action expert warm-up.
• Stage 2: Robot-specific adaptation, transferring the learned active-perception representations.

Empirical Evaluation

Real-World Robotic Tasks

The evaluation encompasses four manipulation tasks of increasing perceptual complexity—Restocking (requiring viewpoint adjustment for object placement), Reaching, Finding (active search through head movement), and Pouring (bimanual coordination).

Figure 3: Robotic evaluation tasks—ranging from Restocking (head and posture movement) to bimanual Pouring.

ActiveMimic achieves strong, consistent gains:

• 90.1% success on Restocking (benchmarking active perception)
• 88.9% (Reaching), 91.7% (Finding), 93.3% (Pouring)

These metrics match or surpass state-of-the-art models pretrained solely on robot data ($\pi_0$), notably outperforming both prior human video pretraining (MotoVLA) and variants without viewpoint supervision.

Figure 4: Aggregate task success rates and decomposition for Restocking, showing the critical role of camera-head action supervision in complex tasks.

Quality of Action Estimation from Video

Label fidelity analyses on HOT3D, a ground-truth annotated egocentric dataset, demonstrate high recovery rates (head pose: 78.82%, wrists: 65.93%/61.72% under strict tolerances), validating that vision-based pose extraction via ActiveMimic yields effective supervision signals for pretraining.

Figure 5: Recovery rates and qualitative trajectory visualization on HOT3D, confirming reliable pose extraction from RGB-only input.

Ablations: Origin and Utilization of Active Perception

Systematic ablation reveals that strong active perception capability is specifically acquired in the egocentric pretraining phase (not just robot SFT), and expressively manifests via use of the head camera. Removal of the head camera at inference in Restocking causes success on placement actions to collapse from 24/27 to 1/27, highlighting the criticality of this feedback channel for fine-grained manipulation via learned active perception.

Figure 6: Layerwise overlap analysis and Restocking skill breakdown—quantifying the representational dependence on pretraining and head-view fusion.

Robustness and Transfer

ActiveMimic exhibits superior robustness to unstructured visual perturbations (e.g., flashing colored lights, unseen object packaging). Success rates under these OOD conditions drop minimally compared to wrist-only or robot-only pretrained baselines.

Additionally, an analysis of representational overlap between head-only and full-view inputs reveals that camera motion supervision during pretraining yields more transferable perceptual structures between human demonstrator and robot embodiment.

Figure 7: ActiveMimic demonstrates enhanced robustness across lighting and object shift scenarios, with the smallest performance degradation amongst all models.

Figure 8: Sensitivity analysis for activation overlap $\left(K\right)$ shows that camera motion learning reinforces head-view–full-view transfer at all perceptual layers.

Implications, Limitations, and Future Directions

Practical/Theoretical Impact

ActiveMimic’s design—recasting egocentric camera motion as viewpoint action and directly leveraging in-the-wild video—overcomes the need for prohibitively expensive robot data or hardware-intrusive capture rigs, greatly enhancing scalability and diversity of pretraining corpora. This shift enables large-scale robotic manipulation model pretraining analogous to what web-scale datasets have done for VLMs/LLMs.

The empirical results provide strong evidence that active perception is the principal missing ingredient for unlocking generalizable robot policies from egocentric video, with observable transfer to real-world, robust robotic skill acquisition.

Limitations acknowledged include:

• Limited embodiment evaluation (single robot morphology)
• No explicit evaluation of full-body loco-manipulation
• Pose labels derived from monocular video estimation (subject to noise)

Future research directions suggested are:

• Scaling pretraining to larger egocentric corpora (such as full Ego4D/Ego-Exo4D)
• Extending to diverse robotic embodiments and evaluating transfer across morphologies
• Incorporating richer task structures such as human-robot interaction in dynamic environments
• Advancing vision-only pose estimation to further improve supervision fidelity

Conclusion

ActiveMimic demonstrates that appropriately modeling and supervising camera viewpoint action during pretraining on egocentric human videos enables effective transfer of active perception to real-world humanoid robots. By unifying active perception and manipulation in training, ActiveMimic consistently outperforms existing human video–pretrained models and rivals robot data–pretrained SOTA. The approach paves the way for scalable, generalizable robot learning from diverse, organically produced human demonstration at scale.