Robot Self-Improvement via Human-Video Dynamics Models

Abstract: A central question in robot learning is how to acquire skills from the kinds of data that humans learn from: passive observation, embodied practice, and the experience of failure. Human videos provide the first of these in abundance, and prior work has shown they can initialize useful policies. Far less clear is whether they can support the second and third: whether priors extracted from human videos can ground a robot's own attempts well enough to evaluate them, correct them, and improve from them. In this work, we show that human videos can be used to learn embodiment-agnostic action, dynamics, and value representations that transfer across robot embodiments, providing the predictive foundation required for robots to autonomously improve from their own rollouts and failures. We introduce Dynamics-Guided Action Correction (DGAC), a training-free approach that leverages these adapted models to repair failed states: each failure becomes a query for which the learned models propose and rank corrective actions, turning failures into supervision for the next policy update. Across seven real-world manipulation tasks spanning both a mobile manipulator and a static manipulator arm, our approach improves success rates from 40% to 81% across multiple policy backbones, demonstrating cross-embodiment robot self-improvement from human-video priors. These results show that human priors and robot failures can be combined to enable scalable autonomous policy improvement. Project page: https://ethz-mrl.github.io/robot-self-improvement-website/.

• The paper introduces DGAC, a dynamics-guided action correction method that repairs failed states using pretrained human video models.
• It leverages joint pretraining of policy, dynamics, and value models on human videos to enable scalable, cross-embodiment robot self-improvement.
• Empirical results show success rates increase from 40% to 81%, illustrating the framework’s significant impact on autonomous robotic learning.

Robot Self-Improvement via Human-Video Dynamics Models

Overview and Motivation

This paper introduces a scalable framework for robot policy self-improvement by leveraging dynamics and value models pretrained on human videos. The central hypothesis is that passive observation of human interactions provides reusable, embodiment-agnostic representations—action, world-state, and task-progress value—that, when transferred and grounded through robot rollouts, enable iterative autonomous policy improvement without online human supervision. This is operationalized by Dynamics-Guided Action Correction (DGAC), which repairs failed robot states via training-free action candidate generation and ranking, converting robot failures into new supervision signals.

Figure 1: The framework pipeline—pretraining on human videos yields transferable policy, dynamics, and value models that enable iterative robot self-improvement via DGAC.

The framework addresses key bottlenecks in real-world robot learning: high dependence on costly human demonstration and intervention, poor scalability of online correction, and limited generalizability across robot embodiments and policy architectures. By decoupling acquisition of interaction priors from embodiment-specific adaptation, the method utilizes the abundance and diversity of human videos as a source of predictive knowledge, thus supporting a paradigm shift from imitation-centric learning toward autonomous error-driven self-improvement.

Embodiment-Agnostic Representation and Model Pretraining

Human videos are processed to build action representations ($\bm{a}_t=[\bm{\xi}_t, c_t]$: absolute wrist 6-DoF motion + hand closure) and compact world-state representations ($\bm{o}_t=[\bm{z}_t, \bm{P}_t]$: DINO-v3 semantic visual tokens + short-horizon 3D point flows). Task progress is captured via value regression to sparse terminal rewards. All representations are designed to maximize transferability across robot morphologies and facilitate downstream prediction and evaluation.

Policy, dynamics, and value models are jointly pretrained on large-scale human video datasets using flow-matching objectives and standard regression for value estimation. The policy model uses history-based action chunking; the dynamics model predicts action-conditioned future states; the value model discriminates success/failure through learned scalar estimators. This pretraining establishes models capable of generalizing from human interaction to robot context—critical for scalable cross-embodiment adaptation.

Robot Adaptation and Iterative Self-Improvement

Upon deployment, models are adapted via autonomous robot exploration, seeded by VLM-generated interaction instructions and executed by the human-pretrained policy. This exposes models to robot-specific state distributions, natural failures, and contact dynamics, bridging the embodiment gap. Task-specific policies are initialized with minimal teleoperation data and then improved through iterative cycles of rollout, model adaptation, and action correction.

Figure 2: Seven manipulation tasks across two robot platforms test reaching, grasping, articulated-object manipulation, placement, and sequencing.

Dynamics-Guided Action Correction (DGAC)

DGAC is the centerpiece: failed states are queried for repair by retrieving progress-aligned successful experiences, generating action candidates via flow-matching velocity composition, and ranking them using the adapted dynamics and value models. Only candidates whose predicted outcomes remain recoverable (thresholded by value and state similarity) are considered. The highest-value action is selected and used to relabel the failed state, creating a repaired dataset for advantage-conditioned policy extraction.

Figure 3: DGAC mechanism—candidate actions are sampled, rolled out in the dynamics model, evaluated via value model, and the highest-value candidate is used as corrective supervision.

Figure 4: Qualitative illustration—DGAC steers failed predictions toward successful outcomes, yielding markedly improved rollouts after training on corrections.

Empirical Evaluation

The framework is empirically validated on seven real-world manipulation tasks using Hello Robot Stretch 3 and Franka Panda platforms. Baselines including SWIM, LPB, AWR, RECAP, and RISE are rigorously compared. DGAC-equipped policies achieve a success rate uplift from 40% to 81% across tasks, with superior results to all baselines. Importantly, policy-agnostic improvement is demonstrated: DGAC applied to a strong $\pi_{0.5}$ backbone boosts performance from 62.7% (fine-tuned) to 88.0%, a 20 point absolute gain beyond value-guided extraction alone.

Ablations show the necessity of both candidate generation and value-based ranking for effective correction—direct trajectory-copy or random selection fail to match the DGAC approach. Cross-embodiment transfer is evident: the same pipeline yields robust improvement for Franka Panda with heterogeneous manipulation tasks.

Figure 5: t-SNE visualization—DGAC-corrected dynamics move failed states toward successful regions, enabling interpretable repairs (e.g., aligning grasps, preventing collisions).

Figure 6: Advantage distributions—failed rollouts are uplifted toward successful distributions post-DGAC without introducing systematic over-optimism, evidencing calibrated corrective supervision.

Data scaling analysis reveals monotonic performance improvements with increased human-video pretraining—final success rates rise from 52% (no human pretrain) to 85.3% (full corpus). Robot adaptation phase is essential for reliable grounding.

Implications and Outlook

This work establishes that internet-scale passive human observation is sufficient to build reusable predictive foundations for robot learning, extending far beyond policy initialization. The combination of embodiment-agnostic representations, offline action correction, and policy-agnostic improvement enables scalable failure-to-supervision learning across diverse morphologies and tasks. DGAC obviates the need for costly online human-in-loop correction and is robust to base policy strength.

Practical implications are substantial: the approach enables robots to continually improve after deployment, leveraging their own rollouts and failures as self-supervision. Theoretically, this supports a shift toward semi-supervised, error-driven robot learning at scale, complementing demonstration- and reinforcement-based paradigms. Integration of foundation models for automatic outcome annotation, hierarchical DGAC for longer-horizon correction, and further scaling of human video corpora are promising future directions.

Conclusion

The presented framework demonstrates autonomous robot self-improvement grounded in predictive models pretrained from human videos. DGAC effectively converts recoverable failures into physically grounded corrections, enabling substantial gains in manipulation success across multiple robot embodiments and policy backbones. The results indicate that human video priors, when coupled with self-supervised robot interaction, provide a scalable route toward robust cross-embodiment policy improvement and failure-driven learning. Further advances in automatic feedback, hierarchical correction, and dataset scaling are likely to deepen the impact of this paradigm on the development of continuously adapting robotic systems.