Closed-loop transfer of H-JEPA gains to physical robots

Determine whether the planning gains achieved by H-JEPA in offline evaluation on real-robot DROID videos translate to closed-loop control on a physical robot.

Background

H-JEPA is evaluated on simulated navigation and manipulation environments and on DROID, a corpus of real teleoperated robot videos. In the DROID experiments, planning is offline and open-loop: predicted action sequences are scored against logged expert trajectories rather than executed by a physical robot with replanning based on new observations.

The paper therefore leaves unresolved whether H-JEPA's reported improvements in planning fidelity and planner-compute efficiency will persist when actions are executed on a real robot in closed loop, where model errors, sensing limitations, contact dynamics, and feedback-dependent replanning can affect performance. Establishing this transfer would test the practical control relevance of the proposed hierarchical world model.