Sufficiency of real‑world teleoperation–trained foundation models for general manipulation

Determine whether the paradigm of collecting large-scale real-world teleoperation datasets and training robotic foundation models exclusively on this data is, by itself, sufficient to achieve general-purpose manipulation capabilities across diverse tasks and environments.

Background

Recent efforts in robotics have sought to replicate the LLM approach by scaling up real-world datasets and training robotic foundation models from teleoperation data. The paper notes that although this direction is promising, the variation encountered in mobile manipulation—especially for humanoids—implies substantially higher data requirements and costs compared to fixed tabletop setups.

This uncertainty directly motivates the paper’s exploration of visual sim-to-real as a complementary path, aiming to reduce the reliance on expensive real-world data collection while enabling robust, long-horizon loco-manipulation on humanoid robots.