Address scarcity of high-quality robot manipulation data for training generalist policies
Develop scalable data collection, curation, or learning approaches that alleviate the scarcity of high-quality robot manipulation demonstrations needed to train generalist robotic manipulation policies effectively.
References
Despite progress in training generalist policies, challenges such as catastrophic forgetting, data heterogeneity, scarcity of high-quality data, multimodal fusion, handling dexterity, and maintaining real-time inference speed remain open research problems.
To mitigate the gap between gameplay and robot execution, inspired by prior works, we leave action-level supervision from game data to future work and instead extract transferable information from trajectories, specifically contact points and sub-goal states that mark task progress.
Modern simulators used for reinforcement learning (RL) cannot fully replicate the required contact complexity, while collecting dexterous demonstrations for imitation learning (IL) remains an open problem.
The expert library is predefined by skill semantics. Learning its structure and reducing the need for human demonstrations remain open directions.
We do not evaluate a 20-human-only condition, so this comparison does not isolate the incremental benefit of adding generated trajectories to those 20 human demonstrations.