Robust Dexterous Grasping in Complex Real-World Clutter

Develop robust dexterous grasping methods that perform reliably in complex real-world cluttered settings, addressing target-object localization, large-scale scene-specific grasp learning, and robustness across diverse supporting bases and spatial layouts.

Background

The paper identifies realistic cluttered-scene dexterous grasping as substantially more difficult than the simplified single-object or uncluttered-tabletop settings commonly used in prior work. In clutter, a grasping system must distinguish semantically different objects with similar geometries, reason about spatial interactions with neighboring objects and supporting surfaces, and generalize across varied scene layouts and bases.

The authors explicitly characterize robust performance in complex real-world settings as unresolved. Their OmniDex benchmark and model address several contributing bottlenecks through multimodal observations, large-scale scene generation, physical validation, and physics-aware grasp generation, but the broader challenge of robust real-world cluttered-scene grasping remains identified as an open problem.