Generalization of manipulation skills across full-body robot morphologies

Determine whether manipulation skills demonstrated by human operators using the handheld Hoi! gripper can generalize across robots with full-body morphologies that impose distinct kinematic and dynamic constraints, ensuring that transfer from human demonstrations respects these constraints in real manipulators.

Background

The Hoi! dataset records articulated object interactions under multiple embodiments, including a human hand, a human hand with a wrist-mounted camera, a handheld UMI gripper, and a custom Hoi! gripper with force–torque and tactile sensing. Although the Hoi! gripper enables human-operated demonstrations that mimic robotic end-effector interactions, these demonstrations are still produced by humans and do not fully reflect the kinematic and dynamic constraints of actual robot manipulators.

Given this embodiment gap, the authors explicitly note that achieving reliable transfer and generalization of skills from human-operated gripper demonstrations to robots with full-body morphologies remains unresolved. The dataset is designed to enable research on this question, but establishing such generalization requires addressing the robot-specific constraints that differ from human operation.