Generalization of manipulation skills across full-body robot morphologies

Determine whether manipulation skills demonstrated by human operators using the handheld Hoi! gripper can generalize across robots with full-body morphologies that impose distinct kinematic and dynamic constraints, ensuring that transfer from human demonstrations respects these constraints in real manipulators.

Background

The Hoi! dataset records articulated object interactions under multiple embodiments, including a human hand, a human hand with a wrist-mounted camera, a handheld UMI gripper, and a custom Hoi! gripper with force–torque and tactile sensing. Although the Hoi! gripper enables human-operated demonstrations that mimic robotic end-effector interactions, these demonstrations are still produced by humans and do not fully reflect the kinematic and dynamic constraints of actual robot manipulators.

Given this embodiment gap, the authors explicitly note that achieving reliable transfer and generalization of skills from human-operated gripper demonstrations to robots with full-body morphologies remains unresolved. The dataset is designed to enable research on this question, but establishing such generalization requires addressing the robot-specific constraints that differ from human operation.

References

This hybrid embodiment simplifies skill transfer in some way but complicates it in others since true generalization across full-body morphology is still an open challenge.

— Hoi! -- A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation  (2512.04884 - Engelbracht et al., 4 Dec 2025) in Section 6: Limitations and Future Work

Finally, while our experiments are conducted on manipulation benchmarks with robotic arms, we will further investigate whether our findings generalize across a broader range of robotic platforms, including dexterous hands, quadruped robots, and humanoid robots.

— What Matters for Latent Actions in Robot Learning  (2608.19613 - Bu et al., 20 Aug 2026) in Section Limitation and Future Work

The result suggests that camera-centric motion provides a transferable intermediate target, while also showing that morphology and kinematic mismatch remain open challenges.

— One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation  (2608.26058 - Team et al., 26 Aug 2026) in Section 4, subsection “New Embodiment Transfer and Adaptation,” paragraph b) Simulation