HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit (2502.13013v2)

Abstract: Generalizable humanoid loco-manipulation poses significant challenges, requiring coordinated whole-body control and precise, contact-rich object manipulation. To address this, this paper introduces HOMIE, a semi-autonomous teleoperation system that combines a reinforcement learning policy for body control mapped to a pedal, an isomorphic exoskeleton arm for arm control, and motion-sensing gloves for hand control, forming a unified cockpit to freely operate humanoids and establish a data flywheel. The policy incorporates novel designs, including an upper-body pose curriculum, a height-tracking reward, and symmetry utilization. These features enable the system to perform walking and squatting to specific heights while seamlessly adapting to arbitrary upper-body poses. The exoskeleton, by eliminating the reliance on inverse dynamics, delivers faster and more precise arm control. The gloves utilize Hall sensors instead of servos, allowing even compact devices to achieve 15 or more degrees of freedom and freely adapt to any model of dexterous hands. Compared to previous teleoperation systems, HOMIE stands out for its exceptional efficiency, completing tasks in half the time; its expanded working range, allowing users to freely reach high and low areas as well as interact with any objects; and its affordability, with a price of just $500. The system is fully open-source, demos and code can be found in our https://homietele.github.io/.

• The paper proposes a robust humanoid loco-manipulation policy using reinforcement learning and an exoskeleton system, omitting costly motion capture data.
• The research presents an affordable isomorphic exoskeleton hardware system allowing a single operator to control the humanoid robot's full body movements.
• The system demonstrates high utility for collecting data used to train autonomous policies through imitation learning, advancing both teleoperation and robot autonomy.

Overview of "HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit"

The paper by Qingwei Ben et al. introduces HOMIE, an innovative approach to humanoid teleoperation, addressing significant challenges in maintaining stable and precise control in humanoid loco-manipulation tasks. The research integrates a novel reinforcemen learning framework and an isomorphic exoskeleton-based cockpit, aiming to enhance the operator's ability to control humanoid robots for diverse loco-manipulation tasks with improved agility and safety.

Core Contributions

1. Humanoid Loco-Manipulation Policy: The paper proposes a robust humanoid loco-manipulation policy that blends reinforcement learning and an exoskeleton-based control system. The policy trains humanoid robots to perform walking and squatting tasks while maintaining arbitrary upper-body poses. Key innovations include the introduction of a curriculum learning approach to handle continuous pose changes and a height-tracking reward that facilitates quick squatting while maintaining balance. This policy omits the need for motion priors, typically derived from cost-intensive motion capture data.
2. Exoskeleton-Based Hardware System: The authors present an affordable hardware system composed of isomorphic exoskeleton arms, motion-sensing gloves, and a foot pedal. The system enables a single operator to teleoperate both the upper and lower body of the humanoid robot. The isomorphic design allows for direct mapping of human motions onto the robot, reducing errors common in traditional vision-based systems. The low-cost configuration ($0.5k as opposed to several thousand dollars typical for MoCap systems) represents a significant advancement in accessibility.
3. Imitation Learning Applications: The research demonstrates the high utility of the data collected using the proposed system for training autonomous policies through imitation learning (IL). This finding underscores the dual capability of the organism in both teleoperation and as a tool for advancing humanoid autonomy.

Experimental Insights

The research presents extensive experiments to validate each component of the system:

• Performance Evaluation in Real-World and Simulation Environments: The paper includes a series of tasks performed by the robot, demonstrating capabilities such as object manipulation at varying heights and complex coordination tasks between multiple robots. The system is deployed in both real-world settings and simulated environments like Isaac Gym, which supports agile policy development through high-fidelity simulations.
• Ablation Study and Comparative Analysis: The evaluation features comprehensive ablation studies, isolating the impact of the novel training features and hardware integration. Furthermore, performance comparisons with existing methods show HOMIE's superiority in executing tasks with higher precision and speed.

Implications and Future Research

The implications of this work are profound, potentially transforming humanoid teleoperation in environments requiring nuanced, real-time interaction between human operators and robots. The authors speculate that the simplicity and cost-efficiency of the HOMIE system can accelerate the deployment of humanoid robots in sectors such as logistics and telepresence.

For future developments, the authors acknowledge challenges related to further enhancing terrain adaptability and incorporating force feedback for haptic interactions. Additionally, further exploration into visual and tactile perception integration is suggested to augment the robot's autonomy.

This research marks a significant step forward in the practical utility of humanoid robots, combining human dexterity with machine endurance. The open-sourcing of this project stands to spur collaborative advancements and community-based innovations in the field of humanoid robotics.