Universal Manipulation Exoskeleton: Learning Compliant Whole-body Policies with Real-time Torque Feedback

Abstract: For robots to work safely in household environments, they need to be compliant and react to torque and force feedback during contact. However, the majority of existing data collection pipelines still lack the ability to capture force and torque data for learning active compliant policies. In this paper, we present Universal Manipulation Exoskeleton (UME), an upper-limb exoskeleton that provides real-time haptic torque feedback while recording whole-arm configurations and joint torque signals for teleoperation. With transparent torque feedback, human operators can even unsheathe kinematically constrained objects while blindfolded. UME is low-cost, lightweight, and portable. Equipped with an embedded IMU, it enables teleoperation for mobile manipulation. With our proposed universal retargeting algorithm, UME can teleoperate a range of robots, including the 7DoF OpenArm, 7DoF Franka, and 6DoF X-ARM. We demonstrate that this combination of capabilities enables learning bimanual, whole-body, and active compliant policies that operate effectively in highly constrained spaces. The learned robust autonomous policies achieve high success rates across a variety of tasks, including long-horizon mobile manipulation, force-mediated box flipping, visually occluded box pushing, and space-constrained tabletop manipulation. Videos, code, and additional information can be found at https://ume-exo.github.io.

• The paper introduces a novel exoskeleton (UME) that provides real-time torque feedback for compliant, whole-body robotic manipulation.
• The design employs a universal retargeting algorithm that decomposes kinematics into sub-manipulators to ensure predictable mapping across diverse robotic platforms.
• Experimental results demonstrate success rates up to 0.95 and a 3.3× increase in demonstration throughput, highlighting improved performance over baseline methods.

Universal Manipulation Exoskeleton (UME): Enabling Scalable Compliant Whole-body Robot Policy Learning via Real-time Haptic Torque Teleoperation

System Overview and Motivation

The Universal Manipulation Exoskeleton (UME) represents a substantial advance in the collection and utilization of demonstration data for learning compliant, whole-body robotic manipulation policies. UME is a low-cost, lightweight, upper-limb exoskeleton system designed to provide transparent real-time haptic torque feedback while simultaneously recording detailed whole-arm configurations and joint torques. This combination enables teleoperators to intuitively and safely perform contact-rich, highly constrained, and force-mediated tasks, critically addressing the limitations of existing teleoperation and in-the-wild paradigms that lack comprehensive force/torque sensing or whole-arm demonstration capture.

Figure 1: Overview of Universal Manipulation Exoskeleton and its multi-robot retargeting capabilities.

UME's universal retargeting algorithm ensures hardware-agnostic teleoperation across diverse robot platforms, including 6- and 7-DoF arms such as OpenArm, Franka, and X-ARM, leveraging sub-manipulator decoupling for stable, near-singularity-robust mapping. The system integrates embedded IMU sensors to further facilitate mobile base control, enabling seamless execution of whole-body and mobile manipulation tasks.

Mechanical Design and Retargeting Methodology

Central to the UME architecture is a mechanical design that mimics the native biomechanics of the human upper limb. The exoskeleton includes a coaxially arranged 3-1-3 DoF actuation structure that mirrors the shoulder, elbow, and wrist kinematics of the human arm, ensuring both comfort and high demonstration fidelity.

Figure 2: UME mechanical design, highlighting coaxial actuators and ergonomic structural choices.

The universal retargeting algorithm decomposes both exoskeleton and target robot into three sub-manipulators—virtual 3-DoF spherical shoulder, 1-DoF elbow, and virtual 3-DoF wrist—retargeting the kinematic and haptic signals independently. This approach avoids global singularities, enables predictable mapping across divergent robot kinematic structures, and uniquely supports direct transmission of joint-level torque feedback to the operator. Joint configuration and torque signals are mapped in real time, with compensation for Coriolis, centrifugal, gravity, and friction effects to ensure interaction torque transparency.

Figure 3: UME retargeting algorithm for each sub-manipulator, supporting robot-agnostic control.

Learning Compliant Whole-body Policies

UME dramatically improves the data collection pipeline by offering synchronized, high-resolution proprioceptive (configuration) and force/torque signals, facilitating robust learning of compliant manipulation behaviors. For autonomous policy learning, Action Chunking with Transformers (ACT) is used to ingest both visual (ResNet18 image features) and proprioceptive (joint configuration and torque) data, producing policy outputs in the form of joint positions that are inherently modulated for compliance based on multimodal contextual cues.

The training process supports high policy robustness and generalization in contact-rich, force-regulated, and visually occluded settings, well beyond the capacity of prior baselines that lack torque/force modalities or whole-arm configuration capture.

Experimental Results

Four representative tasks—box pushing, box flipping, GPU picking in confined geometries, and long-horizon fridge drink retrieval—were used to benchmark UME-based policies versus two representative baselines: (1) No-torque (policy trained without torque signals) and (2) UMI (policy trained only from end-effector pose demonstrations).

Figure 4: Task descriptions for box pushing, box flipping, GPU picking, and fridge drink retrieval.

UME policies achieved significantly higher success rates across all task categories, notably with values of 0.90–0.95 in tightly constrained or force-mediated scenarios. The improvement is most pronounced in tasks requiring nuanced force application or where end-to-end policies must evoke whole-arm coordinated strategies that manage contacts, occlusions, and hybrid long-horizon action sequences. The No-torque and UMI baselines consistently yielded suboptimal outcomes, frequently failing altogether in high-precision force- or contact-centric tasks.

Figure 5: Ablations and representative failure cases of No-torque and UMI baselines.

A user study further quantified the demonstration-per-minute (DPM) throughput, showing that real-time haptic feedback increased data collection efficiency by 3.3× over systems without torque feedback, approaching 71% of the rate achieved by direct human demonstration.

Figure 6: User study illustrating elevated demonstration throughput with UME.

Additionally, UME's universal retargeting allowed cross-embodiment transfer: teleoperating X-ARMs and Franka platforms (in simulation) without retargeting instability or loss of intuitiveness.

Figure 7: UME teleoperates other robotic arms, exemplifying cross-embodiment generality.

Practical and Theoretical Implications

Practically, the deployment of UME provides an accessible, production-scalable pathway for the collection of rich demonstration datasets in contact-intensive environments—a major bottleneck for robust robot learning with compliance requirements. The platform's compatibility with general-purpose, off-the-shelf hardware reduces cost ($\sim$\$1900 for the exoskeleton; see Appendix), which accelerates reproducibility and widespread adoption for both academic and industrial robotic applications.

Theoretically, UME demonstrates that direct joint-level torque feedback and whole-arm state recording are critical enabling technology for learning policies that embrace, rather than avoid, physical contact in manipulation. By supporting end-to-end active compliance, UME enables learning of strategies where robots modulate contact force in real time, analogous to human skills in highly cluttered and dynamic environments. These findings are anticipated to influence future efforts in foundation models for robotics and may inform new architectures for learning-based admittance or impedance control.

Figure 8: UME achieves a wide range of human-equivalent upper-limb motion, critical for high-fidelity demonstration capture.

Figure 9: Key components of the in-house whole-body mobile manipulator platform.

Limitations and Future Directions

While UME establishes a new performance baseline, a number of design and application constraints remain. Structural elements fabricated in PLA plastic affect payload and long-term durability. The system lacks adjustable link lengths for optimized fit and ergonomics across diverse operators. Further, real-world Franka hardware is pending for live teleoperation validation, although simulation results support anticipated success.

Future developments may focus on the following directions:

• Lighter, production-grade exoskeletons with adjustable morphometrics.
• Broadening cross-embodiment policy transfer to more diverse robot families.
• Integration of additional sensing modalities (e.g., tactile, high-bitrate vision) to further enhance compliance and safety.
• Large-scale dataset collection to drive advances in generalist robotic foundation models incorporating compliance.

Conclusion

UME is a comprehensive, accessible system for bimanual whole-body teleoperation and learning, uniquely supporting real-time haptic torque feedback and universal retargeting across robot platforms. Empirical evaluations demonstrate substantially improved policy robustness and efficiency in high-challenge manipulation scenarios, underscoring the critical role of joint torque and configuration data in compliant robot learning. These contributions are expected to have strong downstream impacts, both in practical robot deployment and in foundational neural policy architectures for contact-rich tasks.