Prometheus: Universal, Open-Source Mocap-Based Teleoperation System with Force Feedback for Dataset Collection in Robot Learning (2510.01023v1)

Abstract: This paper presents a novel teleoperation system with force feedback, utilizing consumer-grade HTC Vive Trackers 2.0. The system integrates a custom-built controller, a UR3 robotic arm, and a Robotiq gripper equipped with custom-designed fingers to ensure uniform pressure distribution on an embedded force sensor. Real-time compression force data is transmitted to the controller, enabling operators to perceive the gripping force applied to objects. Experimental results demonstrate that the system enhances task success rates and provides a low-cost solution for large-scale imitation learning data collection without compromising affordability.

• The paper presents an innovative teleoperation system that integrates force feedback with motion capture for enhanced robotic learning dataset collection.
• The system employs consumer-grade HTC Vive trackers, a UR3 robotic arm, and a modified Robotiq gripper to deliver robust, cost-effective haptic feedback.
• Experimental results show a 35.77% reduction in grip force and up to 90% success in handling fragile objects, highlighting its effectiveness in preventing damage.

Prometheus: Universal, Open-Source Mocap-Based Teleoperation System with Force Feedback

Overview of the Prometheus System

"Prometheus" introduces an advanced teleoperation system focused on force feedback and dataset collection for robot learning applications. The system leverages consumer-grade HTC Vive Trackers 2.0, a UR3 robotic arm, and a modified Robotiq 2F-85 gripper to affordably deliver force feedback. All system components are open-source, enabling replication and adaptation by researchers and developers interested in teleoperation and robotic learning.

Figure 1: Main view of the system.

System Architecture and Components

The system architecture includes a custom-built controller with HTC Vive Tracker 2.0, the UR3 robot, and a Robotiq 2F-85 gripper outfitted with custom-designed force-sensitive fingers. This integration aims to provide a comprehensive teleoperation experience by combining real-time visual and haptic feedback.

Controller and Force Feedback Mechanism

The haptic device, central to the controller design, employs a unique implementation where force feedback is delivered via a torque-based mechanism. The internal schematic of the controller highlights simplicity and efficacy, combining commercially available elements to achieve a robust solution.

Figure 2: External view of the controller.

Figure 3: Principle of work.

Figure 4: Internal schematic of the controller.

Force-Sensitive Robotics and Practical Implications

The inclusion of force sensing in robot grippers is a pivotal feature, distinguishing this system by enabling precise interaction with fragile or deformable objects. The gripper's design ensures uniform force distribution across the contact surface, safeguarding force sensors while delivering nuanced feedback to the operator.

Figure 5: Gripper with a force-feedback robotic finger.

Figure 6: Schematic of the robotic finger mechanism.

Performance Evaluation and User Study

In experimental scenarios, the system significantly lowers the grip force applied by users during teleoperation tasks, particularly when handling fragile objects such as eggs. Users applied 35.77% less force with enabled force feedback, highlighting the system's potential to prevent over-gripping and resultant damage.

Figure 7: Eggs experiment.

Figure 8: Gripper compression force: with vs. without force feedback.

Integration with Vision-Language-Action Models

Prometheus evaluates the impact of force feedback on Vision-Language-Action (VLA) models through three variants: position-only, force-only, and position-and-force policies. The combination of positional and force feedback proves superior in tasks involving deformable objects, achieving up to 90% success in handling tomatoes and fragile items, indicating the system's robust adaptability for complex manipulative tasks.

Figure 9: Success rates across three policy variants (position-only, force-only, and position-and-force) evaluated on three object types: toothpaste tube, tomato, and shampoo bottle. Each policy was tested in 10 independent trials per object.

Figure 10: Time-series analysis of grasp execution dynamics across three manipulation tasks. Each task is represented by two subplots: (a) continuous gripper position and (b) measured contact force. Both are standardized to the range [0, 1]. Curves show mean values averaged over 10 trials for three policy variants: position-only (orange), force-only (green), and position-and-force (blue).

Discussion and Future Work

The teleoperation system demonstrates significant progress within robotic teleoperation, emphasizing cost-effectiveness, ease of reproduction, and improved data collection for robot learning models. The findings suggest that integrating tactile feedback in teleoperation can enhance data quality, reduce over-gripping errors, and ultimately refine robot learning algorithms. Future explorations can include expanding system compatibility across diverse robotic platforms and further developing the sensory feedback mechanisms to encompass a wider array of tactile and force sensations.

Conclusion

"Prometheus" represents a critical step in teleoperation systems by integrating comprehensive force feedback mechanisms, improving robotic task accuracy, and enhancing human-robot interaction efficiency. Through its open-source availability, the system sets a foundation for widespread research collaboration and iterative development, supporting innovations in robot learning data collection and manipulation proficiencies.