Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Low-Cost Teleoperation with Haptic Feedback through Vision-based Tactile Sensors for Rigid and Soft Object Manipulation (2403.16764v1)

Published 25 Mar 2024 in cs.RO

Abstract: Haptic feedback is essential for humans to successfully perform complex and delicate manipulation tasks. A recent rise in tactile sensors has enabled robots to leverage the sense of touch and expand their capability drastically. However, many tasks still need human intervention/guidance. For this reason, we present a teleoperation framework designed to provide haptic feedback to human operators based on the data from camera-based tactile sensors mounted on the robot gripper. Partial autonomy is introduced to prevent slippage of grasped objects during task execution. Notably, we rely exclusively on low-cost off-the-shelf hardware to realize an affordable solution. We demonstrate the versatility of the framework on nine different objects ranging from rigid to soft and fragile ones, using three different operators on real hardware.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. A. Yamaguchi and C. G. Atkeson, “Recent progress in tactile sensing and sensors for robotic manipulation: can we turn tactile sensing into vision?,” Advanced Robotics, vol. 33, no. 14, pp. 661–673, 2019.
  2. S. Zhang, Z. Chen, Y. Gao, W. Wan, J. Shan, H. Xue, F. Sun, Y. Yang, and B. Fang, “Hardware technology of vision-based tactile sensor: A review,” IEEE Sensors Journal, vol. 22, no. 22, pp. 21410–21427, 2022.
  3. A. Yamaguchi and C. G. Atkeson, “Tactile behaviors with the vision-based tactile sensor fingervision,” International Journal of Humanoid Robotics, vol. 16, no. 03, p. 1940002, 2019.
  4. X. Zhu, S. Jain, M. Tomizuka, and J. Van Baar, “Learning to synthesize volumetric meshes from vision-based tactile imprints,” in International Conference on Robotics and Automation (ICRA), pp. 4833–4839, 2022.
  5. M. C. Welle, M. Lippi, H. Lu, J. Lundell, A. Gasparri, and D. Kragic, “Enabling robot manipulation of soft and rigid objects with vision-based tactile sensors,” in International Conference on Automation Science and Engineering (CASE), 2023.
  6. K. Darvish, L. Penco, J. Ramos, R. Cisneros, J. Pratt, E. Yoshida, S. Ivaldi, and D. Pucci, “Teleoperation of humanoid robots: A survey,” IEEE Transactions on Robotics, 2023.
  7. I. El Rassi and J.-M. El Rassi, “A review of haptic feedback in tele-operated robotic surgery,” Journal of medical engineering & technology, vol. 44, no. 5, pp. 247–254, 2020.
  8. A. Bolopion and S. Régnier, “A review of haptic feedback teleoperation systems for micromanipulation and microassembly,” IEEE Transactions on Automation Science and Engineering, vol. 10, no. 3, pp. 496–502, 2013.
  9. M. Lambeta, P.-W. Chou, S. Tian, B. Yang, B. Maloon, V. R. Most, D. Stroud, R. Santos, A. Byagowi, G. Kammerer, et al., “Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 3838–3845, 2020.
  10. Y. Xie, X. Hou, and S. Wang, “Design of a novel haptic joystick for the teleoperation of continuum-mechanism-based medical robots,” Robotics, vol. 12, no. 2, p. 52, 2023.
  11. Y. Ishiguro, T. Makabe, Y. Nagamatsu, Y. Kojio, K. Kojima, F. Sugai, Y. Kakiuchi, K. Okada, and M. Inaba, “Bilateral humanoid teleoperation system using whole-body exoskeleton cockpit tablis,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6419–6426, 2020.
  12. P. Gliesche, T. Krick, M. Pfingsthorn, S. Drolshagen, C. Kowalski, and A. Hein, “Kinesthetic device vs. keyboard/mouse: a comparison in home care telemanipulation,” Frontiers in Robotics and AI, vol. 7, p. 561015, 2020.
  13. V. Nandikolla and D. A. Medina Portilla, “Teleoperation robot control of a hybrid eeg-based bci arm manipulator using ros,” Journal of Robotics, vol. 2022, 2022.
  14. M. Wozniak, C. T. Chang, M. B. Luebbers, B. Ikeda, M. Walker, E. Rosen, and T. R. Groechel, “Virtual, augmented, and mixed reality for human-robot interaction (vam-hri),” in Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, pp. 938–940, 2023.
  15. C. Barentine, A. McNay, R. Pfaffenbichler, A. Smith, E. Rosen, and E. Phillips, “A vr teleoperation suite with manipulation assist,” in Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, pp. 442–446, 2021.
  16. S. Xu, S. Moore, and A. Cosgun, “Shared-control robotic manipulation in virtual reality,” arXiv preprint arXiv:2205.10564, 2022.
  17. M. Moletta, M. K. Wozniak, M. C. Welle, and D. Kragic, “A virtual reality framework for human-robot collaboration in cloth folding,” arXiv preprint arXiv:2305.07493, 2023.
  18. K. Chandan, V. Kudalkar, X. Li, and S. Zhang, “Arroch: Augmented reality for robots collaborating with a human,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 3787–3793, IEEE, 2021.
  19. M. K. Wozniak, R. Stower, P. Jensfelt, and A. Pereira, “Happily error after: Framework development and user study for correcting robot perception errors in virtual reality,” arXiv preprint arXiv:2306.14589, 2023.
  20. M. Ostanin and A. Klimchik, “Interactive robot programing using mixed reality,” IFAC-PapersOnLine, vol. 51, no. 22, pp. 50–55, 2018.
  21. M. Lorenzini, S. Ciotti, J. M. Gandarias, S. Fani, M. Bianchi, and A. Ajoudani, “Performance analysis of vibrotactile and slide-and-squeeze haptic feedback devices for limbs postural adjustment,” in 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 707–713, IEEE, 2022.
  22. Z. Chua and A. M. Okamura, “A modular 3-degrees-of-freedom force sensor for robot-assisted minimally invasive surgery research,” Sensors, vol. 23, no. 11, p. 5230, 2023.
  23. A. Smith, B. Ward-Cherrier, A. Etoundi, and M. J. Pearson, “Feeling the pressure: The influence of vibrotactile patterns on feedback perception.,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 634–640, IEEE, 2022.
  24. R. Luz, A. Pereira, J. Corujeira, T. Krueger, J. Beck, E. den Exter, T. Chupin, J. L. Silva, and R. Ventura, “Feeling the slope? teleoperation of a mobile robot using a 7dof haptic device with attitude feedback,”
  25. R. Pocius, N. Zamani, H. Culbertson, and S. Nikolaidis, “Communicating robot goals via haptic feedback in manipulation tasks,” in Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pp. 591–593, 2020.
  26. Z. Al-Saadi, Y. M. Hamad, Y. Aydin, A. Kucukyilmaz, and C. Basdogan, “Resolving conflicts during human-robot co-manipulation,” in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, pp. 243–251, 2023.
  27. R. Hinchet, V. Vechev, H. Shea, and O. Hilliges, “Dextres: Wearable haptic feedback for grasping in vr via a thin form-factor electrostatic brake,” in Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pp. 901–912, 2018.
  28. J. E. Palmer, M. Sarac, A. A. Garza, and A. M. Okamura, “Haptic feedback relocation from the fingertips to the wrist for two-finger manipulation in virtual reality,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 628–633, IEEE, 2022.
  29. M. Coffey and A. Pierson, “Collaborative teleoperation with haptic feedback for collision-free navigation of ground robots,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8141–8148, IEEE, 2022.
  30. A. A. Stanley and K. J. Kuchenbecker, “Evaluation of tactile feedback methods for wrist rotation guidance,” IEEE Transactions on Haptics, vol. 5, no. 3, pp. 240–251, 2012.
  31. I. Andrussow, H. Sun, K. J. Kuchenbecker, and G. Martius, “Minsight: A fingertip-sized vision-based tactile sensor for robotic manipulation,” Advanced Intelligent Systems, p. 2300042, 2023.
  32. W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, p. 2762, 2017.
  33. G. Cao, J. Jiang, N. Mao, D. Bollegala, M. Li, and S. Luo, “Vis2hap: Vision-based haptic rendering by cross-modal generation,” arXiv preprint arXiv:2301.06826, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now