Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TeleFMG: A Wearable Force-Myography Device for Natural Teleoperation of Multi-finger Robotic Hands (2308.13929v2)

Published 26 Aug 2023 in cs.RO, cs.SY, and eess.SY

Abstract: Teleoperation enables a user to perform dangerous tasks (e.g., work in disaster zones or in chemical plants) from a remote location. Nevertheless, common approaches often provide cumbersome and unnatural usage. In this letter, we propose TeleFMG, an approach for teleoperation of a multi-finger robotic hand through natural motions of the user's hand. By using a low-cost wearable Force-Myography (FMG) device, musculoskeletal activities on the user's forearm are mapped to hand poses which, in turn, are mimicked by a robotic hand. The mapping is performed by a spatio-temporal data-based model based on the Temporal Convolutional Network. The model considers spatial positions of the sensors on the forearm along with temporal dependencies of the FMG signals. A set of experiments show the ability of a teleoperator to control a multi-finger hand through intuitive and natural finger motion. A robot is shown to successfully mimic the user's hand in object grasping and gestures. Furthermore, transfer to a new user is evaluated while showing that fine-tuning with a limited amount of new data significantly improves accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. S. R. Razu, T. Yasmin, T. B. Arif, M. S. Islam, S. M. S. Islam, H. A. Gesesew, and P. Ward, “Challenges faced by healthcare professionals during the covid-19 pandemic: A qualitative inquiry from bangladesh,” Frontiers in Public Health, vol. 9, 2021.
  2. H. Chen, P. Huang, and Z. Liu, “Mode switching-based symmetric predictive control mechanism for networked teleoperation space robot system,” IEEE/ASME Trans. on Mechat., vol. 24, no. 6, pp. 2706–2717, 2019.
  3. S. Sivčev, J. Coleman, E. Omerdić, G. Dooly, and D. Toal, “Underwater manipulators: A review,” Ocean Eng., vol. 163, pp. 431–450, 2018.
  4. S. Ozturkcan and E. Merdin-Uygur, “Humanoid service robots: The future of healthcare?” Journal of Information Technology Teaching Cases, vol. 12, no. 2, pp. 163–169, 2022.
  5. L. Hung, G. Hu, J. Wong, H. Ren, N. Ahmed, A. Hussein, E. Young, A. Berndt, J. Mann, R. Corepal, and L. Wong, “Telepresence robots in long-term care settings in british columbia during the covid-19 pandemic: Analyzing the experiences of residents and family members,” Gerontology and Geriatric Medicine, vol. 9, 2023.
  6. M. Fisher, R. C. Cardoso, E. C. Collins, C. Dadswell, L. A. Dennis, C. Dixon, M. Farrell, A. Ferrando, X. Huang, M. Jump, G. Kourtis, A. Lisitsa, M. Luckcuck, S. Luo, V. Page, F. Papacchini, and M. Webster, “An overview of verification and validation challenges for inspection robots,” Robotics, vol. 10, no. 2, 2021.
  7. S. Fani, S. Ciotti, M. G. Catalano, G. Grioli, A. Tognetti, G. Valenza, A. Ajoudani, and M. Bianchi, “Simplifying telerobotics: Wearability and teleimpedance improves human-robot interactions in teleoperation,” IEEE Rob. & Aut. Mag., vol. 25, no. 1, pp. 77–88, 2018.
  8. A. García, J. E. Solanes, A. Muñoz, L. Gracia, and J. Tornero, “Augmented reality-based interface for bimanual robot teleoperation,” Applied Sciences, vol. 12, no. 9, 2022.
  9. J. Singh, A. R. Srinivasan, G. Neumann, and A. Kucukyilmaz, “Haptic-guided teleoperation of a 7-dof collaborative robot arm with an identical twin master,” IEEE Trans. on Haptics, vol. 13, no. 1, pp. 246–252, 2020.
  10. R. Li, H. Wang, and Z. Liu, “Survey on mapping human hand motion to robotic hands for teleoperation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 2647–2665, 2022.
  11. A. Handa, K. Van Wyk, W. Yang, J. Liang, Y.-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox, “Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system,” in IEEE International Conference on Robotics and Automation (ICRA), 2020.
  12. Q. Gao, Z. Ju, Y. Chen, Q. Wang, and C. Chi, “An efficient rgb-d hand gesture detection framework for dexterous robot hand-arm teleoperation system,” IEEE Trans. Human-Machine Sys., vol. 53(1), pp. 13–23, 2023.
  13. P. Weber, E. Rueckert, R. Calandra, J. Peters, and P. Beckerle, “A low-cost sensor glove with vibrotactile feedback and multiple finger joint and hand motion sensing for human-robot interaction,” in IEEE Int. Symp.m on Robot and Human Interactive Comm., 2016, pp. 99–104.
  14. “Haptx gloves,” https://haptx.com/.
  15. L. Bi, A. G. Feleke, and C. Guan, “A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration,” Biomedical Signal Processing and Control, vol. 51, pp. 113 – 127, 2019.
  16. M. T. Wolf, C. Assad, M. T. Vernacchia, J. Fromm, and H. L. Jethani, “Gesture-based robot control with variable autonomy from the jpl biosleeve,” in IEEE Int. Conf. on Rob. and Aut., 2013, pp. 1160–1165.
  17. C. Meeker and M. Ciocarlie, “EMG-controlled non-anthropomorphic hand teleoperation using a continuous teleoperation subspace,” in IEEE Int. Conf. on Rob. and Aut., 2019, pp. 1576–1582.
  18. J. Luo, Z. Lin, Y. Li, and C. Yang, “A teleoperation framework for mobile robots based on shared control,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 377–384, 2020.
  19. E. Fujiwara, Y. T. Wu, C. K. Suzuki, D. T. G. de Andrade, A. R. Neto, and E. Rohmer, “Optical fiber force myography sensor for applications in prosthetic hand control,” in 2018 IEEE 15th International Workshop on Advanced Motion Control (AMC).   IEEE, 2018, pp. 342–347.
  20. O. Amft, H. Junker, P. Lukowicz, G. Troster, and C. Schuster, “Sensing muscle activities with body-worn sensors,” in Int. Workshop on Wearable and Implantable Body Sensor Networks, 2006, pp. 4–pp.
  21. G. Ogris, M. Kreil, and P. Lukowicz, “Using fsr based muscule activity monitoring to recognize manipulative arm gestures,” in IEEE International Symposium on Wearable Computers, 2007, pp. 45–48.
  22. N. Li, D. Yang, L. Jiang, H. Liu, and H. Cai, “Combined use of FSR sensor array and SVM classifier for finger motion recognition based on pressure distribution map,” Journal of Bionic Engineering, vol. 9, no. 1, pp. 39–47, 2012.
  23. J. Gantenbein, C. Ahmadizadeh, O. Heeb, O. Lambercy, and C. Menon, “Feasibility of force myography for the direct control of an assistive robotic hand orthosis in non-impaired individuals,” J NeuroEngineering and Rehabilitation, vol. 20, no. 101, 2023.
  24. H. K. Yap, A. Mao, J. C. H. Goh, and C. Yeow, “Design of a wearable FMG sensing system for user intent detection during hand rehabilitation with a soft robotic glove,” in IEEE Int. Conf. on Biomedical Rob. and Biomech., 2016, pp. 781–786.
  25. X. Jiang, L.-K. Merhi, Z. G. Xiao, and C. Menon, “Exploration of force myography and surface electromyography in hand gesture classification,” Med. Engineering & Physics, vol. 41, pp. 63–73, 2017.
  26. A. Belyea, K. Englehart, and E. Scheme, “FMG versus EMG: A comparison of usability for real-time pattern recognition based control,” IEEE Trans. on Biomed. Eng., vol. 66, no. 11, pp. 3098–3104, 2019.
  27. N. D. Kahanowich and A. Sintov, “Robust classification of grasped objects in intuitive human-robot collaboration using a wearable force-myography device,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1192–1199, 2021.
  28. E. Bamani, N. D. Kahanowich, I. Ben-David, and A. Sintov, “Robust multi-user in-hand object recognition in human-robot collaboration using a wearable force-myography device,” IEEE Robotics and Automation Letters, vol. 7, no. 1, pp. 104–111, 2022.
  29. Y. Yu, X. Si, C. Hu, and J. Zhang, “A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures,” Neural Computation, vol. 31, no. 7, pp. 1235–1270, 07 2019.
  30. A. Orvieto, S. L. Smith, A. Gu, A. Fernando, C. Gulcehre, R. Pascanu, and S. De, “Resurrecting recurrent neural networks for long sequences,” ArXiv, vol. abs/2303.06349, 2023.
  31. S. Wang, J. Cao, and P. S. Yu, “Deep learning for spatio-temporal data mining: A survey,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 8, pp. 3681–3700, 2022.
  32. S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” in Conference on Robotics Research, vol. abs/1803.01271.
  33. P. Tuffield and H. Elias, “The shadow robot mimics human actions,” Industrial Robot: An Int. Journal, vol. 30, no. 1, pp. 56–60, 2003.
  34. J. Butterfass, M. Grebenstein, H. Liu, and G. Hirzinger, “Dlr-hand ii: next generation of a dextrous robot hand,” in IEEE International Conference on Robotics and Automation, vol. 1, 2001, pp. 109–114.
  35. N. J. Jarque-Bou, A. Scano, M. Atzori, and H. Müller, “Kinematic synergies of hand grasps: a comprehensive study on a large publicly available dataset,” J. of NeuroEng. and Rehabilitation, vol. 16, 2019.
  36. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in North American Chapter of the Asso. for Comp. Linguistics, 2019.
  37. L. Biewald, “Experiment tracking with weights and biases,” 2020, software available from wandb.com. [Online]. Available: https://www.wandb.com/
  38. J. Yang, K.-Q. Shen, C. Ong, and X.-P. Li, “Feature selection for mlp neural network: The use of random permutation of probabilistic outputs,” IEEE Trans. on Neural Net., vol. 20, pp. 1911 – 1922, 2010.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube