Papers
Topics
Authors
Recent
Search
2000 character limit reached

Attention-based Estimation and Prediction of Human Intent to augment Haptic Glove aided Control of Robotic Hand

Published 15 Oct 2021 in cs.RO and cs.AI | (2110.07953v2)

Abstract: The letter focuses on Haptic Glove (HG) based control of a Robotic Hand (RH) executing in-hand manipulation of certain objects of interest. The high dimensional motion signals in HG and RH possess intrinsic variability of kinematics resulting in difficulty to establish a direct mapping of the motion signals from HG onto the RH. An estimation mechanism is proposed to quantify the motion signal acquired from the human controller in relation to the intended goal pose of the object being held by the robotic hand. A control algorithm is presented to transform the synthesized intent at the RH and allow relocation of the object to the expected goal pose. The lag in synthesis of the intent in the presence of communication delay leads to a requirement of predicting the estimated intent. We leverage an attention-based convolutional neural network encoder to predict the trajectory of intent for a certain lookahead to compensate for the delays. The proposed methodology is evaluated across objects of different shapes, mass, and materials. We present a comparative performance of the estimation and prediction mechanisms on 5G-driven real-world robotic setup against benchmark methodologies. The test-MSE in prediction of human intent is reported to yield ~ 97.3 -98.7% improvement of accuracy in comparison to LSTM-based benchmark

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. L. Colasanto, R. Suárez, and J. Rosell, “Hybrid mapping for the assistance of teleoperated grasping tasks,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 2, pp. 390–401, 2012.
  2. H. Su, W. Qi, C. Yang, J. Sandoval, G. Ferrigno, and E. De Momi, “Deep neural network approach in robot tool dynamics identification for bilateral teleoperation,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2943–2949, 2020.
  3. Z. Wang, H.-K. Lam, B. Xiao, Z. Chen, B. Liang, and T. Zhang, “Event-triggered prescribed-time fuzzy control for space teleoperation systems subject to multiple constraints and uncertainties,” IEEE Transactions on Fuzzy Systems, 2020.
  4. P. Owan, J. Garbini, and S. Devasia, “Faster confined space manufacturing teleoperation through dynamic autonomy with task dynamics imitation learning,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2357–2364, 2020.
  5. Y. Qin, H. Su, and X. Wang, “From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 873–10 881, 2022.
  6. A. Handa, K. Van Wyk, W. Yang, J. Liang, Y.-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox, “Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 9164–9170.
  7. O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al., “Learning dexterous in-hand manipulation,” The International Journal of Robotics Research, vol. 39, no. 1, pp. 3–20, 2020.
  8. S. Cruciani, B. Sundaralingam, K. Hang, V. Kumar, T. Hermans, and D. Kragic, “Benchmarking in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 588–595, 2020.
  9. S. Li, N. Hendrich, H. Liang, P. Ruppel, C. Zhang, and J. Zhang, “A dexterous hand-arm teleoperation system based on hand pose estimation and active vision,” IEEE Transactions on Cybernetics, 2022.
  10. N. Y. Lii, Z. Chen, M. A. Roa, A. Maier, B. Pleintinger, and C. Borst, “Toward a task space framework for gesture commanded telemanipulation,” in 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.   IEEE, 2012, pp. 925–932.
  11. H. Liu, X. Xie, M. Millar, M. Edmonds, F. Gao, Y. Zhu, V. J. Santos, B. Rothrock, and S.-C. Zhu, “A glove-based system for studying hand-object manipulation via joint pose and force sensing,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2017, pp. 6617–6624.
  12. J. H. Low, W. W. Lee, P. M. Khin, N. V. Thakor, S. L. Kukreja, H. L. Ren, and C. H. Yeow, “Hybrid tele-manipulation system using a sensorized 3-d-printed soft robotic gripper and a soft fabric-based haptic glove,” IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 880–887, 2017.
  13. A. Peer, S. Einenkel, and M. Buss, “Multi-fingered telemanipulation-mapping of a human hand to a three finger gripper,” in RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication.   IEEE, 2008, pp. 465–470.
  14. T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” arXiv preprint arXiv:2304.13705, 2023.
  15. A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, et al., “Dextreme: Transfer of agile in-hand manipulation from simulation to reality,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 5977–5984.
  16. A. S. Morgan, K. Hang, B. Wen, K. Bekris, and A. M. Dollar, “Complex in-hand manipulation via compliance-enabled finger gaiting and multi-modal planning,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4821–4828, 2022.
  17. Y. Liu, Y.-Y. Tsai, B. Huang, and J. Guo, “Virtual reality based tactile sensing enhancements for bilateral teleoperation system with in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6998–7005, 2022.
  18. O. Azulay, I. Ben-David, and A. Sintov, “Learning haptic-based object pose estimation for in-hand manipulation control with underactuated robotic hands,” IEEE Transactions on Haptics, vol. 16, no. 1, pp. 73–85, 2023.
  19. L. Yang, B. Huang, Q. Li, Y.-Y. Tsai, W. W. Lee, C. Song, and J. Pan, “Tacgnn: Learning tactile-based in-hand manipulation with a blind robot using hierarchical graph neural network,” IEEE Robotics and Automation Letters, vol. 8, no. 6, pp. 3605–3612, 2023.
  20. X. Gu, Y. Zhang, W. Sun, Y. Bian, D. Zhou, and P. O. Kristensson, “Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in vr,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016, pp. 1991–1995.
  21. S. V. Shah, S. K. Saha, and J. K. Dutt, “Dynamics of tree-type robotic systems,” in Dynamics of Tree-Type Robotic Systems.   Springer, 2013, pp. 73–88.
  22. V. R. Kumar and K. J. Waldron, “Force distribution in closed kinematic chains,” IEEE Journal on Robotics and Automation, vol. 4, no. 6, pp. 657–664, 1988.
  23. J. K. Davidson, K. H. Hunt, and G. R. Pennock, “Robots and screw theory: applications of kinematics and statics to robotics,” J. Mech. Des., vol. 126, no. 4, pp. 763–764, 2004.
  24. Y. Zhang, F. Gao, and W. A. Gruver, “Determination of contact forces in grasping,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS’96, vol. 3.   IEEE, 1996, pp. 1038–1044.
  25. R. Kumar and S. Mukherjee, “Enhanced dynamic capability of cable-driven parallel manipulators by reconfiguration,” Robotica, pp. 1–19, 2021.
  26. J. C. A. Barata and M. S. Hussein, “The moore–penrose pseudoinverse: A tutorial review of the theory,” Brazilian Journal of Physics, vol. 42, no. 1-2, pp. 146–165, 2012.
  27. Y. Miao, Y. Jiang, L. Peng, M. S. Hossain, and G. Muhammad, “Telesurgery robot based on 5g tactile internet,” Mobile Networks and Applications, vol. 23, pp. 1645–1654, 2018.
  28. X. Zhou, Z. Yang, Y. Ren, W. Bai, B. Lo, and E. M. Yeatman, “Modified bilateral active estimation model: A learning-based solution to the time delay problem in robotic tele-control,” IEEE Robotics and Automation Letters, vol. 8, no. 5, pp. 2653–2660, 2023.
  29. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  30. Y. Qin, W. Yang, B. Huang, K. Van Wyk, H. Su, X. Wang, Y.-W. Chao, and D. Fox, “Anyteleop: A general vision-based dexterous robot arm-hand teleoperation system,” arXiv preprint arXiv:2307.04577, 2023.
  31. M. Ahmed, R. Kumar, Q. Abbas, B. Lall, A. A. Kherani, S. Jain, and S. Mukherjee, “Teleoperation work,” https://doi.org/10.5281/zenodo.7082832, 2022.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up