Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Visual Attention Based Cognitive Human-Robot Collaboration for Pedicle Screw Placement in Robot-Assisted Orthopedic Surgery (2405.09359v1)

Published 15 May 2024 in cs.RO and cs.HC

Abstract: Current orthopedic robotic systems largely focus on navigation, aiding surgeons in positioning a guiding tube but still requiring manual drilling and screw placement. The automation of this task not only demands high precision and safety due to the intricate physical interactions between the surgical tool and bone but also poses significant risks when executed without adequate human oversight. As it involves continuous physical interaction, the robot should collaborate with the surgeon, understand the human intent, and always include the surgeon in the loop. To achieve this, this paper proposes a new cognitive human-robot collaboration framework, including the intuitive AR-haptic human-robot interface, the visual-attention-based surgeon model, and the shared interaction control scheme for the robot. User studies on a robotic platform for orthopedic surgery are presented to illustrate the performance of the proposed method. The results demonstrate that the proposed human-robot collaboration framework outperforms full robot and full human control in terms of safety and ergonomics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. P. E. Dupont, B. J. Nelson, M. Goldfarb, B. Hannaford, A. Menciassi, M. K. O’Malley, N. Simaan, P. Valdastri, and G.-Z. Yang, “A decade retrospective of medical robotics research from 2010 to 2020,” Science Robotics, vol. 6, no. 60, p. eabi8017, Nov. 2021.
  2. Y. Guo, W. Chen, J. Zhao, and G.-Z. Yang, “Medical robotics: Opportunities in china,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, no. 1, pp. 361–383, May 2022.
  3. B. Jiang, T. D. Azad, E. Cottrill, C. C. Zygourakis, A. M. Zhu, N. Crawford, and N. Theodore, “New spinal robotic technologies,” Frontiers of Medicine, vol. 13, no. 6, pp. 723–729, Dec. 2019.
  4. J. Li, L. Huang, W. Zhou, Z. Wang, Z. Li, L. Zeng, Z. Liu, H. Shen, Z. Cai, H. Gu, X. Yang, R. Zhang, W. Hu, M. Yu, and J. Chen, “Evaluation of a new spinal surgical robotic system of Kirschner wire placement for lumbar fusion: A multi-centre, randomised controlled clinical study,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 17, no. 2, p. e2207, Apr. 2021.
  5. B. Hagag, R. Abovitz, H. Kang, B. Schmitz, and M. Conditt, “RIO: Robotic-arm interactive orthopedic system MAKOplasty: User interactive haptic orthopedic robotics,” in Surgical Robotics: Systems Applications and Visions, J. Rosen, B. Hannaford, and R. M. Satava, Eds.   Boston, MA: Springer US, 2011, pp. 219–246.
  6. C. Lauretti, F. Cordella, C. Tamantini, C. Gentile, F. S. di Luzio, and L. Zollo, “A surgeon-robot shared control for ergonomic pedicle screw fixation,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2554–2561, Apr. 2020.
  7. A. D. Smith, J. Chapin, P. V. Birinyi, P. V. Bhagvath, and A. F. Hall, “Automated polyaxial screw placement using a commercial-robot-based, image-guided spine surgery system,” IEEE Transactions on Medical Robotics and Bionics, vol. 3, no. 1, pp. 74–84, Feb. 2021.
  8. R. H. Taylor, A. Menciassi, G. Fichtinger, P. Fiorini, and P. Dario, “Medical robotics and computer-integrated surgery,” in Springer Handbook of Robotics, ser. Springer Handbooks, B. Siciliano and O. Khatib, Eds.   Cham: Springer International Publishing, 2016, pp. 1657–1684.
  9. G. Boschetti, G. Rosati, and Aldo Rossi, “A haptic system for robotic assisted spine surgery,” in Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005.   Toronto, Canada: IEEE, 2005, pp. 19–24.
  10. J. Lee, I. Hwang, K. Kim, S. Choi, W. Kyun Chung, and Y. Soo Kim, “Cooperative robotic assistant with drill-by-wire end-effector for spinal fusion surgery,” Industrial Robot: An International Journal, vol. 36, no. 1, pp. 60–72, Jan. 2009.
  11. N. Yilmaz, B. Burkhart, A. Deguet, P. Kazanzides, and U. Tumerdem, “Sensorless transparency optimized haptic teleoperation on the da Vinci research kit,” IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 971–978, Feb. 2024.
  12. L. Qian, J. Y. Wu, S. P. DiMaio, N. Navab, and P. Kazanzides, “A review of augmented reality in robotic-assisted surgery,” IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 1, pp. 1–16, Feb. 2020.
  13. R. Suzuki, A. Karim, T. Xia, H. Hedayati, and N. Marquardt, “Augmented reality and robotics: A survey and taxonomy for AR-enhanced human-robot interaction and robotic interfaces,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems.   New Orleans, LA, USA: ACM, Apr. 2022, pp. 1–33.
  14. H. Iqbal, F. Tatti, and F. Rodriguez Y Baena, “Augmented reality in robotic assisted orthopaedic surgery: A pilot study,” Journal of Biomedical Informatics, vol. 120, p. 103841, Aug. 2021.
  15. P. Tu, C. Qin, Y. Guo, D. Li, A. J. Lungu, H. Wang, and X. Chen, “Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement,” IEEE Transactions on Biomedical Engineering, vol. 69, no. 8, pp. 2593–2603, Aug. 2022.
  16. J. Schreiter, D. Schott, L. Schwenderling, C. Hansen, F. Heinrich, and F. Joeres, “AR-supported supervision of conditional autonomous robots: Considerations for pedicle screw placement in the future,” Journal of Imaging, vol. 8, no. 10, p. 255, Sept. 2022.
  17. M. Selvaggio, M. Cognetti, S. Nikolaidis, S. Ivaldi, and B. Siciliano, “Autonomy in physical human-robot interaction: A brief survey,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7989–7996, Oct. 2021.
  18. D. P. Losey, C. G. McDonald, E. Battaglia, and M. K. O’Malley, “A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction,” Applied Mechanics Reviews, vol. 70, no. 1, p. 010804, Jan. 2018.
  19. A. E. Abdelaal, P. Mathur, and S. E. Salcudean, “Robotics in vivo: A perspective on human–robot interaction in surgical robotics,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, no. 1, pp. 221–242, May 2020.
  20. A. Attanasio, B. Scaglioni, E. De Momi, P. Fiorini, and P. Valdastri, “Autonomy in surgical robotics,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, no. 1, pp. 651–679, May 2021.
  21. S. Javdani, H. Admoni, S. Pellegrinelli, S. S. Srinivasa, and J. A. Bagnell, “Shared autonomy via hindsight optimization for teleoperation and teaming,” The International Journal of Robotics Research, vol. 37, no. 7, pp. 717–742, June 2018.
  22. A. D. Dragan and S. S. Srinivasa, “A policy-blending formalism for shared control,” The International Journal of Robotics Research, vol. 32, no. 7, pp. 790–805, June 2013.
  23. G. Hoffman, T. Bhattacharjee, and S. Nikolaidis, “Inferring human intent and predicting human action in human–robot collaboration,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 7, no. 1, May 2024.
  24. Q. Wang, D. Liu, M. G. Carmichael, S. Aldini, and C.-T. Lin, “Computational model of robot trust in human co-worker for physical human-robot collaboration,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3146–3153, Apr. 2022.
  25. Z. Prasov, “Eye gaze for reference resolution in multimodal conversational interfaces,” Ph.D. dissertation, Michigan State University, 2011.
  26. H. Admoni and B. Scassellati, “Social eye gaze in human-robot interaction: A review,” Journal of Human-Robot Interaction, vol. 6, no. 1, p. 25, Mar. 2017.
  27. M. S. H. Sunny, M. I. I. Zarif, I. Rulik, J. Sanjuan, M. H. Rahman, S. I. Ahamed, I. Wang, K. Schultz, and B. Brahmi, “Eye-gaze control of a wheelchair mounted 6DOF assistive robot for activities of daily living,” Journal of NeuroEngineering and Rehabilitation, vol. 18, no. 1, p. 173, Dec. 2021.
  28. O. Palinko, F. Rea, G. Sandini, and A. Sciutti, “Robot reading human gaze: Why eye tracking is better than head tracking for human-robot collaboration,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   Daejeon, South Korea: IEEE, Oct. 2016, pp. 5048–5054.
  29. R. M. Aronson, T. Santini, T. C. Kübler, E. Kasneci, S. Srinivasa, and H. Admoni, “Eye-hand behavior in human-robot shared manipulation,” in Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction.   Chicago, IL, USA: ACM, Feb. 2018, pp. 4–13.
  30. R. M. Aronson and H. Admoni, “Gaze complements control input for goal prediction during assisted teleoperation,” in Proceedings of Robotics: Science and Systems, New York City, NY, USA, June 2022.
  31. R. M. Aronson, N. Almutlak, and H. Admoni, “Inferring goals with gaze during teleoperated manipulation,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   Prague, Czech Republic: IEEE, Sept. 2021, pp. 7307–7314.
  32. E. Tafaj, G. Kasneci, W. Rosenstiel, and M. Bogdan, “Bayesian online clustering of eye movement data,” in Proceedings of the Symposium on Eye Tracking Research and Applications.   Santa Barbara, CA, USA: ACM, Mar. 2012, pp. 285–288.
  33. K. M. Lynch and F. C. Park, “Robot control,” in Modern Robotics: Mechanics, Planning, and Control.   Cambridge, UK: Cambridge University Press, 2017, pp. 403–460.
  34. S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall, “Robot Operating System 2: Design, architecture, and uses in the wild,” Science Robotics, vol. 7, no. 66, p. eabm6074, May 2022.
  35. l. A. Sucan and S. Chitta, “MoveIt.” [Online]. Available: moveit.ros.org
  36. S. Chitta, E. Marder-Eppstein, W. Meeussen, V. Pradeep, A. R. Tsouroukdissian, J. Bohren, D. Coleman, B. Magyar, G. Raiola, M. Lüdtke, and E. F. Perdomo, “ros_control: A generic and simple control framework for ROS,” Journal of Open Source Software, vol. 2, no. 20, Dec. 2017.
  37. Abraham. Savitzky and M. J. E. Golay, “Smoothing and differentiation of data by simplified least squares procedures,” Analytical Chemistry, vol. 36, no. 8, pp. 1627–1639, July 1964.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets