Papers
Topics
Authors
Recent
Search
2000 character limit reached

Looking Together $\neq$ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction

Published 21 Mar 2024 in cs.HC | (2403.14561v1)

Abstract: Shared gaze visualizations have been found to enhance collaboration and communication outcomes in diverse HCI scenarios including computer supported collaborative work and learning contexts. Given the importance of gaze in surgery operations, especially when a surgeon trainer and trainee need to coordinate their actions, research on the use of gaze to facilitate intra-operative coordination and instruction has been limited and shows mixed implications. We performed a field observation of 8 surgeries and an interview study with 14 surgeons to understand their visual needs during operations, informing ways to leverage and augment gaze to enhance intra-operative coordination and instruction. We found that trainees have varying needs in receiving visual guidance which are often unfulfilled by the trainers' instructions. It is critical for surgeons to control the timing of the gaze-based visualizations and effectively interpret gaze data. We suggest overlay technologies, e.g., gaze-based summaries and depth sensing, to augment raw gaze in support of surgical coordination and instruction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Gaze-enhanced Crossmodal Embeddings for Emotion Recognition. Proceedings of the ACM on Human-Computer Interaction 6, ETRA (2022), 1–18.
  2. Defining the deficit in US surgical training: the trainee’s perspective. Journal of the American College of Surgeons 232, 4 (2021), 623–627.
  3. State of the art and perspectives in catalytic processes for CO2 conversion into chemicals and fuels: The distinctive contribution of chemical catalysis and biotechnology. Journal of Catalysis 343 (2016), 2–45.
  4. Eye-tracking technology in medical education: A systematic review. Medical teacher 40, 1 (2018), 62–69.
  5. Factors Influencing Team Behaviors in Surgery: A Qualitative Study to Inform Teamwork Interventions. The Annals of Thoracic Surgery 106, 1 (2018), 115–120. https://doi.org/10.1016/j.athoracsur.2017.12.045
  6. Surgical Video Summarization: Multifarious Uses, Summarization Process and Ad-Hoc Coordination. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.
  7. Is motion capture-based biomechanical simulation valid for hci studies? study and implications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3215–3224.
  8. Poor resident-attending intraoperative communication may compromise patient safety. Journal of Surgical Research 171, 2 (2011), 386–394.
  9. Depth perception of surgeons in minimally invasive surgery. Surgical innovation 23, 5 (2016), 515–524.
  10. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
  11. Virginia Braun and Victoria Clarke. 2012. Thematic analysis. (2012).
  12. Sensing (co) operations: Articulation and compensation in the robotic operating room. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–26.
  13. Collaborative eye tracking: a potential training tool in laparoscopic surgery. Surgical endoscopy 26, 7 (2012), 2003–2009.
  14. Feasibility of Team Training in Virtual Reality for Robot-Assisted Minimally Invasive Surgery. In Proceedings of the Virtual Reality International Conference - Laval Virtual (Laval, France) (VRIC ’18). Association for Computing Machinery, New York, NY, USA, Article 8, 4 pages. https://doi.org/10.1145/3234253.3234295
  15. Richard D. Coats and Randall S. Burd. 2002. Intraoperative Communication of Residents with Faculty: Perception versus Reality. Journal of Surgical Research 104, 1 (2002), 40–45. https://doi.org/10.1006/jsre.2002.6402
  16. Making meaning from sensory cues: a qualitative investigation of postgraduate learning in the operating room. Academic Medicine 90, 8 (2015), 1125–1131.
  17. A theory-based model for teaching and assessing residents in the operating room. Journal of surgical education 70, 1 (2013), 24–30.
  18. Supervisors’ transformational leadership style and residents’ job crafting in surgical training: the residents’ views. International Journal of Medical Education 13 (03 2022), 74–83. https://doi.org/10.5116/ijme.622d.e2f6
  19. Comparison of surgeon gaze behaviour against objective skill assessment in laparoscopic cholecystectomy-a prospective cohort study. International Journal of Surgery 82 (2020), 149–155.
  20. Communication cost of single-user gesturing tool in laparoscopic surgical training. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
  21. Communication Cost of Single-User Gesturing Tool in Laparoscopic Surgical Training. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300841
  22. Virtual pointer for gaze guidance in laparoscopic surgery. Surgical endoscopy 34, 8 (2020), 3533–3539.
  23. Learning With Pedagogical Models: Videos As Adjuncts to Apprenticeship for Surgical Training. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–40.
  24. Resident training in a teaching hospital: how do attendings teach in the real operative environment? The American Journal of Surgery 214, 1 (2017), 141–146.
  25. Chronicle: capture, exploration, and playback of document workflow histories. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. 143–152.
  26. Assessing Surgeons’ Skill Level in Laparoscopic Cholecystectomy Using Eye Metrics (ETRA ’19). Association for Computing Machinery, New York, NY, USA, Article 30, 8 pages. https://doi.org/10.1145/3314111.3319832
  27. In the flow, being heard, and having opportunities: Sources of power and power dynamics in global teams. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 864–875.
  28. Protecting patients from an unsafe system: the etiology and recovery of intraoperative deviations in care. Annals of surgery 256, 2 (2012), 203–210.
  29. Surgeons’ leadership styles and team behavior in the operating room. Journal of the American College of Surgeons 222, 1 (2016), 41–51.
  30. Identifying Collaborative Learning States Using Unsupervised Machine Learning on Eye-Tracking, Physiological and Motion Sensor Data. International Educational Data Mining Society (2019).
  31. The Impact of Sharing Gaze Behaviours in Collaborative Mixed Reality. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1–27.
  32. Development of Virtual Reality Based Robotic Surgical Trainer for Patient-Specific Deformable Anatomy. In Proceedings of the 2015 Conference on Advances In Robotics (Goa, India) (AIR ’15). Association for Computing Machinery, New York, NY, USA, Article 16, 5 pages. https://doi.org/10.1145/2783449.2783465
  33. Spatial ability, experience, and skill in laparoscopic surgery. The American Journal of Surgery 188, 1 (2004), 71–75.
  34. Effects of shared gaze on audio-versus text-based remote collaborations. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–25.
  35. Michael Lankes and Argenis Ramirez Gomez. 2022. GazeCues: Exploring the Effects of Gaze-based Visual Cues in Virtual Reality Exploration Games. Proceedings of the ACM on Human-Computer Interaction 6, CHI PLAY (2022), 1–25.
  36. Evaluation of Game Engines for Simulated Surgical Training. In Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia (Perth, Australia) (GRAPHITE ’07). Association for Computing Machinery, New York, NY, USA, 273–280. https://doi.org/10.1145/1321261.1321311
  37. Swifter: improved online video scrubbing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1159–1168.
  38. A surgical scene replay system for learning gastroenterological endoscopic surgery skill by multiple synchronized-video and gaze representation. Proceedings of the ACM on Human-Computer Interaction 5, EICS (2021), 1–22.
  39. A Surgical Scene Replay System for Learning Gastroenterological Endoscopic Surgery Skill by Multiple Synchronized-Video and Gaze Representation. Proc. ACM Hum.-Comput. Interact. 5, EICS, Article 204 (may 2021), 22 pages. https://doi.org/10.1145/3461726
  40. Learning to see the body: supporting instructional practices in laparoscopic surgical procedures. In Proceedings of the SIGCHI conference on human factors in computing systems. 2113–2122.
  41. Remotely Shaping the View in Surgical Telementoring. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376622
  42. Lorenza Mondada. 2016. Operating together: The collective achievement of surgical action. (2016).
  43. Jan Nikodem and Ryszard Klempous. 2015. Novel Force Feedback Interface for Improving Human Perception in Laparoscopic Surgical Trainer (MSM ’15). Society for Computer Simulation International, San Diego, CA, USA, 41–46.
  44. Eye-based Continuous Affect Prediction. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 137–143.
  45. Phillip M. Pierorazio and Mohamad E. Allaf. 2009. Minimally invasive surgical training: Challenges and solutions. Urologic Oncology: Seminars and Original Investigations 27, 2 (2009), 208–213. https://doi.org/10.1016/j.urolonc.2008.09.017 Training the Urologic Oncologist of the Future: Where are the Challenges?
  46. The eye in extended reality: A survey on gaze interaction and eye tracking in head-worn extended reality. ACM Computing Surveys (CSUR) 55, 3 (2022), 1–39.
  47. Ethical recommendations for video recording in the operating room. Annals of Surgery 264, 1 (2016), 34–35.
  48. Collaboration within the Surgical Suite: BoardProbe Design for and with the Surgical Team. In Actes de La 28ième Conference Francophone Sur l’Interaction Homme-Machine (Fribourg, Switzerland) (IHM ’16). Association for Computing Machinery, New York, NY, USA, 271–277. https://doi.org/10.1145/3004107.3004138
  49. Eye metrics as an objective assessment of surgical skill. Annals of surgery 252, 1 (2010), 177–182.
  50. A Systematic Review of Human–Computer Interaction (HCI) Research in Medical and Other Engineering Fields. International Journal of Human–Computer Interaction (2022), 1–22.
  51. OpTrust: validity of a tool assessing intraoperative entrustment behaviors. Annals of surgery 267, 4 (2018), 670–676.
  52. Minimally invasive surgical technologies: Challenges in education and training. Asian Journal of Endoscopic Surgery 3, 3 (2010), 101–108.
  53. Bertrand Schneider and Roy Pea. 2017. Real-time mutual gaze perception enhances collaborative learning and collaboration quality. In Educational media and technology yearbook. Springer, 99–125.
  54. Philip Sedgwick and Nan Greenwood. 2015. Understanding the Hawthorne effect. Bmj 351 (2015).
  55. How Trainees Use the Information from Telepointers in Remote Instruction. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 93 (nov 2019), 20 pages. https://doi.org/10.1145/3359195
  56. Interpretation of three-dimensional structure from two-dimensional endovascular images: implications for educators in vascular surgery. Journal of Vascular Surgery 39, 6 (2004), 1305–1311.
  57. Learners learn more and instructors track better with real-time gaze sharing. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.
  58. Minimally invasive surgical technologies: Challenges in education and training. Asian Journal of Endoscopic Surgery 3, 3 (2010), 101–108. https://doi.org/10.1111/j.1758-5910.2010.00050.x arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1758-5910.2010.00050.x
  59. AB Tobii. 2016. Tobii pro glasses 2.
  60. Data Source Triangulation. 2014. The use of triangulation in qualitative research. In Oncol Nurs Forum, Vol. 41. 545–7.
  61. Towards understanding collaboration around interactive surfaces: Exploring joint visual attention. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 219–220.
  62. Roel Vertegaal. 1999. The GAZE groupware system: mediating joint attention in multiparty communication and collaboration. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 294–301.
  63. Maureen Villamor and Ma Mercedes Rodrigo. 2018. Predicting successful collaboration in a pair programming eye tracking experiment. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization. 263–268.
  64. Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, Hawaii, USA) (CHI ’24). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3613904.3642587
  65. Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Annals of surgery 237, 4 (2003), 460.
  66. ARTEMIS: Mixed-Reality Environment for Immersive Surgical Telementoring. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–4. https://doi.org/10.1145/3334480.3383169
  67. Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts. Surgical endoscopy 24, 10 (2010), 2458–2464.
  68. Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: a randomized, controlled study. Surgical endoscopy 25, 12 (2011), 3731–3739.
  69. A van Wynsberghe and C Gastmans. 2008. Telesurgery: an ethical appraisal. Journal of Medical Ethics 34, 10 (2008), e22–e22. https://doi.org/10.1136/jme.2007.023952 arXiv:https://jme.bmj.com/content/34/10/e22.full.pdf
  70. Learning in multimodal training: Visual guidance can be both appealing and disadvantageous in spatial tasks. International journal of human-computer studies 69, 3 (2011), 113–122.
  71. Revealing Team Cognition from Dual Eye-Tracking in the Surgical Setting. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (Charleston, South Carolina) (ETRA ’16). Association for Computing Machinery, New York, NY, USA, 321–322. https://doi.org/10.1145/2857491.2884062
  72. Bringing Patient Mannequins to Life: 3D Projection Enhances Nursing Simulation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 565, 15 pages. https://doi.org/10.1145/3491102.3517562
  73. What Does Shared Understanding in Students’ Face-to-Face Collaborative Learning Gaze Behaviours “Look Like”?. In International Conference on Artificial Intelligence in Education. Springer, 588–593.
  74. Gaze following requires early visual experience. Proceedings of the National Academy of Sciences 119, 20 (2022), e2117184119.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 6 tweets with 69 likes about this paper.

Sign Up for Free