Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BronchoCopilot: Towards Autonomous Robotic Bronchoscopy via Multimodal Reinforcement Learning (2403.01483v1)

Published 3 Mar 2024 in cs.RO

Abstract: Bronchoscopy plays a significant role in the early diagnosis and treatment of lung diseases. This process demands physicians to maneuver the flexible endoscope for reaching distal lesions, particularly requiring substantial expertise when examining the airways of the upper lung lobe. With the development of artificial intelligence and robotics, reinforcement learning (RL) method has been applied to the manipulation of interventional surgical robots. However, unlike human physicians who utilize multimodal information, most of the current RL methods rely on a single modality, limiting their performance. In this paper, we propose BronchoCopilot, a multimodal RL agent designed to acquire manipulation skills for autonomous bronchoscopy. BronchoCopilot specifically integrates images from the bronchoscope camera and estimated robot poses, aiming for a higher success rate within challenging airway environment. We employ auxiliary reconstruction tasks to compress multimodal data and utilize attention mechanisms to achieve an efficient latent representation of this data, serving as input for the RL module. This framework adopts a stepwise training and fine-tuning approach to mitigate the challenges of training difficulty. Our evaluation in the realistic simulation environment reveals that BronchoCopilot, by effectively harnessing multimodal information, attains a success rate of approximately 90\% in fifth generation airways with consistent movements. Additionally, it demonstrates a robust capacity to adapt to diverse cases.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. G. J. Criner, R. Eberhardt, S. Fernandez-Bussy, D. Gompelmann, F. Maldonado, N. Patel, P. L. Shah, D.-J. Slebos, A. Valipour, M. M. Wahidi, et al., “Interventional bronchoscopy,” American journal of respiratory and critical care medicine, vol. 202, no. 1, pp. 29–50, 2020.
  2. R. J. Miller, R. F. Casal, D. R. Lazarus, D. E. Ost, and G. A. Eapen, “Flexible bronchoscopy,” Clinics in Chest Medicine, vol. 39, no. 1, pp. 1–16, 2018.
  3. A. Agrawal, D. K. Hogarth, and S. Murgu, “Robotic bronchoscopy for pulmonary lesions: a review of existing technologies and clinical data,” Journal of thoracic disease, vol. 12, no. 6, p. 3279, 2020.
  4. M. Davoudi and H. G. Colt, “Bronchoscopy simulation: a brief review,” Advances in health sciences education, vol. 14, pp. 287–296, 2009.
  5. J. Hoelscher, M. Fu, I. Fried, M. Emerson, T. E. Ertop, M. Rox, A. Kuntz, J. A. Akulian, R. J. Webster III, and R. Alterovitz, “Backward planning for a multi-stage steerable needle lung robot,” IEEE robotics and automation letters, vol. 6, no. 2, pp. 3987–3994, 2021.
  6. J. Rosell, A. Pérez, P. Cabras, and A. Rosell, “Motion planning for the virtual bronchoscopy,” in 2012 IEEE International Conference on Robotics and Automation, pp. 2932–2937, IEEE, 2012.
  7. J. Sganga, D. Eng, C. Graetzel, and D. B. Camarillo, “Autonomous driving in the lung using deep learning for localization,” arXiv preprint arXiv:1907.08136, 2019.
  8. J. Zhang, L. Liu, P. Xiang, Q. Fang, X. Nie, H. Ma, J. Hu, R. Xiong, Y. Wang, and H. Lu, “Ai co-pilot bronchoscope robot,” Nature communications, vol. 15, no. 1, p. 241, 2024.
  9. X. Tan, C.-B. Chng, Y. Su, K.-B. Lim, and C.-K. Chui, “Robot-assisted training in laparoscopy using deep reinforcement learning,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 485–492, 2019.
  10. J. Kweon, K. Kim, C. Lee, H. Kwon, J. Park, K. Song, Y. I. Kim, J. Park, I. Back, J.-H. Roh, et al., “Deep reinforcement learning for guidewire navigation in coronary artery phantom,” IEEE Access, vol. 9, pp. 166409–166422, 2021.
  11. A. Segato, L. Sestini, A. Castellano, and E. De Momi, “Ga3c reinforcement learning for surgical steerable catheter path planning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 2429–2435, IEEE, 2020.
  12. L. Karstensen, J. Ritter, J. Hatzl, T. Pätz, J. Langejürgen, C. Uhl, and F. Mathis-Ullrich, “Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver,” International Journal of Computer Assisted Radiology and Surgery, vol. 17, no. 11, pp. 2033–2040, 2022.
  13. S. Athiniotis, R. Srivatsan, and H. Choset, “Deep q reinforcement learning for autonomous navigation of surgical snake robot in confined spaces,” in Proceedings of the The Hamlyn Symposium on Medical Robotics, London, UK, pp. 23–26, 2019.
  14. D. Ramachandram and G. W. Taylor, “Deep multimodal learning: A survey on recent advances and trends,” IEEE signal processing magazine, vol. 34, no. 6, pp. 96–108, 2017.
  15. J. Ma, F. Wu, Y. Chen, X. Ji, and Y. Ding, “Effective multimodal reinforcement learning with modality alignment and importance enhancement,” arXiv preprint arXiv:2302.09318, 2023.
  16. M. A. Lee, Y. Zhu, P. Zachares, M. Tan, K. Srinivasan, S. Savarese, L. Fei-Fei, A. Garg, and J. Bohg, “Making sense of vision and touch: Learning multimodal representations for contact-rich tasks,” IEEE Transactions on Robotics, vol. 36, no. 3, pp. 582–596, 2020.
  17. J. Chen, M. Chen, Q. Zhao, S. Wang, Y. Wang, Y. Xiao, J. Hu, D. T. M. Chan, K. T. L. Yeung, D. Y. C. Chan, et al., “Design and visual servoing control of a hybrid dual-segment flexible neurosurgical robot for intraventricular biopsy,” arXiv preprint arXiv:2402.09679, 2024.
  18. K. Cleary, A. Melzer, V. Watson, G. Kronreif, and D. Stoianovici, “Interventional robotic systems: applications and technology state-of-the-art,” Minimally Invasive Therapy & Allied Technologies, vol. 15, no. 2, pp. 101–113, 2006.
  19. M. Yip and N. Das, “Robot autonomy for surgery,” in The Encyclopedia of MEDICAL ROBOTICS: Volume 1 Minimally Invasive Surgical Robotics, pp. 281–313, World Scientific, 2019.
  20. A. Segato, M. Di Marzo, S. Zucchelli, S. Galvan, R. Secoli, and E. De Momi, “Inverse reinforcement learning intra-operative path planning for steerable needle,” IEEE Transactions on Biomedical Engineering, vol. 69, no. 6, pp. 1995–2005, 2021.
  21. Y. Wang, J. Wang, Y. Li, T. Yang, and C. Ren, “The deep reinforcement learning-based vr training system with haptic guidance for catheterization skill transfer,” IEEE Sensors Journal, vol. 22, no. 23, pp. 23356–23366, 2022.
  22. A. Pore, Z. Li, D. Dall’Alba, A. Hernansanz, E. De Momi, A. Menciassi, A. C. Gelpí, J. Dankelman, P. Fiorini, and E. Vander Poorten, “Autonomous navigation for robot-assisted intraluminal and endovascular procedures: A systematic review,” IEEE Transactions on Robotics, 2023.
  23. A. Kuntz, L. G. Torres, R. H. Feins, R. J. Webster, and R. Alterovitz, “Motion planning for a three-stage multilumen transoral lung access system,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3255–3261, IEEE, 2015.
  24. R. Khare, R. Bascom, and W. E. Higgins, “Hands-free system for bronchoscopy planning and guidance,” IEEE Transactions on Biomedical Engineering, vol. 62, no. 12, pp. 2794–2811, 2015.
  25. C. F. Graetzel, A. Sheehy, and D. P. Noonan, “Robotic bronchoscopy drive mode of the auris monarch platform,” in 2019 International Conference on Robotics and Automation (ICRA), pp. 3895–3901, IEEE, 2019.
  26. Y. Huang, C. Du, Z. Xue, X. Chen, H. Zhao, and L. Huang, “What makes multi-modal learning better than single (provably),” Advances in Neural Information Processing Systems, vol. 34, pp. 10944–10956, 2021.
  27. S. Lacey and K. Sathian, “Crossmodal and multisensory interactions between vision and touch,” in Scholarpedia of touch, pp. 301–315, Springer, 2015.
  28. G.-H. Liu, A. Siravuru, S. Prabhakar, M. Veloso, and G. Kantor, “Learning end-to-end multimodal sensor policies for autonomous navigation,” in Conference on Robot Learning, pp. 249–261, PMLR, 2017.
  29. D. S. Chaplot, L. Lee, R. Salakhutdinov, D. Parikh, and D. Batra, “Embodied multimodal multitask learning,” arXiv preprint arXiv:1902.01385, 2019.
  30. D. Misra, J. Langford, and Y. Artzi, “Mapping instructions and visual observations to actions with reinforcement learning,” arXiv preprint arXiv:1704.08795, 2017.
  31. J. Hansen, F. Hogan, D. Rivkin, D. Meger, M. Jenkin, and G. Dudek, “Visuotactile-rl: learning multimodal manipulation policies with deep reinforcement learning,” in 2022 International Conference on Robotics and Automation (ICRA), pp. 8298–8304, IEEE, 2022.
  32. S. Omidshafiei, D.-K. Kim, J. Pazis, and J. P. How, “Crossmodal attentive skill learner,” arXiv preprint arXiv:1711.10314, 2017.
  33. A. Manchin, E. Abbasnejad, and A. Van Den Hengel, “Reinforcement learning with attention that works: A self-supervised approach,” in Neural Information Processing: 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12–15, 2019, Proceedings, Part V 26, pp. 223–230, Springer, 2019.
  34. H. Zheng, Y. Qin, Y. Gu, F. Xie, J. Yang, J. Sun, and G.-Z. Yang, “Alleviating class-wise gradient imbalance for pulmonary airway segmentation,” IEEE transactions on medical imaging, vol. 40, no. 9, pp. 2452–2462, 2021.
  35. Pearson Prentice Hall Upper Saddle River, NJ:, 2004.
  36. R. Izzo, D. Steinman, S. Manini, and L. Antiga, “The vascular modeling toolkit: a python library for the analysis of tubular structures in medical images,” Journal of Open Source Software, vol. 3, no. 25, p. 745, 2018.
  37. J. Allard, S. Cotin, F. Faure, P.-J. Bensoussan, F. Poyer, C. Duriez, H. Delingette, and L. Grisoni, “Sofa-an open source framework for medical simulation,” in MMVR 15-Medicine Meets Virtual Reality, vol. 125, pp. 13–18, IOP Press, 2007.
  38. F. Faure, C. Duriez, H. Delingette, J. Allard, B. Gilles, S. Marchesseau, H. Talbot, H. Courtecuisse, G. Bousquet, I. Peterlik, et al., “Sofa: A multi-model framework for interactive physical simulation,” Soft tissue biomechanical modeling for computer assisted surgery, pp. 283–321, 2012.
  39. X. Liu, J. Chen, J. Hu, H. Chen, Y. Huang, and H. Liu, “Multi-interface strain transfer modelling for flexible endoscope shape sensing,” IEEE Robotics and Automation Letters, 2024.
  40. X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-c. Woo, “Convolutional lstm network: A machine learning approach for precipitation nowcasting,” Advances in neural information processing systems, vol. 28, 2015.
  41. M. Tschannen, O. Bachem, and M. Lucic, “Recent advances in autoencoder-based representation learning,” arXiv preprint arXiv:1812.05069, 2018.
  42. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
  43. J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High-dimensional continuous control using generalized advantage estimation,” arXiv preprint arXiv:1506.02438, 2015.
  44. P. Lo, B. Van Ginneken, J. M. Reinhardt, T. Yavarna, P. A. De Jong, B. Irving, C. Fetita, M. Ortner, R. Pinho, J. Sijbers, et al., “Extraction of airways from ct (exact’09),” IEEE Transactions on Medical Imaging, vol. 31, no. 11, pp. 2093–2107, 2012.
  45. K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision, pp. 1026–1034, 2015.
  46. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now