Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Autonomous Path Planning for Intercostal Robotic Ultrasound Imaging Using Reinforcement Learning (2404.09927v1)

Published 15 Apr 2024 in cs.RO and cs.LG

Abstract: Ultrasound (US) has been widely used in daily clinical practice for screening internal organs and guiding interventions. However, due to the acoustic shadow cast by the subcutaneous rib cage, the US examination for thoracic application is still challenging. To fully cover and reconstruct the region of interest in US for diagnosis, an intercostal scanning path is necessary. To tackle this challenge, we present a reinforcement learning (RL) approach for planning scanning paths between ribs to monitor changes in lesions on internal organs, such as the liver and heart, which are covered by rib cages. Structured anatomical information of the human skeleton is crucial for planning these intercostal paths. To obtain such anatomical insight, an RL agent is trained in a virtual environment constructed using computational tomography (CT) templates with randomly initialized tumors of various shapes and locations. In addition, task-specific state representation and reward functions are introduced to ensure the convergence of the training process while minimizing the effects of acoustic attenuation and shadows during scanning. To validate the effectiveness of the proposed approach, experiments have been carried out on unseen CTs with randomly defined single or multiple scanning targets. The results demonstrate the efficiency of the proposed RL framework in planning non-shadowed US scanning trajectories in areas with limited acoustic access.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. M. C. Fiorentino, E. Cipolletta, E. Filippucci, W. Grassi, E. Frontoni, and S. Moccia, “A deep-learning framework for metacarpal-head cartilage-thickness estimation in ultrasound rheumatological images,” Computers in Biology and Medicine, vol. 141, p. 105117, 2022.
  2. A. S. B. Mustafa, T. Ishii, Y. Matsunaga, R. Nakadate, H. Ishii, K. Ogawa, A. Saito, M. Sugawara, K. Niki, and A. Takanishi, “Development of robotic system for autonomous liver screening using ultrasound scanning device,” in 2013 ROBIO.   IEEE, 2013, pp. 804–809.
  3. Z. Jiang, Y. Gao, L. Xie, and N. Navab, “Towards autonomous atlas-based ultrasound acquisitions in presence of articulated motion,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7423–7430, 2022.
  4. S. Ipsen, D. Wulff, I. Kuhlemann, A. Schweikard, and F. Ernst, “Towards automated ultrasound imaging—robotic image acquisition in liver and prostate for long-term motion monitoring,” Physics in Medicine & Biology, vol. 66, no. 9, p. 094002, 2021.
  5. W. Si, N. Wang, and C. Yang, “Design and quantitative assessment of teleoperation-based human–robot collaboration method for robot-assisted sonography,” IEEE Trans. Autom. Sci. Eng., 2024.
  6. M. Dyck, A. Weld, J. Klodmann, A. Kirst, L. Dixon, G. Anichini, S. Camp, A. Albu-Schäffer, and S. Giannarou, “Towards safe and collaborative robotic ultrasound tissue scanning in neurosurgery,” IEEE Transactions on Medical Robotics and Bionics, 2024.
  7. Z. Jiang, S. E. Salcudean, and N. Navab, “Robotic ultrasound imaging: State-of-the-art and future perspectives,” Medical image analysis, p. 102878, 2023.
  8. K. Li, Y. Xu, and M. Q.-H. Meng, “An overview of systems and techniques for autonomous robotic ultrasound acquisitions,” IEEE Transactions on Medical Robotics and Bionics, vol. 3, no. 2, pp. 510–524, 2021.
  9. Z. Jiang, M. Grimm, M. Zhou, Y. Hu, J. Esteban, and N. Navab, “Automatic force-based probe positioning for precise robotic ultrasound acquisition,” IEEE Transactions on Industrial Electronics, vol. 68, no. 11, pp. 11 200–11 211, 2020.
  10. M. Akbari, J. Carriere, R. Sloboda, T. Meyer, N. Usmani, S. Husain, and M. Tavakoli, “Robot-assisted breast ultrasound scanning using geometrical analysis of the seroma and image segmentation,” in 2021 IROS.   IEEE, 2021, pp. 3784–3791.
  11. Q. Huang, B. Gao, and M. Wang, “Robot-assisted autonomous ultrasound imaging for carotid artery,” IEEE Transactions on Instrumentation and Measurement, 2024.
  12. Y. Huang, W. Xiao, C. Wang, H. Liu, R. Huang, and Z. Sun, “Towards fully autonomous ultrasound scanning robot with imitation learning based on clinical protocols,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3671–3678, 2021.
  13. A. Bal, A. Gupta, F. Abhimanyu, J. Galeotti, and H. Choset, “A curvature and trajectory optimization-based 3d surface reconstruction pipeline for ultrasound trajectory generation,” in ICRA.   IEEE, 2023, pp. 2724–2730.
  14. J. Tan, J. Li, Y. Li, B. Li, Y. Leng, Y. Rong, and C. Fu, “Autonomous trajectory planning for ultrasound-guided real-time tracking of suspicious breast tumor targets,” IEEE Trans. Autom. Sci. Eng., 2023.
  15. R. Göbl, S. Virga, J. Rackerseder, B. Frisch, N. Navab, and C. Hennersperger, “Acoustic window planning for ultrasound acquisition,” International Journal of Computer Assisted Radiology and Surgery, vol. 12, pp. 993–1001, 2017.
  16. K. Li, Y. Xu, J. Wang, D. Ni, L. Liu, and M. Q.-H. Meng, “Image-guided navigation of a robotic ultrasound probe for autonomous spinal sonography using a shadow-aware dual-agent framework,” IEEE Trans. Med. Robot. Bionics, vol. 4, no. 1, pp. 130–144, 2021.
  17. Y. Bi, Z. Jiang, Y. Gao, T. Wendler, A. Karlas, and N. Navab, “Vesnet-rl: Simulation-based reinforcement learning for real-world us probe navigation,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6638–6645, 2022.
  18. H. Hase, M. F. Azampour, M. Tirindelli, M. Paschali, W. Simson, E. Fatemizadeh, and N. Navab, “Ultrasound-guided robotic navigation with deep reinforcement learning,” in 2020 IROS.   IEEE, 2020, pp. 5534–5541.
  19. X. Deng, Y. Chen, F. Chen, and M. Li, “Learning robotic ultrasound scanning skills via human demonstrations and guided explorations,” in ROBIO.   IEEE, 2021, pp. 372–378.
  20. L. Soler, A. Hostettler, V. Agnus, A. Charnoz, J. Fasquel, J. Moreau, A. Osswald, M. Bouhadjar, and J. Marescaux, “3d image reconstruction for comparison of algorithm database: A patient specific anatomical and medical image database,” IRCAD, Strasbourg, France, Tech. Rep, vol. 1, no. 1, 2010.
  21. Z. Jiang, X. Li, C. Zhang, Y. Bi, W. Stechele, and N. Navab, “Skeleton graph-based ultrasound-ct non-rigid registration,” IEEE Robotics and Automation Letters, 2023.
  22. Z. Jiang, C. Li, X. Lil, and N. Navab, “Thoracic cartilage ultrasound-ct registration using dense skeleton graph,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2023, pp. 6586–6592.
  23. Z. Wang, B. Zhao, P. Zhang, L. Yao, Q. Wang, B. Li, M. Q.-H. Meng, and Y. Hu, “Full-coverage path planning and stable interaction control for automated robotic breast ultrasound scanning,” IEEE Transactions on Industrial Electronics, vol. 70, no. 7, pp. 7051–7061, 2022.
  24. C. Yang, M. Jiang, M. Chen, M. Fu, J. Li, and Q. Huang, “Automatic 3-d imaging and measurement of human spines with a robotic ultrasound system,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–13, 2021.
  25. C. Hennersperger, B. Fuerst, S. Virga, O. Zettinig, B. Frisch, T. Neff, and N. Navab, “Towards mri-based autonomous robotic us acquisitions: a first feasibility study,” IEEE transactions on medical imaging, vol. 36, no. 2, pp. 538–548, 2016.
  26. L. Al-Zogbi, V. Singh, B. Teixeira, A. Ahuja, P. S. Bagherzadeh, A. Kapoor, H. Saeidi, T. Fleiter, and A. Krieger, “Autonomous robotic point-of-care ultrasound imaging for monitoring of covid-19–induced pulmonary diseases,” Frontiers in Robotics and AI, vol. 8, p. 645756, 2021.
  27. Y. Shida, S. Kumagai, R. Tsumura, and H. Iwata, “Automated image acquisition of parasternal long-axis view with robotic echocardiography,” IEEE Robotics and Automation Letters, 2023.
  28. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015.
  29. X. Yang, Y. Huang, R. Huang, H. Dou, R. Li, J. Qian, X. Huang, W. Shi, C. Chen, Y. Zhang et al., “Searching collaborative agents for multi-plane localization in 3d ultrasound,” Medical Image Analysis, vol. 72, p. 102119, 2021.
  30. A. Alansary, O. Oktay, Y. Li, L. Le Folgoc, B. Hou, G. Vaillant, K. Kamnitsas, A. Vlontzos, B. Glocker, B. Kainz et al., “Evaluating reinforcement learning agents for anatomical landmark detection,” Medical image analysis, vol. 53, pp. 156–164, 2019.
  31. F. Sahba, H. R. Tizhoosh, and M. M. Salama, “Application of reinforcement learning for segmentation of transrectal ultrasound images,” BMC medical imaging, vol. 8, pp. 1–10, 2008.
  32. T. Liu, Q. Meng, A. Vlontzos, J. Tan, D. Rueckert, and B. Kainz, “Ultrasound video summarization using deep reinforcement learning,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23.   Springer, 2020, pp. 483–492.
  33. G. Ning, X. Zhang, and H. Liao, “Autonomic robotic ultrasound imaging system based on reinforcement learning,” IEEE Transactions on Biomedical Engineering, vol. 68, no. 9, pp. 2787–2797, 2021.
  34. K. Li, A. Li, Y. Xu, H. Xiong, and M. Q.-H. Meng, “Rl-tee: Autonomous probe guidance for transesophageal echocardiography based on attention-augmented deep reinforcement learning,” IEEE Trans. Autom. Sci. Eng., 2023.
  35. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
  36. H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016.
  37. Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in International conference on machine learning.   PMLR, 2016, pp. 1995–2003.
  38. J. Staal, B. van Ginneken, and M. A. Viergever, “Automatic rib segmentation and labeling in computed tomography scans using a general framework for detection, recognition and segmentation of objects in volumetric data,” Medical image analysis, vol. 11, no. 1, pp. 35–46, 2007.
  39. M. Salehi, S.-A. Ahmadi, R. Prevost, N. Navab, and W. Wein, “Patient-specific 3d ultrasound simulation based on convolutional ray-tracing and appearance optimization,” in MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part II 18.   Springer, 2015, pp. 510–518.
  40. T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” arXiv preprint arXiv:1511.05952, 2015.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now