2000 character limit reached
A Framework For Automated Dissection Along Tissue Boundary (2310.09669v4)
Published 14 Oct 2023 in cs.RO
Abstract: Robotic surgery promises enhanced precision and adaptability over traditional surgical methods. It also offers the possibility of automating surgical interventions, resulting in reduced stress on the surgeon, better surgical outcomes, and lower costs. Cholecystectomy, the removal of the gallbladder, serves as an ideal model procedure for automation due to its distinct and well-contrasted anatomical features between the gallbladder and liver, along with standardized surgical maneuvers. Dissection is a frequently used subtask in cholecystectomy where the surgeon delivers the energy on the hook to detach the gallbladder from the liver. Hence, dissection along tissue boundaries is a good candidate for surgical automation. For the da Vinci surgical robot to perform the same procedure as a surgeon automatically, it needs to have the ability to (1) recognize and distinguish between the two different tissues (e.g. the liver and the gallbladder), (2) understand where the boundary between the two tissues is located in the 3D workspace, (3) locate the instrument tip relative to the boundary in the 3D space using visual feedback, and (4) move the instrument along the boundary. This paper presents a novel framework that addresses these challenges through AI-assisted image processing and vision-based robot control. We also present the ex-vivo evaluation of the automated procedure on chicken and pork liver specimens that demonstrates the effectiveness of the proposed framework.
- E. Ayvali, R. A. Srivatsan, L. Wang, R. Roy, N. Simaan, and H. Choset, “Using bayesian optimization to guide probing of a flexible environment for simultaneous registration and stiffness mapping,” CoRR, vol. abs/1509.05830, 2015. [Online]. Available: http://arxiv.org/abs/1509.05830
- A. Shademan, R. S. Decker, J. D. Opfermann, S. Leonard, A. Krieger, and P. C. W. Kim, “Supervised autonomous robotic soft tissue surgery,” Science Translational Medicine, vol. 8, no. 337, pp. 337ra64–337ra64, 2016. [Online]. Available: https://www.science.org/doi/abs/10.1126/scitranslmed.aad9398
- R. Jackson, V. Desai, J. Castillo, and M. Çavuşoğlu, “Needle-tissue interaction force state estimation for robotic surgical suturing,” Rep U S, vol. 2016, pp. 3659–3664, Oct 2016, pMID: 29214097; PMCID: PMC5713916.
- B. Lu, B. Li, W. Chen, Y. Jin, Z. Zhao, Q. Dou, P.-A. Heng, and Y. Liu, “Toward image-guided automated suture grasping under complex environments: A learning-enabled and optimization-based holistic framework,” IEEE Transactions on Automation Science and Engineering, vol. 19, no. 4, pp. 3794–3808, 2022.
- S. Iyer, T. Looi, and J. Drake, “A single arm, single camera system for automated suturing,” in 2013 IEEE International Conference on Robotics and Automation. IEEE, 2013, pp. 239–244.
- S. Sen, A. Garg, D. V. Gealy, S. McKinley, Y. Jen, and K. Goldberg, “Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 4178–4185.
- R. Jackson, R. Yuan, D. Chow, W. Newman, and M. Çavuşoğlu, “Real-time visual tracking of dynamic surgical suture threads,” IEEE Trans Autom Sci Eng, vol. 15, no. 3, pp. 1078–1090, Jul 2018, epub 2017 Aug 11. PMID: 29988978; PMCID: PMC6034738.
- A. Sagitov, T. Tsoy, H. Li, and E. Magid, “Automated open wound suturing: detection and planning algorithm,” Journal of Robotics, Networking and Artificial Life, vol. 5, pp. 144–148, 2018. [Online]. Available: https://doi.org/10.2991/jrnal.2018.5.2.16
- Y. Kumazu, N. Kobayashi, N. Kitamura, E. Rayan, P. Neculoiu, T. Misumi, Y. Hojo, T. Nakamura, T. Kumamoto, Y. Kurahashi, Y. Ishida, M. Masuda, and H. Shinohara, “Automated segmentation by deep learning of loose connective tissue fibers to define safe dissection planes in robot-assisted gastrectomy,” Sci Rep, vol. 11, no. 1, p. 21198, Oct 2021, pMID: 34707141; PMCID: PMC8551298.
- A. Attanasio, B. Scaglioni, E. De Momi, P. Fiorini, and P. Valdastri, “Autonomy in surgical robotics,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, no. 1, pp. 651–679, 2021. [Online]. Available: https://doi.org/10.1146/annurev-control-062420-090543
- H. Lijun, H. Wang, Z. Liu, W. Chen, and X. Zhang, “Vision-based cutting control of deformable objects with surface tracking,” IEEE/ASME Transactions on Mechatronics, vol. PP, pp. 1–1, 10 2020.
- P. Kazanzides, Z. Chen, A. Deguet, G. S. Fischer, R. H. Taylor, and S. P. DiMaio, “An open-source research kit for the da vinci surgical system,” in IEEE Intl. Conf. on Robotics and Auto. (ICRA), Hong Kong, China, 2014, pp. 6434–6439.
- S. Garrido-Jurado, R. Muñoz-Salinas, F. Madrid-Cuevas, and M. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280–2292, 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320314000235
- Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2, 2019.
- O. Özgüner, T. Shkurti, S. Huang, R. Hao, R. C. Jackson, W. S. Newman, and M. C. Çavuşoğlu, “Camera-robot calibration for the da vinci robotic surgery system,” IEEE Transactions on Automation Science and Engineering, vol. 17, no. 4, pp. 2154–2161, 2020.
- Z. Ugray, L. Lasdon, J. Plummer, F. Glover, J. Kelly, and R. Martí, “Scatter search and local nlp solvers: A multistart framework for global optimization,” INFORMS Journal on Computing, vol. 19, no. 3, pp. 328–340, 2007. [Online]. Available: https://doi.org/10.1287/ijoc.1060.0175
- A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollár, and R. Girshick, “Segment anything,” arXiv:2304.02643, 2023.
- W.-Y. Hong, C.-L. Kao, Y.-H. Kuo, J.-R. Wang, W.-L. Chang, and C.-S. Shih, “Cholecseg8k: a semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80,” arXiv preprint arXiv:2012.12453, 2020.
- C. I. Nwoye, D. Alapatt, T. Yu, A. Vardazaryan, F. Xia, Z. Zhao, T. Xia, F. Jia, Y. Yang, H. Wang et al., “Cholectriplet2021: A benchmark challenge for surgical action triplet recognition,” Medical Image Analysis, vol. 86, p. 102803, 2023.
- J. Brooks, “COCO Annotator,” https://github.com/jsbroks/coco-annotator/, 2019.
- T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollár, “Microsoft coco: Common objects in context,” 2014. [Online]. Available: https://arxiv.org/abs/1405.0312
- Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 11, pp. 1330–1334, 2000.
- G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.
- H. Hirschmuller, “Stereo processing by semiglobal matching and mutual information,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328–341, 2008.
- S. Maneewongvatana and D. M. Mount, “Analysis of approximate nearest neighbor searching with clustered point sets,” CoRR, vol. cs.CG/9901013, 1999. [Online]. Available: https://arxiv.org/abs/cs/9901013
- D. J. Rosenkrantz, R. E. Stearns, and P. M. Lewis, II, “An analysis of several heuristics for the traveling salesman problem,” SIAM journal on computing, vol. 6, no. 3, pp. 563–581, 1977.
- K.-H. Oh, L. Borgioli, A. Mangano, V. Valle, M. D. Pangrazio, F. Toti, G. Pozza, L. Ambrosini, A. Ducas, M. Zefran, L. Chen, and P. C. Giulianotti, “Comprehensive robotic cholecystectomy dataset (crcd): Integrating kinematics, pedal signals, and endoscopic videos,” 2023.
- C. Molnár, T. D. Nagy, R. N. Elek, and T. Haidegger, “Visual servoing-based camera control for the da vinci surgical system,” in 2020 IEEE 18th International Symposium on Intelligent Systems and Informatics (SISY), 2020, pp. 107–112.