Real-time guidewire tracking and segmentation in intraoperative x-ray (2404.08805v1)
Abstract: During endovascular interventions, physicians have to perform accurate and immediate operations based on the available real-time information, such as the shape and position of guidewires observed on the fluoroscopic images, haptic information and the patients' physiological signals. For this purpose, real-time and accurate guidewire segmentation and tracking can enhance the visualization of guidewires and provide visual feedback for physicians during the intervention as well as for robot-assisted interventions. Nevertheless, this task often comes with the challenge of elongated deformable structures that present themselves with low contrast in the noisy fluoroscopic image sequences. To address these issues, a two-stage deep learning framework for real-time guidewire segmentation and tracking is proposed. In the first stage, a Yolov5s detector is trained, using the original X-ray images as well as synthetic ones, which is employed to output the bounding boxes of possible target guidewires. More importantly, a refinement module based on spatiotemporal constraints is incorporated to robustly localize the guidewire and remove false detections. In the second stage, a novel and efficient network is proposed to segment the guidewire in each detected bounding box. The network contains two major modules, namely a hessian-based enhancement embedding module and a dual self-attention module. Quantitative and qualitative evaluations on clinical intra-operative images demonstrate that the proposed approach significantly outperforms our baselines as well as the current state of the art and, in comparison, shows higher robustness to low quality images.
- Baert, S. A., Viergever, M. A., and Niessen, W. J., “Guide-wire tracking during endovascular interventions,” IEEE Transactions on Medical Imaging 22(8), 965–972 (2003).
- Mazomenos, E. B., Chang, P.-L., Rolls, A., Hawkes, D. J., Bicknell, C. D., Poorten, E. V., Riga, C. V., Desjardins, A., and Stoyanov, D., “A survey on the current status and future challenges towards objective skills assessment in endovascular surgery,” Journal of Medical Robotics Research 1(03), 1640010 (2016).
- Chen, B.-J., Wu, Z., Sun, S., Zhang, D., and Chen, T., “Guidewire tracking using a novel sequential segment optimization method in interventional x-ray videos,” in [2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) ], 103–106, IEEE (2016).
- Vandini, A., Glocker, B., Hamady, M., and Yang, G.-Z., “Robust guidewire tracking under large deformations combining segment-like features (seglets),” Medical image analysis 38, 150–164 (2017).
- Wu, Y.-D., Xie, X.-L., Bian, G.-B., Hou, Z.-G., Cheng, X.-R., Chen, S., Liu, S.-Q., and Wang, Q.-L., “Automatic guidewire tip segmentation in 2d x-ray fluoroscopy using convolution neural networks,” in [2018 International Joint Conference on Neural Networks (IJCNN) ], 1–7, IEEE (2018).
- Li, R.-Q., Bian, G., Zhou, X., Xie, X., Ni, Z., and Hou, Z., “A two-stage framework for real-time guidewire endpoint localization,” in [International Conference on Medical Image Computing and Computer-Assisted Intervention ], 357–365, Springer (2019).
- Ambrosini, P., Ruijters, D., Niessen, W. J., Moelker, A., and van Walsum, T., “Fully automatic and real-time catheter segmentation in x-ray fluoroscopy,” in [International conference on medical image computing and computer-assisted intervention ], 577–585, Springer (2017).
- Zhou, Y.-J., Xie, X.-L., Zhou, X.-H., Liu, S.-Q., Bian, G.-B., and Hou, Z.-G., “A real-time multi-functional framework for guidewire morphological and positional analysis in interventional x-ray fluoroscopy,” IEEE Transactions on Cognitive and Developmental Systems (2020).
- Jocher, G., Nishimura, K., Mineeva, T., and Vilariño, R., “Yolov5,” Code repository https://github. com/ultralytics/yolov5 (2020).
- Jerman, T., Pernuš, F., Likar, B., and Špiclin, Ž., “Enhancement of vascular structures in 3d and 2d angiographic images,” IEEE transactions on medical imaging 35(9), 2107–2118 (2016).
- Mou, L., Zhao, Y., Fu, H., Liu, Y., Cheng, J., Zheng, Y., Su, P., Yang, J., Chen, L., Frangi, A. F., et al., “Cs2-net: Deep learning segmentation of curvilinear structures in medical imaging,” Medical image analysis 67, 101874 (2021).
- Taha, A. A. and Hanbury, A., “Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool,” BMC medical imaging 15(1), 1–28 (2015).
- Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmentation,” in [International Conference on Medical image computing and computer-assisted intervention ], 234–241, Springer (2015).
- Alom, M. Z., Hasan, M., Yakopcic, C., Taha, T. M., and Asari, V. K., “Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation,” (2018).
- Zhang, Z., Liu, Q., and Wang, Y., “Road extraction by deep residual u-net,” IEEE Geoscience and Remote Sensing Letters 15(5), 749–753 (2018).