Motion Informed Needle Segmentation in Ultrasound Images (2312.01239v3)
Abstract: Segmenting a moving needle in ultrasound images is challenging due to the presence of artifacts, noise, and needle occlusion. This task becomes even more demanding in scenarios where data availability is limited. In this paper, we present a novel approach for needle segmentation for 2D ultrasound that combines classical Kalman Filter (KF) techniques with data-driven learning, incorporating both needle features and needle motion. Our method offers three key contributions. First, we propose a compatible framework that seamlessly integrates into commonly used encoder-decoder style architectures. Second, we demonstrate superior performance compared to recent state-of-the-art needle segmentation models using our novel convolutional neural network (CNN) based KF-inspired block, achieving a 15\% reduction in pixel-wise needle tip error and an 8\% reduction in length error. Third, to our knowledge we are the first to implement a learnable filter to incorporate non-linear needle motion for improving needle segmentation.
- High accuracy optical flow estimation based on a theory for warping. In Computer Vision-ECCV 2004: 8th European Conference on Computer Vision, Prague, Czech Republic, May 11-14, 2004. Proceedings, Part IV 8, pages 25–36. Springer, 2004.
- One-shot video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 221–230, 2017.
- Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306, 2021.
- Automatic and accurate needle detection in 2d ultrasound during robot-assisted needle insertion process. International Journal of Computer Assisted Radiology and Surgery, pages 1–9, 2022.
- Flownet: Learning optical flow with convolutional networks. In Proceedings of the IEEE international conference on computer vision, pages 2758–2766, 2015.
- Siam-u-net: encoder-decoder siamese network for knee cartilage tracking in ultrasound images. Medical Image Analysis, 60:101631, 2020.
- A neural network approach for flexible needle tracking in ultrasound images using kalman filter. In 5th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, pages 70–75. IEEE, 2014.
- Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021.
- The advances in computer vision that are enabling more autonomous actions in surgery: A systematic review of the literature. Sensors, 22(13):4918, 2022.
- Automatic needle segmentation in 3d ultrasound data using a hough transform approach. In Bildverarbeitung für die Medizin 2009, pages 341–345. Springer, 2009.
- Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
- R. E. Kalman. A New Approach to Linear Filtering and Prediction Problems. Journal of Basic Engineering, 82(1):35–45, 03 1960.
- Needle localization using gabor filtering in 2d ultrasound images. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 4881–4886. IEEE, 2014.
- Ultrasound needle segmentation and trajectory prediction using excitation network. International Journal of Computer Assisted Radiology and Surgery, 15(3):437–443, 2020.
- Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
- Despeckling of medical ultrasound images. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 53(1):64–78, 2006.
- Deep learning based needle tracking in prostate fusion biopsy. In Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling, volume 11598, pages 605–613. SPIE, 2021.
- Learning needle tip localization from digital subtraction in 2d ultrasound. International journal of computer assisted radiology and surgery, 14(6):1017–1026, 2019.
- Single shot needle tip localization in 2d ultrasound. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part V, pages 637–645. Springer, 2019.
- Methods for segmenting curved needles in ultrasound images. Medical image analysis, 10(3):330–342, 2006.
- Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.
- Robust visual tracking by segmentation. In European Conference on Computer Vision, pages 571–588. Springer, 2022.
- Kalmannet: Neural network aided kalman filtering for partially known dynamics. IEEE Transactions on Signal Processing, 70:1532–1547, 2022.
- U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
- opencv/cvat: v1.1.0, Aug. 2020.
- Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems, 28, 2015.
- Needle tip tracking in 2d ultrasound based on improved compressive tracking and adaptive kalman filter. IEEE Robotics and Automation Letters, 6(2):3224–3231, 2021.
- Associating objects with transformers for video object segmentation. Advances in Neural Information Processing Systems, 34:2491–2502, 2021.