Papers
Topics
Authors
Recent
Search
2000 character limit reached

BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization

Published 20 Feb 2024 in cs.CV | (2402.12763v1)

Abstract: Localizing the bronchoscope in real time is essential for ensuring intervention quality. However, most existing methods struggle to balance between speed and generalization. To address these challenges, we present BronchoTrack, an innovative real-time framework for accurate branch-level localization, encompassing lumen detection, tracking, and airway association.To achieve real-time performance, we employ a benchmark lightweight detector for efficient lumen detection. We are the first to introduce multi-object tracking to bronchoscopic localization, mitigating temporal confusion in lumen identification caused by rapid bronchoscope movement and complex airway structures. To ensure generalization across patient cases, we propose a training-free detection-airway association method based on a semantic airway graph that encodes the hierarchy of bronchial tree structures.Experiments on nine patient datasets demonstrate BronchoTrack's localization accuracy of 85.64 \%, while accessing up to the 4th generation of airways.Furthermore, we tested BronchoTrack in an in-vivo animal study using a porcine model, where it successfully localized the bronchoscope into the 8th generation airway.Experimental evaluation underscores BronchoTrack's real-time performance in both satisfying accuracy and generalization, demonstrating its potential for clinical applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. “Cancer statistics, 2023” In CA: a cancer journal for clinicians 73.1 Wiley Online Library, 2023, pp. 17–48
  2. “Epidemiology of lung cancer” In Contemporary Oncology/Współczesna Onkologia 25.1 Termedia, 2021, pp. 45–52
  3. “The role of bronchoscopy in the diagnosis of early lung cancer: a review” In Journal of thoracic disease 8.11 AME Publications, 2016, pp. 3329
  4. Hossein Sadjadi, Keyvan Hashtrudi-Zaad and Gabor Fichtinger “Simultaneous electromagnetic tracking and calibration for dynamic field distortion compensation” In IEEE Transactions on Biomedical Engineering 63.8 IEEE, 2015, pp. 1771–1781
  5. “Shape sensing techniques for continuum robots in minimally invasive surgery: A survey” In IEEE Transactions on Biomedical Engineering 64.8 IEEE, 2016, pp. 1665–1678
  6. “Generative localization with uncertainty estimation through video-CT data for bronchoscopic biopsy” In IEEE Robotics and Automation Letters 5.1 IEEE, 2019, pp. 258–265
  7. “Autonomous driving in the lung using deep learning for localization” In arXiv preprint arXiv:1907.08136, 2019
  8. “Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images” In Medical Image Analysis 6.3 Elsevier, 2002, pp. 321–336
  9. “Selective image similarity measure for bronchoscope tracking based on image registration” In Medical Image Analysis 13.4 Elsevier, 2009, pp. 621–633
  10. “Context-aware depth and pose estimation for bronchoscopic navigation” In IEEE Robotics and Automation Letters 4.2 IEEE, 2019, pp. 732–739
  11. “Visually navigated bronchoscopy using three cycle-consistent generative adversarial network for depth estimation” In Medical image analysis 73 Elsevier, 2021, pp. 102164
  12. “Interactive CT-video registration for the continuous guidance of bronchoscopy” In IEEE transactions on Medical Imaging 32.8 IEEE, 2013, pp. 1376–1396
  13. “A discriminative structural similarity measure and its application to video-volume registration for endoscope three-dimensional motion tracking” In IEEE transactions on medical imaging 33.6 IEEE, 2014, pp. 1248–1261
  14. “Adversarial domain feature adaptation for bronchoscopic depth estimation” In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, 2021, pp. 300–310 Springer
  15. “Augmenting colonoscopy using extended and directional cyclegan for lossy image translation” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4696–4705
  16. “EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos” In Medical image analysis 71 Elsevier, 2021, pp. 102058
  17. “Offsetnet: Deep learning for localization in the lung using rendered images” In 2019 international conference on robotics and automation (ICRA), 2019, pp. 5046–5052 IEEE
  18. Abhinav Valada, Noha Radwan and Wolfram Burgard “Deep auxiliary learning for visual localization and odometry” In 2018 IEEE international conference on robotics and automation (ICRA), 2018, pp. 6939–6946 IEEE
  19. “Understanding the limitations of cnn-based absolute camera pose regression” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 3302–3312
  20. “Visual SLAM for bronchoscope tracking and bronchus reconstruction in bronchoscopic navigation” In Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling 10951, 2019, pp. 51–57 SPIE
  21. “A visual SLAM-based bronchoscope tracking scheme for bronchoscopic navigation” In International Journal of Computer Assisted Radiology and Surgery 15 Springer, 2020, pp. 1619–1630
  22. “Branch: Bifurcation recognition for airway navigation based on structural characteristics” In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II 20, 2017, pp. 182–189 Springer
  23. “Depth-based branching level estimation for bronchoscopic navigation” In International Journal of Computer Assisted Radiology and Surgery 16 Springer, 2021, pp. 1795–1804
  24. “Stable anatomical structure tracking for video-bronchoscopy navigation” In Clinical Image-Based Procedures. Translational Research in Medical Imaging: 5th International Workshop, CLIP 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Proceedings 5, 2016, pp. 18–26 Springer
  25. “Towards a Videobronchoscopy Localization System from Airway Centre Tracking.” In VISIGRAPP (4: VISAPP), 2017, pp. 352–359
  26. “Alleviating class-wise gradient imbalance for pulmonary airway segmentation” In IEEE transactions on medical imaging 40.9 IEEE, 2021, pp. 2452–2462
  27. Ta-Chih Lee, Rangasami L Kashyap and Chong-Nam Chu “Building skeleton models via 3-D medial surface axis thinning algorithms” In CVGIP: Graphical Models and Image Processing 56.6 Elsevier, 1994, pp. 462–478
  28. Chien-Yao Wang, Alexey Bochkovskiy and Hong-Yuan Mark Liao “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464–7475
  29. “Deeply-supervised nets” In Artificial intelligence and statistics, 2015, pp. 562–570 Pmlr
  30. Rudolph Emil Kalman “A new approach to linear filtering and prediction problems”, 1960
  31. “Strongsort: Make deepsort great again” In IEEE Transactions on Multimedia IEEE, 2023
  32. Nicolai Wojke, Alex Bewley and Dietrich Paulus “Simple online and realtime tracking with a deep association metric” In 2017 IEEE international conference on image processing (ICIP), 2017, pp. 3645–3649 IEEE
  33. “Mat: Motion-aware multi-object tracking” In Neurocomputing 476 Elsevier, 2022, pp. 75–86
  34. “Bytetrack: Multi-object tracking by associating every detection box” In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, 2022, pp. 1–21 Springer
  35. Nir Aharon, Roy Orfaig and Ben-Zion Bobrovsky “BoT-SORT: Robust associations multi-pedestrian tracking” In arXiv preprint arXiv:2206.14651, 2022
  36. “Towards real-time multi-object tracking” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, 2020, pp. 107–122 Springer
  37. Harold W Kuhn “The Hungarian method for the assignment problem” In Naval research logistics quarterly 2.1-2 Wiley Online Library, 1955, pp. 83–97
  38. Raul Mur-Artal, Jose Maria Martinez Montiel and Juan D Tardos “ORB-SLAM: a versatile and accurate monocular SLAM system” In IEEE transactions on robotics 31.5 IEEE, 2015, pp. 1147–1163
  39. Raul Mur-Artal and Juan D Tardós “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras” In IEEE transactions on robotics 33.5 IEEE, 2017, pp. 1255–1262
  40. “LoFTR: Detector-free local feature matching with transformers” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8922–8931
Citations (2)

Summary

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.