BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization
Abstract: Localizing the bronchoscope in real time is essential for ensuring intervention quality. However, most existing methods struggle to balance between speed and generalization. To address these challenges, we present BronchoTrack, an innovative real-time framework for accurate branch-level localization, encompassing lumen detection, tracking, and airway association.To achieve real-time performance, we employ a benchmark lightweight detector for efficient lumen detection. We are the first to introduce multi-object tracking to bronchoscopic localization, mitigating temporal confusion in lumen identification caused by rapid bronchoscope movement and complex airway structures. To ensure generalization across patient cases, we propose a training-free detection-airway association method based on a semantic airway graph that encodes the hierarchy of bronchial tree structures.Experiments on nine patient datasets demonstrate BronchoTrack's localization accuracy of 85.64 \%, while accessing up to the 4th generation of airways.Furthermore, we tested BronchoTrack in an in-vivo animal study using a porcine model, where it successfully localized the bronchoscope into the 8th generation airway.Experimental evaluation underscores BronchoTrack's real-time performance in both satisfying accuracy and generalization, demonstrating its potential for clinical applications.
- “Cancer statistics, 2023” In CA: a cancer journal for clinicians 73.1 Wiley Online Library, 2023, pp. 17–48
- “Epidemiology of lung cancer” In Contemporary Oncology/Współczesna Onkologia 25.1 Termedia, 2021, pp. 45–52
- “The role of bronchoscopy in the diagnosis of early lung cancer: a review” In Journal of thoracic disease 8.11 AME Publications, 2016, pp. 3329
- Hossein Sadjadi, Keyvan Hashtrudi-Zaad and Gabor Fichtinger “Simultaneous electromagnetic tracking and calibration for dynamic field distortion compensation” In IEEE Transactions on Biomedical Engineering 63.8 IEEE, 2015, pp. 1771–1781
- “Shape sensing techniques for continuum robots in minimally invasive surgery: A survey” In IEEE Transactions on Biomedical Engineering 64.8 IEEE, 2016, pp. 1665–1678
- “Generative localization with uncertainty estimation through video-CT data for bronchoscopic biopsy” In IEEE Robotics and Automation Letters 5.1 IEEE, 2019, pp. 258–265
- “Autonomous driving in the lung using deep learning for localization” In arXiv preprint arXiv:1907.08136, 2019
- “Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images” In Medical Image Analysis 6.3 Elsevier, 2002, pp. 321–336
- “Selective image similarity measure for bronchoscope tracking based on image registration” In Medical Image Analysis 13.4 Elsevier, 2009, pp. 621–633
- “Context-aware depth and pose estimation for bronchoscopic navigation” In IEEE Robotics and Automation Letters 4.2 IEEE, 2019, pp. 732–739
- “Visually navigated bronchoscopy using three cycle-consistent generative adversarial network for depth estimation” In Medical image analysis 73 Elsevier, 2021, pp. 102164
- “Interactive CT-video registration for the continuous guidance of bronchoscopy” In IEEE transactions on Medical Imaging 32.8 IEEE, 2013, pp. 1376–1396
- “A discriminative structural similarity measure and its application to video-volume registration for endoscope three-dimensional motion tracking” In IEEE transactions on medical imaging 33.6 IEEE, 2014, pp. 1248–1261
- “Adversarial domain feature adaptation for bronchoscopic depth estimation” In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, 2021, pp. 300–310 Springer
- “Augmenting colonoscopy using extended and directional cyclegan for lossy image translation” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4696–4705
- “EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos” In Medical image analysis 71 Elsevier, 2021, pp. 102058
- “Offsetnet: Deep learning for localization in the lung using rendered images” In 2019 international conference on robotics and automation (ICRA), 2019, pp. 5046–5052 IEEE
- Abhinav Valada, Noha Radwan and Wolfram Burgard “Deep auxiliary learning for visual localization and odometry” In 2018 IEEE international conference on robotics and automation (ICRA), 2018, pp. 6939–6946 IEEE
- “Understanding the limitations of cnn-based absolute camera pose regression” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 3302–3312
- “Visual SLAM for bronchoscope tracking and bronchus reconstruction in bronchoscopic navigation” In Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling 10951, 2019, pp. 51–57 SPIE
- “A visual SLAM-based bronchoscope tracking scheme for bronchoscopic navigation” In International Journal of Computer Assisted Radiology and Surgery 15 Springer, 2020, pp. 1619–1630
- “Branch: Bifurcation recognition for airway navigation based on structural characteristics” In Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II 20, 2017, pp. 182–189 Springer
- “Depth-based branching level estimation for bronchoscopic navigation” In International Journal of Computer Assisted Radiology and Surgery 16 Springer, 2021, pp. 1795–1804
- “Stable anatomical structure tracking for video-bronchoscopy navigation” In Clinical Image-Based Procedures. Translational Research in Medical Imaging: 5th International Workshop, CLIP 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 17, 2016, Proceedings 5, 2016, pp. 18–26 Springer
- “Towards a Videobronchoscopy Localization System from Airway Centre Tracking.” In VISIGRAPP (4: VISAPP), 2017, pp. 352–359
- “Alleviating class-wise gradient imbalance for pulmonary airway segmentation” In IEEE transactions on medical imaging 40.9 IEEE, 2021, pp. 2452–2462
- Ta-Chih Lee, Rangasami L Kashyap and Chong-Nam Chu “Building skeleton models via 3-D medial surface axis thinning algorithms” In CVGIP: Graphical Models and Image Processing 56.6 Elsevier, 1994, pp. 462–478
- Chien-Yao Wang, Alexey Bochkovskiy and Hong-Yuan Mark Liao “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464–7475
- “Deeply-supervised nets” In Artificial intelligence and statistics, 2015, pp. 562–570 Pmlr
- Rudolph Emil Kalman “A new approach to linear filtering and prediction problems”, 1960
- “Strongsort: Make deepsort great again” In IEEE Transactions on Multimedia IEEE, 2023
- Nicolai Wojke, Alex Bewley and Dietrich Paulus “Simple online and realtime tracking with a deep association metric” In 2017 IEEE international conference on image processing (ICIP), 2017, pp. 3645–3649 IEEE
- “Mat: Motion-aware multi-object tracking” In Neurocomputing 476 Elsevier, 2022, pp. 75–86
- “Bytetrack: Multi-object tracking by associating every detection box” In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, 2022, pp. 1–21 Springer
- Nir Aharon, Roy Orfaig and Ben-Zion Bobrovsky “BoT-SORT: Robust associations multi-pedestrian tracking” In arXiv preprint arXiv:2206.14651, 2022
- “Towards real-time multi-object tracking” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, 2020, pp. 107–122 Springer
- Harold W Kuhn “The Hungarian method for the assignment problem” In Naval research logistics quarterly 2.1-2 Wiley Online Library, 1955, pp. 83–97
- Raul Mur-Artal, Jose Maria Martinez Montiel and Juan D Tardos “ORB-SLAM: a versatile and accurate monocular SLAM system” In IEEE transactions on robotics 31.5 IEEE, 2015, pp. 1147–1163
- Raul Mur-Artal and Juan D Tardós “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras” In IEEE transactions on robotics 33.5 IEEE, 2017, pp. 1255–1262
- “LoFTR: Detector-free local feature matching with transformers” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8922–8931
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.