Progressive Curriculum Learning with Scale-Enhanced U-Net for Continuous Airway Segmentation (2410.18456v3)
Abstract: Continuous and accurate segmentation of airways in chest CT images is essential for preoperative planning and real-time bronchoscopy navigation. Despite advances in deep learning for medical image segmentation, maintaining airway continuity remains a challenge, particularly due to intra-class imbalance between large and small branches and blurred CT scan details. To address these challenges, we propose a progressive curriculum learning pipeline and a Scale-Enhanced U-Net (SE-UNet) to enhance segmentation continuity. Specifically, our progressive curriculum learning pipeline consists of three stages: extracting main airways, identifying small airways, and repairing discontinuities. The cropping sampling strategy in each stage reduces feature interference between airways of different scales, effectively addressing the challenge of intra-class imbalance. In the third training stage, we present an Adaptive Topology-Responsive Loss (ATRL) to guide the network to focus on airway continuity. The progressive training pipeline shares the same SE-UNet, integrating multi-scale inputs and Detail Information Enhancers (DIEs) to enhance information flow and effectively capture the intricate details of small airways. Additionally, we propose a robust airway tree parsing method and hierarchical evaluation metrics to provide more clinically relevant and precise analysis. Experiments on both in-house and public datasets demonstrate that our method outperforms existing approaches, significantly improving the accuracy of small airways and the completeness of the airway tree. The code will be released upon publication.
- S. Momtazmanesh, S. S. Moghaddam, S.-H. Ghamari, E. M. Rad, N. Rezaei, P. Shobeiri, A. Aali, M. Abbasi-Kangevari, Z. Abbasi-Kangevari, M. Abdelmasseh et al., “Global burden of chronic respiratory diseases and risk factors, 1990–2019: an update from the global burden of disease study 2019,” EClinicalMedicine, vol. 59, 2023.
- T. Ishiwata, A. Gregor, T. Inage, and K. Yasufuku, “Bronchoscopic navigation and tissue diagnosis,” General thoracic and cardiovascular surgery, vol. 68, pp. 672–678, 2020.
- D. Aykac, E. A. Hoffman, G. McLennan, and J. M. Reinhardt, “Segmentation and analysis of the human airway tree from three-dimensional x-ray ct images,” IEEE transactions on medical imaging, vol. 22, no. 8, pp. 940–950, 2003.
- J. Tschirren, E. A. Hoffman, G. McLennan, and M. Sonka, “Intrathoracic airway trees: segmentation and airway morphology analysis from low-dose ct scans,” IEEE transactions on medical imaging, vol. 24, no. 12, pp. 1529–1539, 2005.
- A. P. Kiraly, W. E. Higgins, G. McLennan, E. A. Hoffman, and J. M. Reinhardt, “Three-dimensional human airway segmentation methods for clinical virtual bronchoscopy,” Academic radiology, vol. 9, no. 10, pp. 1153–1168, 2002.
- S. Born, D. Iwamaru, M. Pfeifle, and D. Bartz, “Three-step segmentation of the lower airways with advanced leakage-control,” in Proc. of Second International Workshop on Pulmonary Image Analysis. Citeseer, 2009, pp. 239–249.
- J.-P. Charbonnier, E. M. Van Rikxoort, A. A. Setio, C. M. Schaefer-Prokop, B. van Ginneken, and F. Ciompi, “Improving airway segmentation in computed tomography using leak detection with convolutional networks,” Medical image analysis, vol. 36, pp. 52–60, 2017.
- A. Garcia-Uceda Juarez, H. A. Tiddens, and M. de Bruijne, “Automatic airway segmentation in chest ct using convolutional neural networks,” in Image Analysis for Moving Organ, Breast, and Thoracic Images: Third International Workshop, RAMBO 2018, Fourth International Workshop, BIA 2018, and First International Workshop, TIA 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16 and 20, 2018, Proceedings 3. Springer, 2018, pp. 238–250.
- Y. Qin, M. Chen, H. Zheng, Y. Gu, M. Shen, J. Yang, X. Huang, Y.-M. Zhu, and G.-Z. Yang, “Airwaynet: a voxel-connectivity aware approach for accurate airway segmentation using convolutional neural networks,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 212–220.
- Y. Qin, H. Zheng, Y. Gu, X. Huang, J. Yang, L. Wang, and Y.-M. Zhu, “Learning bronchiole-sensitive airway segmentation cnns by feature recalibration and attention distillation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 221–231.
- H. Zheng, Y. Qin, Y. Gu, F. Xie, J. Yang, J. Sun, and G.-Z. Yang, “Alleviating class-wise gradient imbalance for pulmonary airway segmentation,” IEEE transactions on medical imaging, vol. 40, no. 9, pp. 2452–2462, 2021.
- Y. Nan, J. Del Ser, Z. Tang, P. Tang, X. Xing, Y. Fang, F. Herrera, W. Pedrycz, S. Walsh, and G. Yang, “Fuzzy attention neural network to tackle discontinuity in airway segmentation,” IEEE Transactions on Neural Networks and Learning Systems, 2023.
- P. Wang, D. Guo, D. Zheng, M. Zhang, H. Yu, X. Sun, J. Ge, Y. Gu, L. Lu, X. Ye et al., “Accurate airway tree segmentation in ct scans via anatomy-aware multi-class segmentation and topology-guided iterative learning,” IEEE Transactions on Medical Imaging, 2024.
- J. Tschirren, G. McLennan, K. Palágyi, E. A. Hoffman, and M. Sonka, “Matching and anatomical labeling of human airway tree,” IEEE transactions on medical imaging, vol. 24, no. 12, pp. 1540–1547, 2005.
- C. Bauer, H. Bischof, and R. Beichel, “Segmentation of airways based on gradient vector flow,” in International workshop on pulmonary image analysis, Medical image computing and computer assisted intervention, 2009, pp. 191–201.
- P. Lo, B. Van Ginneken, J. M. Reinhardt, T. Yavarna, P. A. De Jong, B. Irving, C. Fetita, M. Ortner, R. Pinho, J. Sijbers et al., “Extraction of airways from ct (exact’09),” IEEE Transactions on Medical Imaging, vol. 31, no. 11, pp. 2093–2107, 2012.
- R. Selvan, T. Kipf, M. Welling, A. G.-U. Juarez, J. H. Pedersen, J. Petersen, and M. de Bruijne, “Graph refinement based airway extraction using mean-field networks and graph neural networks,” Medical image analysis, vol. 64, p. 101751, 2020.
- Q. Meng, H. R. Roth, T. Kitasaka, M. Oda, J. Ueno, and K. Mori, “Tracking and segmentation of the airways in chest ct using a fully convolutional network,” in Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II 20. Springer, 2017, pp. 198–207.
- P. J. Burt and E. H. Adelson, “The laplacian pyramid as a compact image code,” in Readings in computer vision. Elsevier, 1987, pp. 671–679.
- N. Abraham and N. M. Khan, “A novel focal tversky loss function with improved attention u-net for lesion segmentation,” in 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, 2019, pp. 683–687.
- H. Fu, J. Cheng, Y. Xu, D. W. K. Wong, J. Liu, and X. Cao, “Joint optic disc and cup segmentation based on multi-label deep network and polar transformation,” IEEE transactions on medical imaging, vol. 37, no. 7, pp. 1597–1605, 2018.
- H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2881–2890.
- F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in 2016 fourth international conference on 3D vision (3DV). Ieee, 2016, pp. 565–571.
- C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
- H. Zheng, Y. Qin, Y. Gu, F. Xie, J. Sun, J. Yang, and G.-Z. Yang, “Refined local-imbalance-based weight for airway segmentation in ct,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2021, pp. 410–419.
- M. Zhang, Y. Wu, H. Zhang, Y. Qin, H. Zheng, W. Tang, C. Arnold, C. Pei, P. Yu, Y. Nan et al., “Multi-site, multi-domain airway tree modeling,” Medical Image Analysis, vol. 90, p. 102957, 2023.
- W. Yu, H. Zheng, M. Zhang, H. Zhang, J. Sun, and J. Yang, “Break: Bronchi reconstruction by geodesic transformation and skeleton embedding,” in 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, 2022, pp. 1–5.
- M. Zhang, H. Zhang, G.-Z. Yang, and Y. Gu, “Cfda: Collaborative feature disentanglement and augmentation for pulmonary airway tree modeling of covid-19 cts,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part I. Springer, 2022, pp. 506–516.
- S. G. Armato III, G. McLennan, L. Bidaut, M. F. McNitt-Gray, C. R. Meyer, A. P. Reeves, B. Zhao, D. R. Aberle, C. I. Henschke, E. A. Hoffman et al., “The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans,” Medical physics, vol. 38, no. 2, pp. 915–931, 2011.
- W. Liu, H. Yang, T. Tian, Z. Cao, X. Pan, W. Xu, Y. Jin, and F. Gao, “Full-resolution network and dual-threshold iteration for retinal vessel and coronary angiograph segmentation,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 9, pp. 4623–4634, 2022.
- Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19. Springer, 2016, pp. 424–432.
- F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,” Nature methods, vol. 18, no. 2, pp. 203–211, 2021.
- Y. Qin, H. Zheng, Y. Gu, X. Huang, J. Yang, L. Wang, F. Yao, Y.-M. Zhu, and G.-Z. Yang, “Learning tubule-sensitive cnns for pulmonary airway and artery-vein segmentation in ct,” IEEE transactions on medical imaging, vol. 40, no. 6, pp. 1603–1617, 2021.