Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation (1702.00045v1)

Abstract: Accurate and automatic organ segmentation from 3D radiological scans is an important yet challenging problem for medical image analysis. Specifically, the pancreas demonstrates very high inter-patient anatomical variability in both its shape and volume. In this paper, we present an automated system using 3D computed tomography (CT) volumes via a two-stage cascaded approach: pancreas localization and segmentation. For the first step, we localize the pancreas from the entire 3D CT scan, providing a reliable bounding box for the more refined segmentation step. We introduce a fully deep-learning approach, based on an efficient application of holistically-nested convolutional networks (HNNs) on the three orthogonal axial, sagittal, and coronal views. The resulting HNN per-pixel probability maps are then fused using pooling to reliably produce a 3D bounding box of the pancreas that maximizes the recall. We show that our introduced localizer compares favorably to both a conventional non-deep-learning method and a recent hybrid approach based on spatial aggregation of superpixels using random forest classification. The second, segmentation, phase operates within the computed bounding box and integrates semantic mid-level cues of deeply-learned organ interior and boundary maps, obtained by two additional and separate realizations of HNNs. By integrating these two mid-level cues, our method is capable of generating boundary-preserving pixel-wise class label maps that result in the final pancreas segmentation. Quantitative evaluation is performed on a publicly available dataset of 82 patient CT scans using 4-fold cross-validation (CV). We achieve a Dice similarity coefficient (DSC) of 81.27+/-6.27% in validation, which significantly outperforms previous state-of-the art methods that report DSCs of 71.80+/-10.70% and 78.01+/-8.20%, respectively, using the same dataset.

Overview of "Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation"

The paper proposes a novel automated system designed for pancreas localization and segmentation from 3D CT volumes, utilizing a two-stage cascaded approach. This work is particularly significant due to the inherent challenges associated with pancreas segmentation in medical imaging. The pancreas, being a small and morphologically variable organ, exhibits substantial inter-patient variability, hampering the performance of conventional segmentation techniques.

Methodology

The approach is structured in two stages: localization and segmentation, both employing holistically-nested convolutional networks (HNNs). The core innovation lies in the holistic application of these networks across three orthogonal views—axial, sagittal, and coronal—to generate per-pixel probability maps for the pancreas. These maps are merged using pooling strategies to create a 3D bounding box for accurate pancreas localization.

1. Localization:
   • An innovative multi-view HNN methodology is utilized, maximizing spatial recall by generating bounding boxes that capture nearly the entire pancreas volume within CT scans. This step significantly reduces the irrelevant volumetric region for subsequent processing.
2. Segmentation:
   • Inside the bounding box, semantic mid-level cues including organ interior and boundary maps are derived from HNNs. These cues facilitate robust generation of boundary-preserving, pixel-wise class label maps for final pancreas segmentation. The task exploits both interior and boundary detections as mid-level cues for enhanced accuracy.

Results

Quantitative evaluations were conducted on a public dataset of CT scans, using 4-fold cross-validation. The proposed system achieved an impressive mean Dice similarity coefficient (DSC) of 81.27% with a relatively low standard deviation of 6.27%. This performance starkly exceeds that of prior state-of-the-art methods, which reported DSCs of around 71.80% and 78.01%. The computational efficiency is notable as well, with the system typically processing an image in 2-3 minutes.

Implications and Future Directions

The proposed deep learning framework sets a benchmark in pancreas segmentation, indicating its potential applicability to other complex organ segmentation tasks where anatomical variability presents a substantial challenge. The work underscores the importance of leveraging multi-view strategies and spatial aggregation of CNN outputs to enhance organ localization and segmentation accuracy.

Future research could extend this approach to other organs and investigate its robustness in pathological cases such as tumors, where the anatomical structures vary even more drastically. Additionally, while the paper presents segmentation results with impressive precision, further refinement in segmentation techniques and boundary detection using advanced deep learning architectures could pave the way for improvements in clinical diagnosis and better aiding radiologists in accurate medical imaging analyses.

In conclusion, this paper significantly contributes to the field of medical image analysis, offering insights into the design and application of deep learning-based methods for complex organ segmentation problems.