DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy (1702.08192v1)

Abstract: We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7 million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.

• The paper introduces a novel 3D deep CNN that segments neuroanatomy in T1-weighted MRI using a two-stage hierarchical framework.
• It leverages spectral coordinates and applies a fully connected CRF to maintain spatial context and enforce segmentation consistency.
• Empirical results show a higher median Dice score compared to traditional methods, indicating its potential for clinical application.

Overview of DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy

The paper presents DeepNAT, a 3D deep convolutional neural network (DCNN) designed for the automatic segmentation of neuroanatomy in T1-weighted brain MRI images. This model stands out due to its end-to-end learning framework, which concurrently learns feature representation and multi-class classification. Researchers employed a 3D patch-based approach, mitigating class imbalance by using a two-stage hierarchical structure. The first stage separates foreground from background, while the second identifies distinct brain structures within the segmented foreground.

Methodology

DeepNAT introduces several novel features and methodological innovations:

1. Hierarchical Segmentation: This strategy primarily addresses the class imbalance problem, separating segmentation into two layered networks. Initially, foreground is separated from background, and subsequently, specific structures are categorized within the foreground. Such a hierarchical method allows for more accurate differentiation of brain structures by initially focusing on the broader context provided by the foreground.
2. Spectral Coordinates: The authors implement an intrinsic parameterization of the brain's spatial arrangement using spectral coordinates derived from eigenfunctions of the Laplace-Beltrami operator. This technique retains context information within patches, which is crucial since such spatial context can be missing in standard patch-based segmentation approaches.
3. Network Architecture: DeepNAT employs three convolutional layers complemented by pooling, batch normalization, and ReLU activations. The architecture concludes with fully connected layers using dropout for regularization. A key component is the usage of approximately 2.7 million parameters, which are optimized through stochastic gradient descent.
4. Conditional Random Field (CRF) Post-processing: To ensure continuity and agreement in the segmented areas, a fully connected CRF is applied to the probabilistic output of DeepNAT. This CRF uses Gaussian kernels to efficiently compute pairwise voxel potentials and improves segmentation accuracy by enforcing local consistency.

Results

Empirical evaluations indicate that DeepNAT performs favorably when compared with conventional segmentation techniques such as FreeSurfer and other state-of-the-art approaches like PICSL and STAPLE. Specifically, DeepNAT achieves a higher median Dice score, underscoring its efficacy in segmenting brain structures. By fine-tuning with additional data, particularly populations of young, old, or diseased individuals, DeepNAT's accuracy could potentially be enhanced further.

Implications and Future Directions

Practically, DeepNAT's outcomes highlight the growing capabilities of DCNNs in medical imaging, specifically MRI segmentation tasks that traditionally challenged algorithmic segmentation due to high-dimensional data and subtle anatomical variances. Theoretically, the introduction of spectral coordinates demonstrates a promising direction in representing 3D spatial data in neural networks.

Deep learning strategies continue to show tremendous potential in improving segmentation accuracy by leveraging large datasets and more intricate network architectures. Future extensions of this work might involve augmenting these neural networks with unsupervised learning techniques to automatically adapt to various anatomical features, enabling better utility across different MRI modalities or pathological conditions.

The presented methodology invites further research in the domain of transfer learning within volumetric data, particularly in fine-tuning pre-trained 3D networks for diverse clinical applications. As the availability of labeled datasets grows and computational resources become more accessible, the deployment of networks like DeepNAT will become increasingly viable, with potential spillover effects into other medical imaging applications.