Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks (1803.02579v2)

Abstract: Fully convolutional neural networks (F-CNNs) have set the state-of-the-art in image segmentation for a plethora of applications. Architectural innovations within F-CNNs have mainly focused on improving spatial encoding or network connectivity to aid gradient flow. In this paper, we explore an alternate direction of recalibrating the feature maps adaptively, to boost meaningful features, while suppressing weak ones. We draw inspiration from the recently proposed squeeze & excitation (SE) module for channel recalibration of feature maps for image classification. Towards this end, we introduce three variants of SE modules for image segmentation, (i) squeezing spatially and exciting channel-wise (cSE), (ii) squeezing channel-wise and exciting spatially (sSE) and (iii) concurrent spatial and channel squeeze & excitation (scSE). We effectively incorporate these SE modules within three different state-of-the-art F-CNNs (DenseNet, SD-Net, U-Net) and observe consistent improvement of performance across all architectures, while minimally effecting model complexity. Evaluations are performed on two challenging applications: whole brain segmentation on MRI scans (Multi-Atlas Labelling Challenge Dataset) and organ segmentation on whole body contrast enhanced CT scans (Visceral Dataset).

• The paper introduces concurrent spatial and channel SE modules that enhance feature recalibration in fully convolutional networks.
• It integrates three SE module variants (cSE, sSE, scSE) into DenseNet, SD-Net, and U-Net, achieving Dice score improvements on MALC and Visceral datasets.
• The approach maintains low computational overhead while significantly improving segmentation accuracy, indicating potential for dynamic, hybrid models.

Concurrent Spatial and Channel `Squeeze Excitation' in Fully Convolutional Networks: An Overview

In the field of medical image segmentation, advancements in Fully Convolutional Neural Networks (F-CNNs) have set a new standard for performance. However, a critical challenge remains in efficiently recalibrating feature maps to enhance relevant features while suppressing less meaningful ones. The paper "Concurrent Spatial and Channel `Squeeze Excitation' in Fully Convolutional Networks" by Abhijit Guha Roy et al. introduces novel architectural modules aimed at addressing this challenge.

Methodological Innovations

The paper introduces three variants of Squeeze Excitation (SE) modules designed specifically for F-CNNs in image segmentation tasks:

1. Channel Squeeze and Spatial Excitation (cSE): This module performs spatial squeezing through global average pooling, followed by channel-wise excitation. This approach focuses on the importance of each channel, adapting the feature maps by emphasizing the channels that carry more significant information.
2. Spatial Squeeze and Channel Excitation (sSE): This module conversely performs squeezing along the channel dimension and excitation spatially. By re-calibrating at each spatial location, the approach aims to capture fine-grained spatial details critical for accurate segmentation.
3. Concurrent Spatial and Channel Squeeze Excitation (scSE): This module concurrently applies both spatial and channel squeezing, followed by their respective excitations. The output is a combined recalibration that harnesses the benefits of both methods, providing a composite reweighting mechanism.

These SE modules are seamlessly integrated within three different F-CNN architectures: DenseNet, SD-Net, and U-Net. The integration involves placing SE blocks after each encoder and decoder block, ensuring consistent recalibration throughout the network.

Experimental Results

The paper conducts extensive experiments to validate the effectiveness of the proposed SE modules, using two significant datasets:

1. Multi-Atlas Labeling Challenge (MALC) Dataset: This dataset involves the segmentation of 27 cortical and subcortical structures in MRI T1 brain scans.
2. Visceral Dataset: This dataset focuses on segmenting 10 visceral organs in whole-body contrast-enhanced CT scans.

Quantitative Analysis

The inclusion of SE modules resulted in consistent performance improvements across all tested architectures. Key findings include:

• For the MALC dataset, scSE blocks improved Dice scores by 4-8%, with DenseNet+scSE achieving a mean Dice score of 0.882 ± 0.063.
• For the Visceral dataset, scSE blocks increased Dice scores by 2-3%, with DenseNet+scSE achieving a mean Dice score of 0.918 ± 0.051.

The scSE module consistently outperformed the other variants, indicating the advantage of concurrent recalibration. The experiments also demonstrated that the addition of SE blocks incurs a negligible increase in model complexity—approximately 1.5% for U-Net— making this approach highly efficient.

Implications and Future Directions

The introduction of SE blocks in F-CNNs not only boosts segmentation performance but also opens pathways for more intricate recalibration strategies. The demonstrated efficacy across different network architectures and segmentation tasks indicates that SE modules could become a standard component in medical image segmentation pipelines.

The paper leaves room for further exploration in several directions:

• Dynamic Adaptation: Developing methods to dynamically adjust the weights of SE modules during training could further enhance performance.
• Cross-Domain Applications: Extending the approach to non-medical image segmentation tasks can validate the generalizability of SE modules.
• Hybrid Models: Combining SE blocks with other segmentation-enhancing techniques could result in more robust models.

Conclusion

The paper "Concurrent Spatial and Channel `Squeeze Excitation' in Fully Convolutional Networks" makes a significant contribution to F-CNN architectures by introducing SE modules that recalibrate features both spatially and channel-wise. The consistent performance enhancements verified through rigorous experiments underline the potential of these modules in advancing the state-of-the-art in medical image segmentation. The negligible increase in model complexity further solidifies the practicality of incorporating SE blocks into existing and future neural network architectures.