AFFSegNet: Adaptive Feature Fusion Segmentation Network for Microtumors and Multi-Organ Segmentation (2409.07779v3)

Abstract: Medical image segmentation, a crucial task in computer vision, facilitates the automated delineation of anatomical structures and pathologies, supporting clinicians in diagnosis, treatment planning, and disease monitoring. Notably, transformers employing shifted window-based self-attention have demonstrated exceptional performance. However, their reliance on local window attention limits the fusion of local and global contextual information, crucial for segmenting microtumors and miniature organs. To address this limitation, we propose the Adaptive Semantic Segmentation Network (ASSNet), a transformer architecture that effectively integrates local and global features for precise medical image segmentation. ASSNet comprises a transformer-based U-shaped encoder-decoder network. The encoder utilizes shifted window self-attention across five resolutions to extract multi-scale features, which are then propagated to the decoder through skip connections. We introduce an augmented multi-layer perceptron within the encoder to explicitly model long-range dependencies during feature extraction. Recognizing the constraints of conventional symmetrical encoder-decoder designs, we propose an Adaptive Feature Fusion (AFF) decoder to complement our encoder. This decoder incorporates three key components: the Long Range Dependencies (LRD) block, the Multi-Scale Feature Fusion (MFF) block, and the Adaptive Semantic Center (ASC) block. These components synergistically facilitate the effective fusion of multi-scale features extracted by the decoder while capturing long-range dependencies and refining object boundaries. Comprehensive experiments on diverse medical image segmentation tasks, including multi-organ, liver tumor, and bladder tumor segmentation, demonstrate that ASSNet achieves state-of-the-art results. Code and models are available at: \url{https://github.com/lzeeorno/ASSNet}.

• The paper introduces a novel Transformer-based U-shaped architecture that fuses local and global features for precise microtumor and organ segmentation.
• The Adaptive Feature Fusion decoder, featuring LRD, MFF, and ASC blocks, significantly enhances boundary refinement and multi-scale feature integration.
• Experiments on LiTS2017, ISICDM2019, and Synapse datasets demonstrate state-of-the-art performance, outperforming models like TransUNet and SwinUNet.

ASSNet: Adaptive Semantic Segmentation Network for Microtumors and Multi-Organ Segmentation

ASSNet: Adaptive Semantic Segmentation Network for Microtumors and Multi-Organ Segmentation is a significant contribution to the field of medical image segmentation, authored by Fuchen Zheng et al. The authors address critical challenges in the precise delineation of anatomical structures and pathologies in medical images, leveraging state-of-the-art deep learning architectures. ASSNet's novel architecture integrates both local and global features, enhancing segmentation performance especially for small-scale objects such as microtumors and miniature organs.

Key Contributions

1. Novel Architecture: ASSNet introduces a Transformer-based U-shaped encoder-decoder network, which effectively combines the strengths of Vision Transformers (ViTs) and convolutional neural networks (CNNs). The encoder employs shifted window self-attention to extract multi-scale features, while a novel Adaptive Feature Fusion (AFF) decoder ensures efficient feature integration and boundary refinement.
2. Enhanced Feature Extraction: The encoder within ASSNet incorporates an augmented multi-layer perceptron (MLP) for long-range dependency modeling. This component enhances the network's ability to capture intricate details essential for effective medical image segmentation.
3. Adaptive Feature Fusion Decoder: The AFF decoder comprises three critical blocks - the Long Range Dependencies (LRD) block, the Multi-Scale Feature Fusion (MFF) block, and the Adaptive Semantic Center (ASC) block. These blocks collectively enhance the fusion of features at multiple scales while preserving boundary integrity, which is crucial for precise segmentation of small and complex structures.

Experimental Results

The efficacy of ASSNet is demonstrated through comprehensive experiments on three prominent medical image segmentation datasets: LiTS2017, ISICDM2019, and Synapse.

• LiTS2017 Dataset: Focused on liver tumor segmentation, ASSNet achieved an average DSC of 95.47% and an mIoU of 94.88%, outperforming other state-of-the-art methods such as TransUNet (DSC: 93.29%) and SwinUNet (DSC: 91.95%).
• ISICDM2019 Dataset: Targeting bladder tumor segmentation, ASSNet outperformed previous models with an average DSC of 96.75% and an mIoU of 96.04%. This is a notable improvement over the TransUNet model which achieved a DSC of 94.56%.
• Synapse Multi-Organ Dataset: For multi-organ segmentation, ASSNet achieved an average DSC of 90.73%, demonstrating robust performance across various organs. This includes achieving the highest DSC scores for liver (97.11%) and right kidney (93.06%).

Methodology

ASSNet's methodology revolves around a robust and adaptive architectural framework. The hierarchical U-shaped structure integrates both local and global context through the following components:

• MWA Transformer Block: The Multi-scale Window Attention (MWA) Transformer block substitutes the standard multi-head self-attention (MSA) module with a shifted window-based MSA, significantly enhancing the ability to capture detailed local information while maintaining global context.
• Enhanced Feed-Forward Network (EFFN) within the MWA block utilizes depth-wise and pixel-wise convolutions to enrich local contextual information, further improving feature extraction.
• Adaptive Feature Fusion (AFF) Decoder: Central to ASSNet's performance, the AFF decoder incorporates:
  • LRD Block: Enhances the capture of long-range dependencies which is critical for preserving important features across the encoding and decoding stages.
  • MFF Block: Facilitates the fusion of features at varying scales, addressing the challenge of segmenting structures of different sizes and resolutions.
  • ASC Block: Utilizes adaptive average pooling and edge detection techniques to refine boundaries and enhance the segmentation of critical regions.

Conclusion and Future Implications

ASSNet showcases significant advancements in medical image segmentation, particularly through its innovative integration of local and global feature extraction mechanisms. The model's ability to achieve state-of-the-art results across multiple challenging datasets highlights its robustness and versatility.

While ASSNet proves effective in current experiments, future work could explore its applicability across other medical imaging modalities, and extend its use in even more complex clinical scenarios. Additionally, further refinement of the AFF Decoder and exploration of alternative architectures could enhance its performance and efficiency.

Through this research, the authors contribute to both the theoretical understanding and practical application of deep learning in medical image analysis, providing a valuable framework for future advancements in this critical field.