Compact Split Attention Blocks for DCSAU-Net

• The paper introduces CSA blocks that use dual branch convolutions and split-attention mechanisms to enhance multiscale feature extraction in DCSAU-Net.
• Ablation studies demonstrate significant mIoU and F1 score gains while reducing parameters and FLOPs compared to vanilla U-Net.
• Integrating CSA blocks after each downsampling and upsampling stage enables effective fusion of low- and high-level semantic features with preserved spatial details.

Compact Split Attention (CSA) Blocks are the core architectural innovation underpinning DCSAU-Net, a convolutional encoder-decoder framework designed for medical image segmentation. In contrast to vanilla U-Net, which applies uniform downsampling and simple stacked convolutions at each encoder level, DCSAU-Net leverages a deeper, more compact design by inserting CSA blocks after every downsampling and upsampling stage, allowing for more efficient extraction and fusion of multiscale semantic features. The CSA block utilizes split-attention mechanisms, channel grouping, global context aggregation, and lightweight multi-layer perceptron (MLP) weighting to enhance representational capacity while maintaining low parameter count and computational cost (Xu et al., 2022).

1. Detailed Architecture of Compact Split Attention Blocks

Each CSA block processes an input tensor $X\in\mathbb{R}^{H\times W\times C}$ by splitting its channel dimension into $S=2$ equal-width groups: $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$. Distinct convolutional sequences are then applied to each split:

• Branch 1: $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$
• Branch 2: $$F_2'(X_2) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_2))))))$$, with further merging as $$F_2(X_2) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(F_2'(X_2) + F_1(X_1))))$$

The outputs are fused without attention as $\hat{U} = F_1(X_1) + F_2(X_2)$. Global average pooling then produces channel statistics $S_c = \frac{1}{HW} \sum_{i,j} \hat{U}_c(i,j)$. These statistics are fed to a small MLP composed of two $1\times1$ convolutions (with BatchNorm and ReLU in between), yielding split-specific logits $G_1(S), G_2(S) \in \mathbb{R}^C$. A softmax is computed across splits for each channel, generating weights $S=2$0.

The two group outputs are re-weighted and merged per channel:

$S=2$1

A residual connection is then applied: $S=2$2 (or $S=2$3 if channel shapes differ). If needed, a $S=2$4 convolution or convolution plus pooling matches channel dimensions.

2. Placement and Integration within DCSAU-Net

CSA blocks are tightly woven into the DCSAU-Net structure, replacing standard convolutional blocks in specific locations as follows:

• After each of the four encoder downsampling stages (except the initial Primary Feature Conservation (PFC) block), a CSA block follows the $S=2$5 max-pool.
• At the bottleneck ("bottom" of the U), a CSA processes the deepest feature.
• Within the decoder, after each upsampling and concatenation with the encoder's feature map, a CSA block further refines the fused representation.

This placement yields nine CSA blocks in total (four in the encoder, one at the bottleneck, and four in the decoder), integrating multiscale attention into every significant stage of down- and up-sampling.

3. Mathematical Formulation

Formally, the CSA operation for a feature map $S=2$6 is:

• Channel splitting: $S=2$7
• Branch transformations as above
• Fusion: $S=2$8
• Channel statistics: $S=2$9
• Attention MLP: $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$0; $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$1 split as $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$2
• Softmax weights: $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$3
• Weighted sum: $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$4
• Residual: $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$5, where $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$6 is identity or a $X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$7 conv.

All convolutions employ BatchNorm and ReLU except at the final attention softmax and residual sum. No dropout is used in CSA blocks due to lack of benefit observed on medical segmentation tasks.

4. Hyperparameters and Layer Configuration

Key configuration parameters for a CSA block include:

Component	Setting	Purpose
Number of splits ($X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$8)	2	Channel grouping
Branch convolutions	1$X_1, X_2 \in \mathbb{R}^{H\times W\times (C/2)}$91, then 3$$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$03 (BN+ReLU)	Local feature extraction
Extra 3$$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$13 in Branch 2	Applied after $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$2	Expanded context/receptive
Attention MLP	Two 1$$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$31 convs, hidden $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$4	Split-channel weighting
Residual	Direct or $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$5 conv as needed	Feature preservation

Empirical settings are $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$6, $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$7 (with hidden size $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$8 for the attention MLP), and total CSA parameter count is approximately 2.6M with $$F_1(X_1) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_1))))))$$9. Placing all CSA blocks into DCSAU-Net yields a total parameter count of ~2.6M, compared to ~13.4M for vanilla U-Net, and reduces FLOPs from 31.11G to 6.91G on CVC-ClinicDB data.

5. Interaction with Primary Feature Conservation (PFC)

While not strictly a component of the CSA block, the PFC module operates synergistically within DCSAU-Net. The PFC module sits at the front of the U-Net:

• $$F_2'(X_2) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_2))))))$$0 conv (stride 2) for initial downsampling and channel extension,
• $$F_2'(X_2) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_2))))))$$1 depthwise conv followed by $$F_2'(X_2) = \text{ReLU}(\text{BN}(\text{Conv}_{3\times 3}(\text{ReLU}(\text{BN}(\text{Conv}_{1\times 1}(X_2))))))$$2 pointwise conv (each with BN+ReLU),
• Residual skip-connection preserving high-frequency, low-level features.

This PFC output feeds into the first CSA, ensuring that original spatial and high-resolution features are modulated—never discarded—across the network's depths. Residual connections within CSA blocks further preserve original feature information throughout the architecture.

6. Ablation Results and Empirical Impact

Ablation studies (Sect. 4.5, (Xu et al., 2022)) isolate the contribution of CSA blocks. On four major biomedical segmentation datasets, adding CSA blocks to U-Net, without PFC, yielded the following mIoU and F1 improvements:

Dataset	mIoU Gain (+CSA)	F1 Gain (+CSA)
CVC-ClinicDB	+3.1 p.p.	+1.8 p.p.
2018 DataSci Bowl	+3.1 p.p.	+2.1 p.p.
ISIC-2018	+2.8 p.p.	+1.9 p.p.
SegPC-2021	+1.5 p.p.	+1.5 p.p.

Combining both PFC and CSA consistently yielded an additional ∼1–2 percentage point improvement. Qualitative results (Figs. 7, 8 of (Xu et al., 2022)) demonstrate accelerated convergence (~20 epochs) and enhanced ability to delineate fine-scale detail, relative to other state-of-the-art segmentation models.

7. Theoretical and Practical Implications

The CSA block, by employing two-branch groupwise convolutions, global context extraction via pooling, lightweight attention with softmax weighting, and pervasive residual connections, achieves both increased representational capacity and efficient parameter usage. Placing CSA blocks after all resolution-changing stages enables DCSAU-Net to optimally fuse low-level and high-level semantic cues. Empirically, this design produces robust performance improvements across diverse biomedical segmentation tasks while markedly reducing computational overhead, suggesting broader applicability for resource-constrained or real-time applications (Xu et al., 2022).