Enhanced Attention U-Net Architecture

• Enhanced Attention U-Net is an encoder-decoder network that integrates spatial and channel attention modules with an input image pyramid to improve feature integration.
• It employs deep supervision along with a Focal Tversky loss to address class imbalance and enhance detection of small structures.
• Empirical evaluations on BUS 2017 and ISIC 2018 datasets demonstrate significant Dice improvements, validating its multi-scale and attention-based innovations.

An enhanced Attention U-Net architecture denotes U-Net–based encoder–decoder networks in which classical skip connections are augmented with spatial or channel attention modules, often combined with additional multi-scale feature processing or residual blocks, to focus feature integration on semantically informative regions and address practical challenges such as class imbalance, multi-scale structure, or small target segmentation. This entry details the innovations, design principles, and empirical impacts of enhanced Attention U-Net variants, drawing primarily from the multi-scale pyramid– and focal Tversky–augmented design of Abraham and Khan (Abraham et al., 2018), but also noting methodological directions pursued in contemporary works.

1. Architectural Enhancements: Multi-Scale Pyramid and Attention Gates

The enhanced Attention U-Net extends standard U-Net’s symmetrical encoder–decoder structure and skip connections in several key respects:

• Attention Gates (AGs): At each decoder stage, AGs compute soft, spatially varying gating coefficients $\alpha_i \in [0,1]$ for each location $i$ by integrating encoder features $x^l_i$ with a decoder “gating signal” $g_i$:

$$q^\mathrm{attn}_l(x^l_i,g_i) = \psi^T[\mathrm{ReLU}(W_x^T x^l_i + W_g^T g_i + b_g)] + b_\psi,\quad \alpha^l_i = \sigma_2(q^\mathrm{attn}_l(x^l_i,g_i)),\quad \hat x^l_i = \alpha^l_i \cdot x^l_i$$

Pruned features $\hat x^l$ are then concatenated with upsampled decoder features, suppressing irrelevant background responses.

• Input Image Pyramid: At each encoder block, in addition to convolutional outputs, the original input is down-sampled to the block’s spatial scale and concatenated to the feature map. This multi-scale injection (input pyramid) preserves fine-grained details across resolutions and addresses information loss from repeated downsampling. The pyramid is particularly critical for small objects (e.g., lesions occupying ≈5% of the image) (Abraham et al., 2018).
• Deep Supervision: All decoder outputs, not just the final prediction, are supervised using auxiliary heads via suitable loss functions. Intermediate outputs employ the Focal Tversky loss (see below), while the last is trained with the standard Tversky loss.

This design ensures feature re-use at multiple scales, improved small structure recall, and training stability by safeguarding against vanishing gradients near optimal predictions.

2. Generalized Focal Tversky Loss: Balancing Precision–Recall

Rather than using a simple Dice or cross-entropy loss, the enhanced Attention U-Net employs a generalized Focal Tversky loss tailored for severe class imbalance and small targets. The formulation proceeds as follows:

• Dice Coefficient (DSC):

$$\mathrm{DSC}_c = \frac{\sum_i p_{i,c}g_{i,c} + \epsilon}{\sum_i p_{i,c} + \sum_i g_{i,c} + \epsilon}$$

• Tversky Index:

$$\mathrm{TI}_c = \frac{\sum_i p_{i,c}g_{i,c} + \epsilon}{\sum_i p_{i,c}g_{i,c} + \alpha\sum_i p_{i,\bar{c}}g_{i,c} + \beta\sum_i p_{i,c}g_{i,\bar{c}} + \epsilon}$$

Parameters $\alpha, \beta \geq 0$ balance the penalty between false negatives and false positives, with $\alpha = \beta = 0.5$ reducing to the Dice score. $\alpha> \beta$ increases recall at the cost of precision.

• Focal Tversky Loss:

$$\mathrm{FTL} = \sum_c (1 - \mathrm{TI}_c)^{1/\gamma},\quad \gamma > 1$$

When $\gamma>1$, gradients focus on harder examples (low-TI), and easy predictions have reduced influence. The optimal configuration found is $\alpha=0.7$, $\beta=0.3$, and $\gamma=\frac{4}{3}$.

• Loss Assignment: Deep and intermediate decoder heads use Focal Tversky, final output uses pure Tversky. This prevents vanishing gradients and encourages discriminative representations across scales (Abraham et al., 2018).

3. Implementation and Training Regimen

The enhanced architecture is constructed as follows:

• Encoder: Four downsampling blocks, each block: two 3×3 convolutions (ReLU), pyramid-injected input, 2×2 max-pool.
• Decoder: Four upsampling stages (2×2 up-conv), concatenation with attention-gated encoder features, two 3×3 convolutions (ReLU).
• Attention gates: Inserted at all skip connections, except the very first (highest resolution) skip (Abraham et al., 2018).
• Deep supervision heads: 1×1 convolution + sigmoid at each decoder level.
• Optimization: SGD with momentum (LR=1e-2, decay 1e-6/epoch), batch size 16 (BUS 2017), 8 (ISIC 2018), 100 and 50 epochs respectively, no data augmentation or transfer learning.

4. Empirical Results and Ablation Analysis

Evaluation on small (BUS 2017) and moderate (ISIC 2018) lesion datasets demonstrates the independent and synergistic effects of each architectural component:

Model	BUS 2017 Dice (±std)	ISIC 2018 Dice (±std)
U-Net+Dice	0.547 ± 0.04	0.820 ± 0.013
U-Net+Tversky	0.657 ± 0.02	0.838 ± 0.026
U-Net+Focal Tversky	0.669 ± 0.033	0.829 ± 0.027
Attn U-Net+Dice	0.615 ± 0.020	0.806 ± 0.033
Attn+MultiInput+Dice	0.716 ± 0.041	0.827 ± 0.055
Attn+Multi+Tversky	0.751 ± 0.042	0.841 ± 0.012
Attn+Multi+Focal Tversky	0.804 ± 0.024	0.856 ± 0.007

Key conclusions:

• Input image pyramid yields ≈10% Dice gain for small lesions (BUS 2017)
• Loss re-weighting (FTL) provides ~5% additional gain
• The combination of all enhancements yields a 25.7% relative Dice gain over plain U-Net on BUS, and 3.6% on ISIC (Abraham et al., 2018)

Contribution analysis reveals that attention gates alone are of limited benefit on extremely small targets unless accompanied by the pyramid and FTL. Deep supervision stabilizes optimization and accelerates convergence.

5. Generalization and Broader Impact

The two core principles—multi-scale pyramid input and the Focal Tversky loss—are applicable to any segmentation task characterized by:

• Severe class imbalance (e.g., small ROIs, vessel or calcification segmentation)
• Multi-scale object appearance
• Need for tunable precision–recall trade-offs

Practitioners can retain local context (image pyramid input), suppress irrelevant background (attention gating), and control loss gradient focus (tunable $\alpha$, $\beta$, $\gamma$ in FTL) for robust performance across domains.

The enhanced Attention U-Net represents a systematic, well-validated methodology for producing sharper, more sensitive, and class-imbalance–resilient segmentation models. Detailed ablation evidence confirms that each design decision independently delivers measurable improvements and, in concert, achieves a model that is both accurate and more robust than standard U-Net or naive attention variants (Abraham et al., 2018).