---
title: 'AMD-HookNet++: Hybrid CNN-Transformer for Glacier Segmentation'
url: https://www.emergentmind.com/topics/amd-hooknet
type: topic
---

# AMD-HookNet++: Hybrid CNN-Transformer for Glacier Segmentation

AMD-HookNet++ is a hybrid convolutional neural network (CNN)–Transformer framework designed for glacier segmentation and calving front delineation in synthetic aperture radar (SAR) imagery. Building on the two-branch “HookNet” paradigm originally developed for histopathology image segmentation, AMD-HookNet++ integrates Transformer-based global context modeling with CNN-driven local feature extraction. Its architecture introduces a feature fusion mechanism based on enhanced spatial-channel attention (ESCA) and employs pixel-to-pixel contrastive deep supervision, setting a new state of the art on the CaFFe glacier benchmark [2512.14639].

## 1. Principle and Model Architecture

AMD-HookNet++ generalizes the original two-branch HookNet design by pairing a Swin-UNet Transformer for context with a U-Net CNN for high-resolution target features. Each branch processes a different spatial resolution derived from the same 224×224×3 SAR input patch: the context branch receives a downsampled “coarse” view, and the target branch a center-cropped, higher-resolution patch at twice the context resolution ($r_t = 2 r_c$).

**Transformer Context Branch (Swin-UNet):**
- Patch embedding via $4 \times 4$ convolution to $56 \times 56 \times 96$.
- Four Swin Transformer stages implementing alternating window-based (W-MSA) and shifted-window (SW-MSA) self-attention, with residuals and MLPs:
  $$
  \hat{z}^l = \mathrm{W\text{-}MSA}(\mathrm{LN}(z^{l-1})) + z^{l-1}, \;
  z^l = \mathrm{MLP}(\mathrm{LN}(\hat{z}^l)) + \hat{z}^l
  $$
- Output reshaped to $224 \times 224 \times 4$ logits.

**CNN Target Branch (U-Net):**
- Nine convolutional blocks: $3 \times 3$ Conv $\rightarrow$ BN $\rightarrow$ ReLU, MaxPool (encoder) or ConvTranspose (decoder).
- Channel progression: $32\rightarrow64\rightarrow128\rightarrow256\rightarrow320 \rightarrow256\rightarrow128\rightarrow64\rightarrow32$.
- Final $1\times1$ convolution for four output semantic classes.

**Feature Exchange and Matching:**
- At two depths (Swin stages 6–7 and CNN conv-blocks 5–6), features are aligned using cropping or up/downsampling to ensure spatial correspondence before fusion via ESCA.

## 2. Enhanced Spatial-Channel Attention (ESCA) Fusion

The ESCA module enables expressive and adaptive fusion of features between the Transformer and CNN branches beyond simple concatenation or single-head self-attention.

**Spatial Attention:**
- Crop context features, concatenate with target features, yielding tensor $M\in\mathbb{R}^{H'\times W'\times C''}$.
- Channel-independent $q, k, v$ generated by depth-wise convolutions:
  $$
  q,k,v = \mathrm{DwConv}(M)
  $$
- Attention-weighted combination:
  $$
  A_s = \mathrm{Softmax}(q k^\top/\sqrt{d}) v;\quad \widetilde{M} = M + \theta A_s
  $$
  with $\theta$ learnable (residual).

**Channel Attention:**
- Reshape $\widetilde{M}$ to $H'W'\times C''$.
- Learn spatially independent affinity $U\in\mathbb{R}^{H'W'\times C''}$.
- Channelwise refinement: $U' = \mathrm{Softmax}(U, \mathrm{dim}=C'')$, $A_c = U' \times \widetilde{M}$.

**Final Fusion:**
- $1\times1$ convolution restores the hooked feature:
  $$
  \mathrm{ESCA\_hook}(F_c, F_t) = \mathrm{Conv}_{1\times1}(A_c)
  $$
- LayerNorm and ReLU applied within convolutional steps.

ESCA consistently outperforms CBAM and single-head fusions by more effectively leveraging both spatial and channel interactions, yielding increased IoU (+0.6%) and improved Hausdorff distance compared to baseline fusions.

## 3. Pixel-to-Pixel Contrastive Deep Supervision

AMD-HookNet++ exploits deep supervision at intermediate decoder levels using a pixelwise contrastive (NCE-style) loss. Denoting pixel $i$’s embedding as $i$, the contrastive loss at decoder depths $D=1,2$ is:
$$
\mathcal{L}_i^{\mathrm{NCE}} = -\frac{1}{|\mathcal{P}_i|}\sum_{i^+\in\mathcal{P}_i}
\log\frac{\exp(i\cdot i^+/\tau)}{\exp(i\cdot i^+/\tau)
+ \sum_{i^-\in\mathcal{N}_i} \exp(i\cdot i^-/\tau)}
$$
where $\mathcal{P}_i$/$\mathcal{N}_i$ are positive/negative sets, and $\tau$ is a temperature parameter.

**Total loss:**
$$
\mathcal{L} =
\lambda_1 [\mathrm{CE}(p_t, y_t) + \mathrm{Dice}(p_t, y_t)]
+ \lambda_2 [\mathrm{CE}(p_c, y_c) + \mathrm{Dice}(p_c, y_c)]
+ \lambda_3 \mathcal{L}_{\mathrm{cds}}
$$
with $\lambda_1 = \lambda_2 = 1$, $\lambda_3 = 0.5$. Here, $(p_t, y_t)$ and $(p_c, y_c)$ denote logits/labels for target and context outputs, respectively.

Contrastive supervision encourages discriminative intermediate embeddings, empirically stabilizing training and reducing mean distance error (MDE) by approximately 13.6%.

## 4. Training Protocol and Implementation

**Data and Preprocessing:**
- Dataset: CaFFe (681 SAR images, 7 glaciers spanning Greenland, Antarctic Peninsula, Alaska), split into 559 train and 122 test images. Annotations span “ocean + ice-melange”, “rock outcrop”, “glacier”, and “NA-area.”
- Input: Patches of $224\times224$; target patches use sliding window, context from downsampling.
- Augmentation: Random rotations to [0,360°], flips (p=0.5), standard normalization.

**Optimization:**
- Optimizer: SGD, momentum 0.9, weight decay $1\times10^{-4}$.
- Learning rate: initial 0.01, exponential decay (0.9 per epoch).
- 130 epochs, batch size 170.
- Context branch (Swin-UNet) initialized from ImageNet-pretrained weights.
- PyTorch implementation on single NVIDIA A100 GPU.

## 5. Empirical Performance

**“Zones” Segmentation (mean ± std over five runs):**
- Precision: $87.9 \pm 0.2\%$
- Recall: $86.3 \pm 0.5\%$
- F1-score: $86.3 \pm 0.3\%$
- IoU: $78.2 \pm 0.4\%$ (up 2.7% vs. HookFormer, 3.8% vs. AMD-HookNet, 8.5% vs. CaFFe baseline)

**Calving Front Delineation:**
- MDE: $367 \pm 30\,\mathrm{m}$ (on par with HookFormer’s $353 \pm 16\,\mathrm{m}$)
- HD95: $1\,318 \pm 115\,\mathrm{m}$ ($\downarrow$ 3.8% vs. HookFormer’s $1\,370 \pm 75\,\mathrm{m}$)
- Zero missed-front cases ($0/122$ test images)

**Summary Table: Calving Front Metrics**  
| Method                | IoU (%) | MDE (m) | HD95 (m) |
|-----------------------|---------|---------|----------|
| CaFFe baseline        | 69.7    | 753     | 2180     |
| AMD-HookNet (CNN)     | 74.4    | 438     | 1631     |
| HookFormer (ViT)      | 75.5    | 353     | 1370     |
| Trans-UNet (hybrid)   | 66.0    | 574     | 1836     |
| **AMD-HookNet++**     | **78.2**| 367     | **1318** |

Qualitatively, AMD-HookNet++ produces smoother and more geophysically plausible calving front delineations than pure-Transformer methods, which are prone to jittery, jagged boundaries in noisy SAR imagery.

## 6. Analysis and Limitations

The combination of Transformer branch (global context) and CNN branch (local continuity) accounts for the significant segmentation improvements. ESCA avoids uniform attention limitations of CBAM or single-head self-attention by decoupling spatial and channel recalibration, and contributes to superior IoU and HD95.

Contrastive deep supervision ensures intermediate pixel embeddings are more class-discriminative, improving both MDE and training stability.

Despite incurring greater computational cost (22 GFLOPs vs. 15 GFLOPs for HookFormer), AMD-HookNet++ achieves higher throughput (165 img/s vs. 91 img/s) due to efficient CNN decoding. The primary limitation is reliance on ImageNet pretraining, suggesting a need for large-scale, domain-specific foundation models for SAR applications.

## 7. Extensions and Prospective Directions

Prospective research avenues include:
- Extending ESCA fusion architecture to other dual-branch domains (e.g., multi-modal fusion in remote sensing).
- Pretraining on large-scale unlabeled SAR corpora using self-supervised learning objectives (e.g., SSL4SAR).
- Adapting AMD-HookNet++ for multi-class glacier zone segmentation and processing of three-dimensional radar echo data.

These developments indicate the method’s broad relevance for SAR-based environmental monitoring and geophysical analysis, beyond glacier segmentation [2512.14639].

Source: https://www.emergentmind.com/topics/amd-hooknet