GRAD-Former: Robust Change Detection

• GRAD-Former is a Siamese encoder-decoder framework that fuses bitemporal VHR satellite images to produce precise binary change maps.
• The model integrates a gated AFRAR module and differential attention to amplify meaningful features while filtering out irrelevant noise.
• It outperforms CNN, transformer, and SSM-based methods on benchmarks like LEVIR-CD, DSIFN-CD, and CDD with improved boundary delineation and efficiency.

Searching arXiv for the GRAD-Former paper and closely related change-detection context. GRAD-Former, short for Gated Robust Attention-based Differential Transformer, is a Siamese encoder-decoder framework for remote sensing change detection (CD) that operates on bitemporal satellite images and predicts a binary map of changed versus unchanged pixels. It was introduced to address the difficulty of precisely delineating change regions in very high-resolution (VHR) imagery, where background clutter, structurally irregular changes, and pseudo-changes caused by shadows, illumination, seasonality, vehicle movement, and roof color variations can obscure semantic differences. The method combines a shared encoder, an Adaptive Feature Relevance and Refinement (AFRAR) module, a Differential Amalgamation (DA) fusion block, and a lightweight decoder, with the stated goal of enhancing contextual understanding while maintaining efficiency through reduced model size (Ameta et al., 1 Mar 2026).

1. Problem setting and design motivation

Remote sensing CD seeks to identify semantic differences between satellite images captured at different times. In the formulation used by GRAD-Former, the task is to produce a binary change map from a pre-change image and a post-change image. The problem is especially difficult in VHR scenes because relevant changes may be small, sparse, and boundary-sensitive, while irrelevant appearance variations can dominate the visual signal (Ameta et al., 1 Mar 2026).

The paper situates GRAD-Former against three established model families. CNN-based methods are described as effective for local spatial feature extraction but limited in modeling long-range dependencies and global contextual reasoning, which can impair sensitivity to subtle or spatially distributed changes. Transformer-based methods improve global modeling through self-attention, but in VHR remote sensing they incur quadratic computational complexity, high memory usage, large parameter counts, weak local inductive bias, and a diffused attention problem in which attention spreads too broadly and captures irrelevant background. Selective State Space Models (SSMs) such as Mamba are presented as linearly scaling alternatives that are strong at long-range sequence modeling but still challenged by fine boundary delineation and local-global balancing (Ameta et al., 1 Mar 2026).

GRAD-Former is designed around a specific response to these limitations: it aims to filter out irrelevant information early, amplify meaningful embeddings, and refine features using sparse differential attention, while avoiding heavy full-width self-attention and not relying on a pretrained backbone. This positioning is central to understanding why the architecture combines gating, channel splitting, reduced-channel attention, and multi-scale differential fusion rather than adopting a conventional transformer encoder wholesale (Ameta et al., 1 Mar 2026).

2. Overall architecture

At the highest level, GRAD-Former uses a shared Siamese encoder for the two input images, followed by stage-wise fusion and a decoder. The encoder produces multi-scale features at 4 stages,

$$\hat{\mathcal{F}}_{\text{pre}}^i,\ \hat{\mathcal{F}}_{\text{post}}^i,\quad i \in \{1,2,3,4\},$$

and these paired features are fused at each stage by the DA module before being passed to the decoder (Ameta et al., 1 Mar 2026).

The encoder stages use channel widths of 64, 96, 128, 256. Each stage contains patch embedding/downsampling, an AFRAR module, and a convolutional MLP layer. Before AFRAR and the MLP, the input undergoes instance normalization across channels. The encoder block counts are reported as 3, 3, 4, 3, with an input image size of 256 × 256 × 3 during training and evaluation (Ameta et al., 1 Mar 2026).

AFRAR is the central architectural component. For a normalized feature tensor at stage $i$,

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$

the tensor is split channel-wise into two equal parts,

$$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$

which are processed by two complementary branches: Selective Embedding Amplification (SEA) and Global-Local Feature Refinement (GLFR). Their outputs are then concatenated,

$$\bar{\mathcal{F}}_{\text{norm}}^i = \text{Concat}(\bar{\mathcal{F}}_{\text{SEA}}^i,\ \bar{\mathcal{F}}_{\text{GLFR}}^i).$$

This split-branch structure is the mechanism by which GRAD-Former distributes local/channel relevance processing and global-local contextual refinement across separate but coordinated operators (Ameta et al., 1 Mar 2026).

3. AFRAR: selective amplification and differential attention

The SEA branch is a gating-based channel reweighting module intended to amplify channels containing useful change information while suppressing noisy or irrelevant channels. Given $\mathcal{F}_{\text{SEA}}^i$, SEA first applies $L_2$ normalization with a stabilizer $\epsilon$, scaled by a learnable scalar $\alpha$,

$$E = \alpha \cdot \sqrt{\sum_{H,W}\left(\mathcal{F}_{\text{SEA}}^i\right)^2 + \epsilon}.$$

It then computes an RMS-based normalization factor using a learnable scalar $i$0,

$i$1

followed by a gated activation with learnable bias $i$2,

$i$3

The output is

$i$4

The paper interprets this as selectively boosting relevant channel responses while attenuating irrelevant ones, with the $i$5 gate used to stabilize scaling behavior (Ameta et al., 1 Mar 2026).

The GLFR branch provides the attention mechanism in AFRAR and is explicitly described as being inspired by differential attention. From the input $i$6, the branch forms

$i$7

with reduced channel dimensions: $i$8 and $i$9. The query and key tensors are then split,

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$0

so that

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$1

These tensors are reshaped for multi-head attention using $$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$2 heads and

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$3

Two softmax attention maps are computed,

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$4

and combined through subtraction,

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$5

where $$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$6 is learnable and defined by

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$7

with

$$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$8

and $$\mathcal{F}_{\text{norm}}^i \in \mathbb{R}^{H^i \times W^i \times C^i},$$9. The attention output is then

$$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$0

After reshaping, the refined attention features are concatenated with local features,

$$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$1

The paper’s interpretation is that the subtraction $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$2 produces sparser attention, canceling common or redundant patterns and suppressing distractors so that meaningful change-related responses remain (Ameta et al., 1 Mar 2026).

A common misunderstanding is to read GRAD-Former as a standard transformer variant with full self-attention over all channels. The architecture described in the paper is narrower and more selective: channels are split, attention is applied only to a reduced channel subset, and local features are explicitly retained through concatenation. In that sense, the “Transformer” component is structurally constrained by the model’s efficiency objective rather than used in its canonical dense form (Ameta et al., 1 Mar 2026).

4. Differential fusion, decoder, and optimization

After encoding, GRAD-Former uses Differential Amalgamation (DA) to fuse the pre-change and post-change features at each stage. For stage $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$3, the fusion is

$$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$4

where $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$5 is GELU and the $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$6 convolution reduces channels. This formulation explicitly injects both semantic content from the two temporal states and their direct difference, while keeping the fusion lightweight (Ameta et al., 1 Mar 2026).

The decoder consumes the four fused feature maps $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$7, concatenates them along channels, applies a $$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$8 convolution, upsamples twice using transposed convolutions, refines twice with residual blocks, and produces 2-channel prediction scores. An argmax over the channel dimension yields the final binary change map. The decoder is described as preserving full image resolution (Ameta et al., 1 Mar 2026).

Training uses pixel-wise cross-entropy loss,

$$\mathcal{F}_{\text{GLFR}}^i,\ \mathcal{F}_{\text{SEA}}^i = \text{Split}(\mathcal{F}_{\text{norm}}^i),$$9

where $$\bar{\mathcal{F}}_{\text{norm}}^i = \text{Concat}(\bar{\mathcal{F}}_{\text{SEA}}^i,\ \bar{\mathcal{F}}_{\text{GLFR}}^i).$$0 is the ground-truth label, $$\bar{\mathcal{F}}_{\text{norm}}^i = \text{Concat}(\bar{\mathcal{F}}_{\text{SEA}}^i,\ \bar{\mathcal{F}}_{\text{GLFR}}^i).$$1 is the predicted probability, and $$\bar{\mathcal{F}}_{\text{norm}}^i = \text{Concat}(\bar{\mathcal{F}}_{\text{SEA}}^i,\ \bar{\mathcal{F}}_{\text{GLFR}}^i).$$2 is the number of pixels. The paper states that cross-entropy outperforms focal loss and mIoU loss for this model (Ameta et al., 1 Mar 2026).

The implementation is in PyTorch and training is performed on a single NVIDIA RTX A4000 GPU with 16 GB memory. Optimization uses AdamW with an initial learning rate of 0.0001, a linear decay schedule with decay steps at 100 and 200 epochs, 400 epochs of total training, and batch size 2. Data augmentation consists of random flipping, scaling, cropping, color jittering, and Gaussian blur (Ameta et al., 1 Mar 2026).

5. Datasets, evaluation protocol, and reported performance

GRAD-Former is evaluated on three CD benchmarks: LEVIR-CD, DSIFN-CD, and CDD. LEVIR-CD contains 637 high-resolution image pairs, with 0.5 m/pixel resolution and original image size 1024 × 1024, cropped to 256 × 256; the split is 7120 train, 1024 val, and 2048 test. DSIFN-CD contains 3988 bitemporal image pairs from Xi’an, Chongqing, Beijing, Chengdu, Wuhan, Shenzhen, cropped to 256 × 256, with split 14,400 train, 1,360 val, and 192 test. CDD contains 11 raw image pairs, at resolutions from 0.03 to 1 m/pixel, including seasonal and disaster-related changes, cropped to 256 × 256, with split 10,000 train, 3,000 val, and 3,000 test (Ameta et al., 1 Mar 2026).

Evaluation uses F1 score, IoU, and OA (overall accuracy), with IoU and F1 computed on the change class. The paper reports that GRAD-Former outperforms the compared methods on all three datasets (Ameta et al., 1 Mar 2026).

Dataset	Reported performance	Comparison statement
CDD	F1 = 97.57%, IoU = 95.26%, OA = 99.43%	Best among all compared methods
DSIFN-CD	F1 = 93.14%, IoU = 87.16%, OA = 97.65%	Best among all compared methods
LEVIR-CD	F1 = 91.52%, IoU = 84.36%, OA = 99.14%	Best among all compared methods

The comparison set includes CNN-based methods such as FC-EF, FC-Siam-diff, FC-Siam-conc, DTCDSCN, SNUNet, ISNet, and SARAS-Net; Transformer-based methods such as BIT, ChangeFormer, ScratchFormer, ELGC-Net, Meta-SGNet, RAHFF-Net, AMDANet, CICD, and AdvCP; and Mamba-based methods such as ChangeMamba, RS-Mamba, CDMamba, and MF-VMamba. The paper highlights three margins in particular: on CDD, GRAD-Former surpasses ScratchFormer by +1.45 F1 and +2.72 IoU; on DSIFN-CD, it outperforms ChangeMamba by +2.93 F1 and about +5.0 IoU; and on LEVIR-CD, it exceeds CICD by +0.49 IoU and +0.10 OA (Ameta et al., 1 Mar 2026).

The paper also includes a parameter and complexity comparison on DSIFN-CD. GRAD-Former is reported with 10.90M parameters, 129.50 GFLOPs, and F1 93.14, compared with ChangeFormer at 41.03M parameters, 211.15 GFLOPs, F1 87.34; ChangeMamba at 85.53M parameters, 179.32 GFLOPs, F1 90.21; and MF-VMamba at 57.84M parameters, 94.06 GFLOPs, F1 88.05. The authors use these results to argue that GRAD-Former achieves strong accuracy with a moderate model size (Ameta et al., 1 Mar 2026).

6. Ablations, qualitative behavior, and limitations

The ablation studies attribute measurable gains to each major module. On LEVIR-CD, using SEA only yields IoU 79.69 and F1 87.93; GLFR only yields IoU 82.92 and F1 90.63; DA only yields IoU 80.30 and F1 88.90; and the full combination SEA + GLFR + DA yields IoU 84.36, F1 91.52, OA 99.14, and 10.90M parameters. The paper states that these results confirm that all components contribute positively and that the full model is best (Ameta et al., 1 Mar 2026).

A second ablation compares attention mechanisms inside GLFR on LEVIR-CD. Simple Attention gives F1 84.32 and IoU 72.89; Pooled-Transpose Attention gives F1 91.08 and IoU 83.62; and Differential Attention gives F1 91.52 and IoU 84.36. The paper concludes that differential attention is best in both accuracy and efficiency. A third study reports that Cross-Entropy Loss performs better than Focal Loss and that mIoU Loss is much worse and unstable (Ameta et al., 1 Mar 2026).

The qualitative findings reported by the authors are that GRAD-Former reduces false positives and false negatives, produces sharper and cleaner boundaries, detects small changes more reliably, ignores seasonal and lighting-induced pseudo-changes, and preserves semantic consistency across objects. The method is described as performing particularly well on small objects like vehicles, narrow structures such as streams, and mixed scenes with both large and tiny change regions (Ameta et al., 1 Mar 2026).

The paper does not discuss major explicit limitations, but it does indicate future directions including further improving efficiency, adaptation for real-time change detection, and deployment on edge devices. A plausible implication is that, although GRAD-Former is more efficient than many transformer-based baselines, it remains a model with a transformer-style attention component and is not extremely lightweight for edge deployment yet. That implication aligns with the paper’s own emphasis on reduced model size rather than minimum possible deployment cost (Ameta et al., 1 Mar 2026).