---
title: Remote Sensing Change Captioning
url: https://www.emergentmind.com/topics/remote-sensing-change-caption-rscc
type: topic
---

# Remote Sensing Change Captioning

Remote Sensing Change Caption (RSCC), also called Remote Sensing Image Change Captioning (RSICC), is the task of generating natural-language descriptions of semantic changes between two co-registered remote sensing images of the same area acquired at different times. It converts bi-temporal visual evidence—such as buildings constructed or demolished, roads extended, vegetation cleared, facilities added, or disaster damage—into captions that must be fluent, spatially grounded, and semantically precise. The topic sits at the intersection of change detection, semantic scene understanding, and vision-language generation: unlike binary or semantic change detection, it must explain changes in open-ended language, and unlike single-image captioning, it must reason explicitly over temporal differences rather than static content [2312.15311][2605.15024][2406.13424].

## 1. Task definition and conceptual boundaries

RSCC commonly assumes two inputs, $I_1$ and $I_2$, representing the same geographic region at times $t_1$ and $t_2$, and a caption $Y=\{y_1,\dots,y_T\}$ describing the change. A representative formulation conditions caption generation on both bi-temporal features and localized change evidence, as in
$$
L_{\mathrm{cap}}=-\sum_{t=1}^{T}\log p(y_t \mid y_{<t}, F_1, F_2, M),
$$
where $M \in \{0,1\}^{H\times W}$ denotes a change map [2312.15311]. Other formulations model the conditional distribution $p(Y \mid I_{t1}, I_{t2})$ directly and augment token-level generation with contrastive or auxiliary objectives [2606.27410].

The field is defined as distinct from both Binary Change Detection and Semantic Change Detection. Binary Change Detection answers where change occurs, typically by predicting a mask $M \in \{0,1\}^{H\times W}$, while Semantic Change Detection assigns predefined class labels to changed regions. RSCC instead requires open-ended language capable of expressing richer semantics, object attributes, spatial relations, counts, and temporal dynamics [2605.15024]. It is also distinct from conventional remote-sensing image captioning, because the output must isolate differences between times rather than summarize a single scene [2406.13424].

Several persistent difficulties recur across the literature. Scenes are often dense and heterogeneous, with subtle or low-contrast changes, illumination variability, atmospheric effects, seasonal appearance shifts, and sensor or regional domain shift. Captions must not only identify changed objects but also localize them with phrases such as “on the right side” or “in the north,” sometimes while distinguishing changed from unchanged context [2312.15311]. Later work frames an additional difficulty as semantic heterogeneity between changed and unchanged pairs: unchanged samples are coarse-grained and often reducible to “no change,” whereas changed samples require fine-grained description, so a unified modeling strategy can entangle coarse change-existence judgment with token-level semantics [2605.15024]. Another line of work argues that purely autoregressive training can bias models toward easy, frequent vocabulary instead of discriminative difference cues [2606.27410].

## 2. Architectural evolution

Early RSCC systems adapted change-captioning designs from natural-image and video settings. Representative baselines include Capt-Rep-Diff, Capt-Att, Capt-Dual-Att, and DUDA, which relied on CNN encoders and RNN-style or attention-based decoders but generally lacked explicit pixel-level grounding [2312.15311]. A major early remote-sensing-specific model, "Changes to Captions: An Attentive Network for Remote Sensing Change Captioning" [2304.01091], introduced Chg2Cap, which combines a Siamese ResNet-101 feature extractor, stacked hierarchical self-attention blocks, a cosine-similarity mask, and a Transformer decoder. Chg2Cap formalized a recurring pattern in the literature: shared-weight bi-temporal encoding, dedicated change-focused fusion, and autoregressive caption generation [2304.01091].

Transformer-based multi-scale modeling then became prominent. "Progressive Scale-aware Network for Remote sensing Image Change Captioning" [2303.00355] proposed PSNet as a pure Transformer architecture with Progressive Difference Perception layers for multi-scale differencing and a Scale-aware Reinforcement module that progressively injects different scale features into different decoder layers. SAT-Cap later reduced reliance on multi-stage fusion by introducing a Spatial-Channel Attention Encoder and a cosine-similarity-based Difference-Guided Fusion module, arguing for single-stage feature integration while reporting CIDEr scores of 140.23 on LEVIR-CC and 97.74 on DUBAI-CC [2501.08114].

Efficiency-oriented variants sought to reduce the computational burden of dense self-attention. "A Lightweight Sparse Focus Transformer for Remote Sensing Image Change Captioning" [2405.06598] proposed a Sparse Focus Transformer whose abstract reports a reduction of over 90% in parameters and computational complexity for the transformer encoder while retaining competitive performance. In parallel, "RSCaMa: Remote Sensing Image Change Captioning with State Space Model" [2404.18895] replaced attention-heavy spatiotemporal modeling with Mamba-based state space modules, introducing Spatial Difference-aware SSM and Temporal-Traversing SSM to achieve global receptive fields with linear complexity. This line established state-space modeling as a viable alternative to Transformer fusion for RSCC [2404.18895].

Diffusion models expanded the design space further. "Diffusion-RSCC: Diffusion Probabilistic Model for Change Captioning in Remote Sensing Images" [2405.12875] reframed caption generation as denoising continuous caption embeddings conditioned on bi-temporal features. "Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning" [2412.19179] instead shifted emphasis from conventional feature learning toward data distribution learning, using a diffusion model to refine change-mask-like representations before caption generation. These models explicitly target robustness to pixel-level nuisance variation, although they also inherit the cost of iterative sampling [2405.12875][2412.19179].

A separate development replaced manual fusion modules with pretrained video encoders. "MV-CC: Mask Enhanced Video Model for Remote Sensing Change Caption" [2410.23946] treats the bi-temporal pair as a two-frame clip and uses InternVideo2 to extract spatiotemporal representations directly, then gates token grids with change masks. This formulation reframes RSCC as a short-video captioning problem with explicit region filtering rather than handcrafted temporal fusion [2410.23946].

## 3. Grounding change semantics and integrating auxiliary supervision

A central trend in modern RSCC is the use of auxiliary localization or semantic signals to improve grounding. "Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning" [2312.15311] made this relation explicit by generating pixel-level pseudo-labels with a pre-trained BIT change detector, training an auxiliary UNet-like CD branch with binary cross-entropy, and fusing its high-level change features into the captioning pathway through the Semantic Fusion Augment module. The method is premised on the idea that pixel-level CD is significant for describing the differences between images through language, especially in complex scenes [2312.15311].

Joint learning of detection and captioning was strengthened by foundation-model-based systems. "Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance" [2407.14032] combines a SAM-based bi-temporal encoder, a multi-task semantic aggregation neck, a multi-scale CD decoder, and a Vicuna-based caption decoder, trained with a three-stage schedule designed to stabilize CD/CC interaction. "ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing" [2410.10047] similarly treats change detection and change captioning as a unified end-to-end problem, using ChangeLSTM to build universal change-aware representations shared by a CD head and a caption head with cross-attention [2410.10047].

Other works integrate change semantics through contrastive or multimodal guidance. "DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning" [2606.27410] adds a Text-guided Gated Contrastive Loss that aligns visual difference embeddings with sentence-level text embeddings while excluding no-change samples from the contrastive objective. "Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework" [2501.10075] fuses RGB and semantic segmentation streams through Cross-Modal Cross Attention, Unimodal Difference Cross Attention, and Multimodal Gated Cross Attention, explicitly using semantic maps to suppress hallucinations in no-change or noisy regions [2501.10075].

Several recent systems push grounding beyond pixel-level detection alone. "SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning" [2511.21420] mines motion-level changed regions with SAM and SuperGlue, semantic regions with Grounding DINO plus SAM, and semantic priors from a relation graph encoded by R-GCN. "UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model" [2605.04409] introduces PTNet, which uses a learnable prototype bank to encode structured change semantics, multi-head gating to disentangle captioning and detection representations, and detection-derived spatial priors as tokens for caption generation. "HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning" [2605.15024] argues that changed and unchanged pairs require different semantic treatments and implements this with coarse image-level routing and fine-grained token-level Mixture-of-Experts. "CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset" [2411.11360] preserves the original MLLM feature pathway while injecting multi-scale difference-aware signals through a dedicated integration module [2511.21420][2605.04409][2605.15024][2411.11360].

Collectively, these systems indicate that RSCC has moved from coarse differencing toward explicit semantic grounding. This suggests that the field increasingly treats caption generation not as a standalone decoder problem but as a downstream language layer over progressively richer change representations.

## 4. Benchmarks, datasets, and supervision resources

The empirical development of RSCC has been shaped by a sequence of datasets that vary in scale, annotation type, modality, and application domain. LEVIR-CC remains the most commonly reported captioning benchmark; Dubai-CC and WHU-CDC broaden evaluation across scale and scene types; LEVIR-MCI and SECOND-CC introduce joint or semantic supervision; UCCD and the disaster-focused RSCC dataset extend the field into new operational regimes; CC-Foundation and RSRCC extend supervision through pretraining corpora and localized QA [2312.15311][2606.27410][2410.23946][2501.10075][2605.04409][2411.11360][2604.20623][2509.01907].

| Resource | Scale | Distinguishing features |
|---|---:|---|
| LEVIR-CC [2312.15311] | 10,077 pairs; 50,385 captions | 5,038 changed and 5,039 unchanged; five captions per pair |
| Dubai-CC [2606.27410] | 500 pairs; 2,500 captions | Early urban benchmark for RSCC |
| WHU-CDC [2511.21420] | 7,434 pairs; 37,170 sentences | Building, parking, road, and related urban changes |
| LEVIR-MCI [2410.23946] | 10,077 pairs | Captions plus CD masks for roads and buildings |
| SECOND-CC [2501.10075] | 6,041 pairs; 30,205 captions | RGB plus semantic segmentation maps; 30 class-to-class transitions |
| UCCD [2605.04409] | 9,000 pairs; 45,000 captions | UAV benchmark; 6 cm/pixel; urban construction focus |
| CC-Foundation [2411.11360] | 200,000 pairs; 1.2 million captions | Continued-pretraining corpus for MLLM-based RSCC |
| RSRCC [2604.20623] | 126k QA instances | Localized change question-answering benchmark |
| RSCC disaster dataset [2509.01907] | 62,351 pairs | Disaster-focused, multi-sentence captions, 31 global events |

Dataset design strongly influences what models learn. LEVIR-CC offers only sentence-level annotations, which motivated pseudo-label approaches such as Pix4Cap [2312.15311]. LEVIR-MCI supplies both captions and change masks, enabling joint detection-captioning systems such as MV-CC and ChangeMinds [2410.23946][2410.10047]. SECOND-CC adds semantic segmentation maps and includes explicit no-change pairs, making robustness to illumination, viewpoint shifts, blur, and registration error part of the benchmark itself [2501.10075]. UCCD moves the field toward UAV-based urban construction monitoring with 3024×4032 originals cropped to 1024×1024 and manually annotated change masks [2605.04409]. The disaster-oriented RSCC dataset adds 62,351 pre-/post-disaster pairs with an average caption length of 72 words, pushing the task from short object-change statements toward richer situational narratives [2509.01907].

Two resources extend RSCC beyond standard caption benchmarking. CC-Foundation supplies 200,000 image pairs and 1.2 million captions for continued pretraining of multimodal large models [2411.11360]. RSRCC reframes scene-level RSCC as localized, region-grounded change question answering with 126k questions, emphasizing that scene-level captions can be correct while still failing to localize or correctly attribute specific changes [2604.20623].

## 5. Objectives, evaluation metrics, and empirical performance

Most RSCC systems still optimize teacher-forced token-level cross-entropy for caption generation, but the training objectives around it have diversified. Pix4Cap jointly minimizes caption loss and binary cross-entropy for an auxiliary CD branch, with $L_{\text{total}}=L_{\text{cap}}+\lambda L_{\text{cd}}$ and $\lambda=1$ in the reported experiments [2312.15311]. DFM augments the autoregressive generation loss with a gated text-image contrastive objective,
$$
\min_\theta \ \alpha \mathcal{L}_{\mathrm{TGC}}+\beta \mathcal{L}_{\mathrm{gen}},
$$
to shape the visual encoder toward text-relevant differences [2606.27410]. Retrieval-oriented work adds InfoNCE-style image-text alignment alongside captioning loss, enabling a single model to support both bi-temporal captioning and text-image retrieval [2406.13424]. Multi-task systems combine caption loss with change-detection losses, often using cross-entropy or BCE for the detection branch [2407.14032][2410.10047].

Evaluation is dominated by BLEU-1/2/3/4, METEOR, ROUGE-L, and CIDEr or CIDEr-D [2312.15311][2606.27410]. Several papers summarize these with
$$
S_m^*=\frac{1}{4}\left(\text{BLEU-4}+\text{ROUGE}_L+\text{METEOR}+\text{CIDEr-D}\right),
$$
or an equivalent $S^*_m$ variant [2606.27410][2605.15024]. Some diffusion and QA-oriented works also report SPICE, BARTScore, MoverScore, BERTScore, or localized QA accuracy, reflecting a broader concern with semantic fidelity beyond $n$-gram overlap [2405.12875][2604.20623].

On LEVIR-CC, "Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning" [2312.15311] reported BLEU-4 63.78, METEOR 39.96, ROUGE-L 75.12, CIDEr 136.76, and $S^*_m$ 78.91. "DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning" [2606.27410] reported CIDEr-D 142.51, METEOR 40.95, ROUGE-L 75.90, BLEU-4 66.26, and $S^*_m$ 81.40. "CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset" [2411.11360] reported BLEU-4 65.49, METEOR 41.82, ROUGE-L 76.55, CIDEr-D 143.32, and $S^*$ 81.80. These results indicate that MLLM-based systems and contrastive difference modeling both pushed the benchmark above earlier Transformer and pseudo-label baselines [2312.15311][2606.27410][2411.11360].

On WHU-CDC, "HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning" [2605.15024] reported BLEU-4 76.52, METEOR 48.77, ROUGE-L 82.00, CIDEr-D 158.35, and $S_m^*$ 91.41. "SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning" [2511.21420] reported BLEU-4 74.42 and CIDEr-D 156.21 on the same dataset. On UCCD, PTNet reported BLEU-4 66.89, METEOR 44.15, ROUGE-L 78.47, CIDEr-D 188.35, F1 72.77, and IoU 57.65, reflecting joint captioning-and-detection evaluation on a UAV benchmark [2605.15024][2511.21420][2605.04409].

Performance should nevertheless be interpreted with dataset context. SECOND-CC and disaster-caption datasets use different caption lengths, vocabularies, and scene distributions, so direct comparison across benchmarks is limited [2501.10075][2509.01907]. This suggests that RSCC metrics remain benchmark-dependent, and that identical BLEU or CIDEr values can reflect different kinds of descriptive behavior.

## 6. Limitations, misconceptions, and future directions

Several limitations recur across the literature. Pseudo-label strategies inherit label noise, especially under domain shift between change-detection training data and captioning imagery [2312.15311]. Mask-guided systems depend on mask quality: false negatives can suppress relevant tokens, while false positives can reintroduce noise [2410.23946]. Systems built on natural-image or video foundation models face a pretraining gap when transferred to remote sensing, even when adapted with LoRA or multi-stage tuning [2407.14032][2410.23946][2411.11360]. Diffusion-based systems improve robustness to noisy conditioning but remain slow to train and sample [2412.19179].

The literature also clarifies several misconceptions. One is that RSCC can be solved by better caption decoding alone. DFM explicitly argues that a single autoregressive generation paradigm tends to prioritize easily generated vocabulary over discriminative visual differences [2606.27410]. Another is that global scene-level captions provide sufficient supervision; RSRCC argues that a caption can be globally plausible while still failing to localize or correctly attribute the change of interest [2604.20623]. A third is that changed and unchanged pairs can be modeled identically; HiSem presents this as a modeling inconsistency that leads to semantic entanglement between change-existence judgment and fine-grained description [2605.15024].

Future directions are correspondingly broad but convergent. Multiple papers point to multi-sensor and multi-modal extensions, including optical/SAR integration, multispectral inputs, GIS layers, and longer temporal sequences beyond the bi-temporal case [2312.15311][2605.04409]. Foundation-model work suggests continued pretraining on large, domain-specific corpora and more faithful integration of difference-aware modules into pretrained MLLMs [2411.11360]. Region-grounded QA resources such as RSRCC suggest a path toward localized supervision that may reduce hallucination and improve attribution [2604.20623]. Disaster and UAV datasets expand the task from generic urban development toward operational monitoring, where captions must describe severity, extent, and functional impact rather than only object appearance or disappearance [2509.01907][2605.04409].

Taken together, these developments show RSCC evolving from coarse bi-temporal captioning toward a grounded, multi-task, and increasingly foundation-model-driven discipline. The central research problem is no longer only how to describe that change occurred, but how to connect language generation to reliable spatial, temporal, and semantic evidence under real remote-sensing variability.

Source: https://www.emergentmind.com/topics/remote-sensing-change-caption-rscc