Remote Sensing Change Captioning

• RSCC is a technique that generates natural language descriptions explaining semantic changes between co-registered remote sensing images captured at different times.
• It integrates change detection with advanced models like CNNs, Transformers, and diffusion techniques to capture subtle variations such as urban growth or disaster damage.
• Benchmarks and datasets like LEVIR-CC and Dubai-CC, evaluated via metrics such as BLEU and CIDEr, underscore RSCC’s role in precise spatial and semantic change analysis.

Remote Sensing Change Caption (RSCC), also called Remote Sensing Image Change Captioning (RSICC), is the task of generating natural-language descriptions of semantic changes between two co-registered remote sensing images of the same area acquired at different times. It converts bi-temporal visual evidence—such as buildings constructed or demolished, roads extended, vegetation cleared, facilities added, or disaster damage—into captions that must be fluent, spatially grounded, and semantically precise. The topic sits at the intersection of change detection, semantic scene understanding, and vision-language generation: unlike binary or semantic change detection, it must explain changes in open-ended language, and unlike single-image captioning, it must reason explicitly over temporal differences rather than static content (Liu et al., 2023, Wang et al., 14 May 2026, Ferrod et al., 2024).

1. Task definition and conceptual boundaries

RSCC commonly assumes two inputs, $I_1$ and $I_2$, representing the same geographic region at times $t_1$ and $t_2$, and a caption $Y=\{y_1,\dots,y_T\}$ describing the change. A representative formulation conditions caption generation on both bi-temporal features and localized change evidence, as in

$$L_{\mathrm{cap}}=-\sum_{t=1}^{T}\log p(y_t \mid y_{<t}, F_1, F_2, M),$$

where $M \in \{0,1\}^{H\times W}$ denotes a change map (Liu et al., 2023). Other formulations model the conditional distribution $p(Y \mid I_{t1}, I_{t2})$ directly and augment token-level generation with contrastive or auxiliary objectives (Wang et al., 25 Jun 2026).

The field is defined as distinct from both Binary Change Detection and Semantic Change Detection. Binary Change Detection answers where change occurs, typically by predicting a mask $M \in \{0,1\}^{H\times W}$, while Semantic Change Detection assigns predefined class labels to changed regions. RSCC instead requires open-ended language capable of expressing richer semantics, object attributes, spatial relations, counts, and temporal dynamics (Wang et al., 14 May 2026). It is also distinct from conventional remote-sensing image captioning, because the output must isolate differences between times rather than summarize a single scene (Ferrod et al., 2024).

Several persistent difficulties recur across the literature. Scenes are often dense and heterogeneous, with subtle or low-contrast changes, illumination variability, atmospheric effects, seasonal appearance shifts, and sensor or regional domain shift. Captions must not only identify changed objects but also localize them with phrases such as “on the right side” or “in the north,” sometimes while distinguishing changed from unchanged context (Liu et al., 2023). Later work frames an additional difficulty as semantic heterogeneity between changed and unchanged pairs: unchanged samples are coarse-grained and often reducible to “no change,” whereas changed samples require fine-grained description, so a unified modeling strategy can entangle coarse change-existence judgment with token-level semantics (Wang et al., 14 May 2026). Another line of work argues that purely autoregressive training can bias models toward easy, frequent vocabulary instead of discriminative difference cues (Wang et al., 25 Jun 2026).

2. Architectural evolution

Early RSCC systems adapted change-captioning designs from natural-image and video settings. Representative baselines include Capt-Rep-Diff, Capt-Att, Capt-Dual-Att, and DUDA, which relied on CNN encoders and RNN-style or attention-based decoders but generally lacked explicit pixel-level grounding (Liu et al., 2023). A major early remote-sensing-specific model, "Changes to Captions: An Attentive Network for Remote Sensing Change Captioning" (Chang et al., 2023), introduced Chg2Cap, which combines a Siamese ResNet-101 feature extractor, stacked hierarchical self-attention blocks, a cosine-similarity mask, and a Transformer decoder. Chg2Cap formalized a recurring pattern in the literature: shared-weight bi-temporal encoding, dedicated change-focused fusion, and autoregressive caption generation (Chang et al., 2023).

Transformer-based multi-scale modeling then became prominent. "Progressive Scale-aware Network for Remote sensing Image Change Captioning" (Liu et al., 2023) proposed PSNet as a pure Transformer architecture with Progressive Difference Perception layers for multi-scale differencing and a Scale-aware Reinforcement module that progressively injects different scale features into different decoder layers. SAT-Cap later reduced reliance on multi-stage fusion by introducing a Spatial-Channel Attention Encoder and a cosine-similarity-based Difference-Guided Fusion module, arguing for single-stage feature integration while reporting CIDEr scores of 140.23 on LEVIR-CC and 97.74 on DUBAI-CC (Wang et al., 14 Jan 2025).

Efficiency-oriented variants sought to reduce the computational burden of dense self-attention. "A Lightweight Sparse Focus Transformer for Remote Sensing Image Change Captioning" (Sun et al., 2024) proposed a Sparse Focus Transformer whose abstract reports a reduction of over 90% in parameters and computational complexity for the transformer encoder while retaining competitive performance. In parallel, "RSCaMa: Remote Sensing Image Change Captioning with State Space Model" (Liu et al., 2024) replaced attention-heavy spatiotemporal modeling with Mamba-based state space modules, introducing Spatial Difference-aware SSM and Temporal-Traversing SSM to achieve global receptive fields with linear complexity. This line established state-space modeling as a viable alternative to Transformer fusion for RSCC (Liu et al., 2024).

Diffusion models expanded the design space further. "Diffusion-RSCC: Diffusion Probabilistic Model for Change Captioning in Remote Sensing Images" (Yu et al., 2024) reframed caption generation as denoising continuous caption embeddings conditioned on bi-temporal features. "Mask Approximation Net: A Novel Diffusion Model Approach for Remote Sensing Change Captioning" (Sun et al., 2024) instead shifted emphasis from conventional feature learning toward data distribution learning, using a diffusion model to refine change-mask-like representations before caption generation. These models explicitly target robustness to pixel-level nuisance variation, although they also inherit the cost of iterative sampling (Yu et al., 2024, Sun et al., 2024).

A separate development replaced manual fusion modules with pretrained video encoders. "MV-CC: Mask Enhanced Video Model for Remote Sensing Change Caption" (Liu et al., 2024) treats the bi-temporal pair as a two-frame clip and uses InternVideo2 to extract spatiotemporal representations directly, then gates token grids with change masks. This formulation reframes RSCC as a short-video captioning problem with explicit region filtering rather than handcrafted temporal fusion (Liu et al., 2024).

3. Grounding change semantics and integrating auxiliary supervision

A central trend in modern RSCC is the use of auxiliary localization or semantic signals to improve grounding. "Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning" (Liu et al., 2023) made this relation explicit by generating pixel-level pseudo-labels with a pre-trained BIT change detector, training an auxiliary UNet-like CD branch with binary cross-entropy, and fusing its high-level change features into the captioning pathway through the Semantic Fusion Augment module. The method is premised on the idea that pixel-level CD is significant for describing the differences between images through language, especially in complex scenes (Liu et al., 2023).

Joint learning of detection and captioning was strengthened by foundation-model-based systems. "Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance" (Zhu et al., 2024) combines a SAM-based bi-temporal encoder, a multi-task semantic aggregation neck, a multi-scale CD decoder, and a Vicuna-based caption decoder, trained with a three-stage schedule designed to stabilize CD/CC interaction. "ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing" (Wang et al., 2024) similarly treats change detection and change captioning as a unified end-to-end problem, using ChangeLSTM to build universal change-aware representations shared by a CD head and a caption head with cross-attention (Wang et al., 2024).

Other works integrate change semantics through contrastive or multimodal guidance. "DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning" (Wang et al., 25 Jun 2026) adds a Text-guided Gated Contrastive Loss that aligns visual difference embeddings with sentence-level text embeddings while excluding no-change samples from the contrastive objective. "Robust Change Captioning in Remote Sensing: SECOND-CC Dataset and MModalCC Framework" (Karaca et al., 17 Jan 2025) fuses RGB and semantic segmentation streams through Cross-Modal Cross Attention, Unimodal Difference Cross Attention, and Multimodal Gated Cross Attention, explicitly using semantic maps to suppress hallucinations in no-change or noisy regions (Karaca et al., 17 Jan 2025).

Several recent systems push grounding beyond pixel-level detection alone. "SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning" (Wang et al., 26 Nov 2025) mines motion-level changed regions with SAM and SuperGlue, semantic regions with Grounding DINO plus SAM, and semantic priors from a relation graph encoded by R-GCN. "UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model" (Gao et al., 6 May 2026) introduces PTNet, which uses a learnable prototype bank to encode structured change semantics, multi-head gating to disentangle captioning and detection representations, and detection-derived spatial priors as tokens for caption generation. "HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning" (Wang et al., 14 May 2026) argues that changed and unchanged pairs require different semantic treatments and implements this with coarse image-level routing and fine-grained token-level Mixture-of-Experts. "CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset" (Wang et al., 2024) preserves the original MLLM feature pathway while injecting multi-scale difference-aware signals through a dedicated integration module (Wang et al., 26 Nov 2025, Gao et al., 6 May 2026, Wang et al., 14 May 2026, Wang et al., 2024).

Collectively, these systems indicate that RSCC has moved from coarse differencing toward explicit semantic grounding. This suggests that the field increasingly treats caption generation not as a standalone decoder problem but as a downstream language layer over progressively richer change representations.

4. Benchmarks, datasets, and supervision resources

The empirical development of RSCC has been shaped by a sequence of datasets that vary in scale, annotation type, modality, and application domain. LEVIR-CC remains the most commonly reported captioning benchmark; Dubai-CC and WHU-CDC broaden evaluation across scale and scene types; LEVIR-MCI and SECOND-CC introduce joint or semantic supervision; UCCD and the disaster-focused RSCC dataset extend the field into new operational regimes; CC-Foundation and RSRCC extend supervision through pretraining corpora and localized QA (Liu et al., 2023, Wang et al., 25 Jun 2026, Liu et al., 2024, Karaca et al., 17 Jan 2025, Gao et al., 6 May 2026, Wang et al., 2024, Kazoom et al., 22 Apr 2026, Chen et al., 2 Sep 2025).

Resource	Scale	Distinguishing features
LEVIR-CC (Liu et al., 2023)	10,077 pairs; 50,385 captions	5,038 changed and 5,039 unchanged; five captions per pair
Dubai-CC (Wang et al., 25 Jun 2026)	500 pairs; 2,500 captions	Early urban benchmark for RSCC
WHU-CDC (Wang et al., 26 Nov 2025)	7,434 pairs; 37,170 sentences	Building, parking, road, and related urban changes
LEVIR-MCI (Liu et al., 2024)	10,077 pairs	Captions plus CD masks for roads and buildings
SECOND-CC (Karaca et al., 17 Jan 2025)	6,041 pairs; 30,205 captions	RGB plus semantic segmentation maps; 30 class-to-class transitions
UCCD (Gao et al., 6 May 2026)	9,000 pairs; 45,000 captions	UAV benchmark; 6 cm/pixel; urban construction focus
CC-Foundation (Wang et al., 2024)	200,000 pairs; 1.2 million captions	Continued-pretraining corpus for MLLM-based RSCC
RSRCC (Kazoom et al., 22 Apr 2026)	126k QA instances	Localized change question-answering benchmark
RSCC disaster dataset (Chen et al., 2 Sep 2025)	62,351 pairs	Disaster-focused, multi-sentence captions, 31 global events

Dataset design strongly influences what models learn. LEVIR-CC offers only sentence-level annotations, which motivated pseudo-label approaches such as Pix4Cap (Liu et al., 2023). LEVIR-MCI supplies both captions and change masks, enabling joint detection-captioning systems such as MV-CC and ChangeMinds (Liu et al., 2024, Wang et al., 2024). SECOND-CC adds semantic segmentation maps and includes explicit no-change pairs, making robustness to illumination, viewpoint shifts, blur, and registration error part of the benchmark itself (Karaca et al., 17 Jan 2025). UCCD moves the field toward UAV-based urban construction monitoring with 3024×4032 originals cropped to 1024×1024 and manually annotated change masks (Gao et al., 6 May 2026). The disaster-oriented RSCC dataset adds 62,351 pre-/post-disaster pairs with an average caption length of 72 words, pushing the task from short object-change statements toward richer situational narratives (Chen et al., 2 Sep 2025).

Two resources extend RSCC beyond standard caption benchmarking. CC-Foundation supplies 200,000 image pairs and 1.2 million captions for continued pretraining of multimodal large models (Wang et al., 2024). RSRCC reframes scene-level RSCC as localized, region-grounded change question answering with 126k questions, emphasizing that scene-level captions can be correct while still failing to localize or correctly attribute specific changes (Kazoom et al., 22 Apr 2026).

5. Objectives, evaluation metrics, and empirical performance

Most RSCC systems still optimize teacher-forced token-level cross-entropy for caption generation, but the training objectives around it have diversified. Pix4Cap jointly minimizes caption loss and binary cross-entropy for an auxiliary CD branch, with $$L_{\text{total}}=L_{\text{cap}}+\lambda L_{\text{cd}}$$ and $I_2$0 in the reported experiments (Liu et al., 2023). DFM augments the autoregressive generation loss with a gated text-image contrastive objective,

$I_2$1

to shape the visual encoder toward text-relevant differences (Wang et al., 25 Jun 2026). Retrieval-oriented work adds InfoNCE-style image-text alignment alongside captioning loss, enabling a single model to support both bi-temporal captioning and text-image retrieval (Ferrod et al., 2024). Multi-task systems combine caption loss with change-detection losses, often using cross-entropy or BCE for the detection branch (Zhu et al., 2024, Wang et al., 2024).

Evaluation is dominated by BLEU-1/2/3/4, METEOR, ROUGE-L, and CIDEr or CIDEr-D (Liu et al., 2023, Wang et al., 25 Jun 2026). Several papers summarize these with

$I_2$2

or an equivalent $I_2$3 variant (Wang et al., 25 Jun 2026, Wang et al., 14 May 2026). Some diffusion and QA-oriented works also report SPICE, BARTScore, MoverScore, BERTScore, or localized QA accuracy, reflecting a broader concern with semantic fidelity beyond $I_2$4-gram overlap (Yu et al., 2024, Kazoom et al., 22 Apr 2026).

On LEVIR-CC, "Pixel-Level Change Detection Pseudo-Label Learning for Remote Sensing Change Captioning" (Liu et al., 2023) reported BLEU-4 63.78, METEOR 39.96, ROUGE-L 75.12, CIDEr 136.76, and $I_2$5 78.91. "DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning" (Wang et al., 25 Jun 2026) reported CIDEr-D 142.51, METEOR 40.95, ROUGE-L 75.90, BLEU-4 66.26, and $I_2$6 81.40. "CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset" (Wang et al., 2024) reported BLEU-4 65.49, METEOR 41.82, ROUGE-L 76.55, CIDEr-D 143.32, and $I_2$7 81.80. These results indicate that MLLM-based systems and contrastive difference modeling both pushed the benchmark above earlier Transformer and pseudo-label baselines (Liu et al., 2023, Wang et al., 25 Jun 2026, Wang et al., 2024).

On WHU-CDC, "HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning" (Wang et al., 14 May 2026) reported BLEU-4 76.52, METEOR 48.77, ROUGE-L 82.00, CIDEr-D 158.35, and $I_2$8 91.41. "SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioning" (Wang et al., 26 Nov 2025) reported BLEU-4 74.42 and CIDEr-D 156.21 on the same dataset. On UCCD, PTNet reported BLEU-4 66.89, METEOR 44.15, ROUGE-L 78.47, CIDEr-D 188.35, F1 72.77, and IoU 57.65, reflecting joint captioning-and-detection evaluation on a UAV benchmark (Wang et al., 14 May 2026, Wang et al., 26 Nov 2025, Gao et al., 6 May 2026).

Performance should nevertheless be interpreted with dataset context. SECOND-CC and disaster-caption datasets use different caption lengths, vocabularies, and scene distributions, so direct comparison across benchmarks is limited (Karaca et al., 17 Jan 2025, Chen et al., 2 Sep 2025). This suggests that RSCC metrics remain benchmark-dependent, and that identical BLEU or CIDEr values can reflect different kinds of descriptive behavior.

6. Limitations, misconceptions, and future directions

Several limitations recur across the literature. Pseudo-label strategies inherit label noise, especially under domain shift between change-detection training data and captioning imagery (Liu et al., 2023). Mask-guided systems depend on mask quality: false negatives can suppress relevant tokens, while false positives can reintroduce noise (Liu et al., 2024). Systems built on natural-image or video foundation models face a pretraining gap when transferred to remote sensing, even when adapted with LoRA or multi-stage tuning (Zhu et al., 2024, Liu et al., 2024, Wang et al., 2024). Diffusion-based systems improve robustness to noisy conditioning but remain slow to train and sample (Sun et al., 2024).

The literature also clarifies several misconceptions. One is that RSCC can be solved by better caption decoding alone. DFM explicitly argues that a single autoregressive generation paradigm tends to prioritize easily generated vocabulary over discriminative visual differences (Wang et al., 25 Jun 2026). Another is that global scene-level captions provide sufficient supervision; RSRCC argues that a caption can be globally plausible while still failing to localize or correctly attribute the change of interest (Kazoom et al., 22 Apr 2026). A third is that changed and unchanged pairs can be modeled identically; HiSem presents this as a modeling inconsistency that leads to semantic entanglement between change-existence judgment and fine-grained description (Wang et al., 14 May 2026).

Future directions are correspondingly broad but convergent. Multiple papers point to multi-sensor and multi-modal extensions, including optical/SAR integration, multispectral inputs, GIS layers, and longer temporal sequences beyond the bi-temporal case (Liu et al., 2023, Gao et al., 6 May 2026). Foundation-model work suggests continued pretraining on large, domain-specific corpora and more faithful integration of difference-aware modules into pretrained MLLMs (Wang et al., 2024). Region-grounded QA resources such as RSRCC suggest a path toward localized supervision that may reduce hallucination and improve attribution (Kazoom et al., 22 Apr 2026). Disaster and UAV datasets expand the task from generic urban development toward operational monitoring, where captions must describe severity, extent, and functional impact rather than only object appearance or disappearance (Chen et al., 2 Sep 2025, Gao et al., 6 May 2026).

Taken together, these developments show RSCC evolving from coarse bi-temporal captioning toward a grounded, multi-task, and increasingly foundation-model-driven discipline. The central research problem is no longer only how to describe that change occurred, but how to connect language generation to reliable spatial, temporal, and semantic evidence under real remote-sensing variability.