Change Detection Visual Question Answering

• CDVQA is a research area that integrates multi-temporal remote sensing change detection with visual question answering for semantic analysis of geospatial image pairs.
• It employs methods such as Siamese backbones, multi-modal attention, and change enhancing modules to detect, classify, and spatially ground changes between images.
• Standard datasets and benchmarks like CDVQA, QAG-360K, and ChangeChat-105k facilitate evaluation and drive advances in interactive, human-centric remote sensing analysis.

Change Detection Visual Question Answering (CDVQA) is a research area that unifies multi-temporal remote sensing change detection with the interpretability and flexibility of visual question answering (VQA), enabling rich, user-driven semantic interrogation of change phenomena in multi-temporal geospatial imagery. CDVQA systems take as input a pair of co-registered remote-sensing images acquired at different times and a natural-language query about changes; they output a discrete answer (optionally with spatial grounding), providing a versatile, interactive interface for human-centric remote sensing analysis (Yuan et al., 2021, Li et al., 2024, Deng et al., 30 Jul 2025).

1. Formal Task Definition and Scope

CDVQA models operate on paired imagery $I_1,I_2\in\mathbb{R}^{H\times W\times3}$ from times $t_1$ and $t_2$ and a query $q$ about semantic changes. The core task is to produce an answer $a$ (e.g., “yes/no,” land-cover class, ratio range) and, in some systems, an optional spatial mask $M\in[0,1]^{H\times W}$ grounding the answer to the supporting pixels. Formally, the model learns

$$(a, M) = \arg\max_{(\tilde{a}, \tilde{M})} P(\tilde{a}, \tilde{M} \mid I_1, I_2, q)$$

depending on whether only $a$ or both $a,M$ are required (Li et al., 2024). Earlier CDVQA frameworks focus exclusively on text answers; more recent work (CDQAG) benchmarks both text and mask output (Li et al., 2024).

Question types encompass binary change detection, categorical change classification, quantification (counting or ratio estimation), spatiotemporal localization, open-ended semantic queries, and multi-turn dialogue covering progressing complexity (Yuan et al., 2021, Deng et al., 30 Jul 2025).

2. Datasets and Benchmark Construction

Key datasets enable standardized evaluation of CDVQA systems by providing large-scale, high-quality annotation of image pairs and semantically diverse queries:

• CDVQA dataset (Yuan et al. 2021): 2,968 image pairs (“SECOND” dataset, 0.5–3 m resolution, $512\times512$), ~122,000 automatically-generated QA pairs with 19 answer categories and pixel-wise semantic change maps (not annotated as supporting masks) (Yuan et al., 2021).
• QAG-360K (Li et al., 2024): 6,810 pairs from LEVIR-CD, SECOND, and Hi-UCD; 360,000+ triplets of {question, answer, pixel mask}, spanning 10 land-cover categories and 8 question types (e.g., binary change, class transition, increase/decrease, largest/smallest change, ratio queries).
• ChangeChat-105k (Deng et al., 30 Jul 2025): 105,107 bi-temporal pairs (from LEVIR-CC, LEVIR-MCI) with rule-based and GPT-assisted generation for six interaction types (captioning, classification, quantification, localization, open-ended QA, multi-turn dialogue).

Table: Comparison of Major CDVQA Datasets

Dataset	Images	Q–A Pairs	Categories	Pixel Masks?	Notable Features
CDVQA	2,968	122,000	6	No	Generated semantic QAs
QAG-360K	6,810	360,000+	10	Yes	8 Q types, mask support
ChangeChat-105k	105,107	105,107	6	Yes	6 interaction types

The automatic generation pipelines integrate pixel-wise annotation (for QAG-360K, ChangeChat-105k), manual verification, and, for some Q types, LLMs (e.g., GPT-4) to generate diverse, high-quality questions for real-world change scenarios (Li et al., 2024, Deng et al., 30 Jul 2025).

3. Model Architectures and Methodologies

3.1 Standard Architectures

Baselines typically encode $I_1,I_2$ using Siamese backbones (ResNet, Vision Transformer), aggregate features through multi-temporal fusion (concatenation, subtraction, multi-modal attention), and process the question with a recurrent or transformer-based language encoder. The fused representation is passed to an answer predictor (MLP/softmax) (Yuan et al., 2021).

A change enhancing module (CEM) is commonly used to highlight pixels with strong bi-temporal discrepancies:

• Project features via $1\times 1$ convs,
• Compute difference $|\mathbf{Q} - \mathbf{K}|$,
• Generate an attention mask, and
• Reweight features to yield change-aware encodings.

Fusion strategies are empirically evaluated: concatenation + downstream MLP yields superior performance to subtraction or element-wise ops (Yuan et al., 2021).

3.2 Joint Text–Visual Grounding

Recent architectures unify text answer prediction and pixel-level spatial grounding. VisTA (Li et al., 2024) exemplifies this approach:

• Dual CLIP-pretrained ResNet-101 backbones extract multi-level features for both images,
• 1×1 convolutions at each scale compute change-specific features,
• A CLIP-pretrained transformer encodes the question at both token and sentence level,
• Language-guided fusion and multi-stage transformer decoders integrate features and refine the response,
• Task-specific soft gating modulates attention to yield both answer logits and a segmentation mask.

The total loss is a sum of cross-entropy for answer supervision and a text-to-pixel contrastive loss aligning textual concepts to supporting pixels (Li et al., 2024).

3.3 Interactive RSICA and Multi-turn Dialogues

DeltaVLM (Deng et al., 30 Jul 2025) integrates instruction-following and multi-turn reasoning:

• EVA-ViT-g/14 backbone with final layers fine-tuned for the RS domain; first 37 transformer layers frozen,
• Visual difference perception module with cross-semantic relation measuring (CSRM) suppresses irrelevant change (noise, lighting) while focusing on structural change (new roads, buildings),
• Instruction-guided Q-former (set of 32 learnable queries) attends over CSRM-processed features and aligns difference representations with the user instruction,
• Alignment tokens passed to a (frozen) Vicuna-7B LLM decoder, optimized via token-level cross-entropy loss.

The model supports flexible queries, diverse output modes (caption, mask, classification, localization), and coherent multi-turn dialogue for advanced change analysis (Deng et al., 30 Jul 2025).

4. Training Paradigms and Optimization

Most systems utilize cross-entropy loss for textual answer classification; joint frameworks sum this with text-to-pixel contrastive or mask losses (Li et al., 2024). Training uses Adam or AdamW optimizers with moderate batch sizes and extensive data augmentation (cropping, rotation, resizing).

DARFT (Decision-Ambiguity-guided Reinforcement Fine-Tuning) (Dong et al., 31 Dec 2025) addresses the failure mode where models are indecisive between the correct answer and strong distractors—the decision-ambiguous samples (DAS). The approach proceeds:

1. Train reference policy via standard supervised fine-tuning,
2. Identify DAS where the probability margin between top-2 answers is below a threshold,
3. Apply group-relative PPO (policy optimization) to only these samples, using intra-group advantage,
4. KL-regularize policy updates to prevent drift,
5. Leverage multi-sample decoding at inference for stability.

This paradigm significantly improves few-shot performance, robustness under label ambiguity, and model discriminability in fine-grained QA categories.

5. Quantitative Results and Empirical Insights

Performance benchmarks, evaluated on QAG-360K and CDVQA datasets, use metrics including Average Accuracy (AA), Overall Accuracy (OA), and, for spatial outputs, mIoU/oIoU.

Textual and Visual Results (QAG-360K, (Li et al., 2024)):

Method	Text AA (%)	OA (%)	Visual mIoU (%)	Visual oIoU (%)
VisTA	71.16	75.76	39.87	47.98
SOBA	66.27	71.98	32.74	41.83
CDVQA	62.46	68.60	17.66	29.33

DeltaVLM (Deng et al., 30 Jul 2025) achieves state-of-the-art metrics on single-turn and multi-turn tasks:

• BLEU-4 for change captioning: 62.51,
• F1 for binary classification: 93.83% (vs. 85.07% for GPT-4o baseline),
• Substantial reductions in MAE/RMSE for change quantification,
• F1 for localization up to 79% for buildings.

DARFT (Dong et al., 31 Dec 2025), under 0.5K few-shot (QAG-360K), yields:

• +6.05 pp OA and +3.94 pp AA over supervised fine-tuning,
• Notable +18.38 pp improvement for change-ratio category in the ambiguous setting.

6. Technical Challenges and Future Directions

• Semantic Reasoning and Multi-modal Fusion: Deep cross-modal attention and specialization of fusion strategies drive performance improvements; however, fine-grained change identification (e.g., subtle color/spectral shifts) and fragmented object grounding (disconnected buildings) remain open problems (Li et al., 2024).
• Ambiguity-aware Optimization: Addressing decision ambiguity at training time directly enhances model robustness on hard samples and under low-data regimes (Dong et al., 31 Dec 2025).
• Multi-turn and Instruction-guided Interaction: Extending beyond single-turn QA and static captioning toward dialogue-driven reasoning and memory modeling is a marked trend (e.g., RSICA (Deng et al., 30 Jul 2025)).
• Domain Adaptation and Scalability: Generalization across geographic domains and efficient inference with high-res images and long dialogues constrain current approaches. Model compression and multimodal output unification (mask+text) are prospective directions (Deng et al., 30 Jul 2025).

7. Related Tasks and Research Outlook

The CDVQA framework generalizes classical change detection (binary mask output), change captioning (unstructured summaries), RSVQA (VQA for static RS imagery), and visual grounding. It forms part of a broader movement toward interactive, instruction-tuned vision–language systems capable of reasoning over evolving Earth surface phenomena in a user-centric, explainable manner. Advances in large vision–LLMs, open-vocabulary change grounding, and chain-of-thought spatial–temporal reasoning represent key frontiers for the field (Deng et al., 30 Jul 2025, Li et al., 2024, Yuan et al., 2021).