JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering

Abstract: Remote sensing change detection (CD) traditionally focuses on pixel-level binary segmentation, which identifies where changes occur but neither what nor why. To bridge this semantic gap, we introduce JL1-CC&QA, a multi-task benchmark that extends the JL1-CD dataset with two complementary annotation layers: change captioning (CC) and change question answering (QA). Built upon 5,000 bi-temporal image pairs acquired by the Jilin-1 satellite at 0.5-0.75m ground sample distance, the benchmark comprises: (i) JL1-CC, providing 17,021 quality-verified captions that describe diverse land-cover transformations; and (ii) JL1-QA, offering 20,060 question-answer pairs across eight question types, enabling fine-grained, interactive interrogation of surface changes. All annotations are produced via a three-stage pipeline consisting of multi-modal LLM generation, vision-grounded LLM judging, and human expert verification. We hope that JL1-CC&QA, as a benchmark unifying binary change masks, change captions, and change-oriented QA over the same image set, will serve as a valuable resource for the community to advance multi-task change understanding in remote sensing. The dataset is available at https://github.com/circleLZY/JL1-CD.

• The paper introduces a unified benchmark that integrates binary masks, semantic change captions, and QA pairs over 5,000 bi-temporal satellite image pairs.
• It employs a scalable multi-modal LLM pipeline combined with expert audits, achieving high annotation quality with over 80% pass rates.
• Its multi-task evaluation framework enhances cross-modal learning, benefiting applications like disaster response, land cover monitoring, and environmental assessment.

JL1-CC&QA: A Unified Benchmark for Remote Sensing Change Captioning and Question Answering

Context: Evolution of Remote Sensing Change Detection

The field of remote sensing change detection (CD) has undergone a rapid paradigm shift with the emergence of deep learning techniques and the widespread availability of multi-temporal, high-resolution satellite imagery. Classical binary change detection (BCD) benchmarks and algorithmic advances have strongly focused on identifying the spatial location and extent of change at the pixel level, with outputs typically represented by binary masks (Figure 1).

Figure 1: Timeline of the development of mainstream deep learning-based CD methods.

However, these approaches inherently lack semantic expressiveness, neither characterizing the type of change (e.g., conversion from cropland to urban structures) nor enabling flexible, natural-language-based user interaction. Recent years have seen the introduction of semantic change detection (SCD), building damage assessment (BDA), and vision-LLMs (VLMs), but most benchmarks still encode semantic information using closed taxonomies or categorical indices, limiting the potential for holistic, language-grounded remote sensing analyses.

Motivation and Contributions

JL1-CC&QA addresses the critical bottleneck in language-grounded change detection by unifying binary change masks, detailed natural language change captions, and multi-faceted change-oriented question answering (QA) annotations within a single, large-scale benchmark. This multi-task resource establishes a unified evaluation landscape, facilitating both algorithmic development (e.g., in vision-language modeling, multi-task learning, and cross-modal representation learning) and practical deployment in real-world applications such as disaster response, land cover monitoring, and environmental assessment.

The contributions of JL1-CC&QA are threefold:

• Multi-layered Annotations: Provision of binary masks, 17,021 semantically rich, quality-verified change captions, and 20,060 QA pairs, all aligned over 5,000 bi-temporal satellite image pairs.
• Scalable LLM-based Annotation Pipeline: A three-stage process combining multi-modal LLM generation, vision-grounded LLM judging, and human expert verification, ensuring high factual reliability and linguistic diversity in the annotations.
• Comprehensive Task and Metadata Diversity: Coverage of diverse geographies, change types (anthropogenic and natural), and a broad taxonomy of question types (existence, description, location, scale, temporal, causation, visual detail, and comparison), with supporting metadata (e.g., change area ratio, spatial references).

Dataset Construction and Statistical Landscape

Source Data and Distributional Analysis

The JL1-CD dataset, serving as the foundation, comprises 5,000 paired pre- and post-event images from Jilin-1 optical satellites at 0.5–0.75 m GSD, spanning major Chinese provinces. Each pair is associated with a binary change mask; the dataset's change area ratio (CAR) exhibits a highly skewed, long-tailed distribution, with a mean of 9.7% and a median of 2.9%, thus capturing the real-world imbalance in surface change magnitude.

Figure 2: Distribution of change area ratio (CAR) across all 5,000 image pairs in JL1-CD.

Change Captioning (JL1-CC) Pipeline

The captioning annotation protocol leverages a multi-modal LLM (Kimi-K2.6), which generates five candidate captions per image pair leveraging:

• Pre- and post-event images,
• The change mask,
• Associated spatial metadata (e.g., CAR, region localization).

A vision-grounded LLM judge scores each caption on accuracy, specificity, spatial correctness, fluency, and informativeness, retaining approximately the top three per pair. Human experts audit a subset for factual sanity-checks.

Figure 3: Overview of the JL1-CC annotation pipeline, combining multi-modal LLM generation, vision-based evaluation, and expert audit.

Judging results show that 82.1% of captions pass quality thresholds (score $\geq$ 7), with 39.8% scoring in the highest bracket (9–10). The resulting corpus maintains a wide vocabulary and encapsulates diverse land cover changes, as visualized in the word cloud.

Figure 4: Judge score distribution for JL1-CC. Captions scoring below 7 are rejected; those scoring 7-8 and 9-10 are retained.

Figure 5: Word cloud of the 17,021 captions in JL1-CC, highlighting common spatial and semantic land-cover vocabulary.

Change QA (JL1-QA) Pipeline

The QA annotation pipeline extends the inputs with the selected captions from JL1-CC, enhancing grounding and answer relevance. It samples from an eight-type question taxonomy designed to stress multi-aspect semantic reasoning:

• YES/NO, WHAT, WHERE, HOW MUCH, BEFORE/AFTER, CAUSE, DETAIL, COMPARE.

  Figure 6: Overview of the JL1-QA pipeline, where generation is enriched by CC context and robustly filtered by multi-criteria LLM judging.

QA pairs are scored on accuracy, clarity, completeness, and redundancy, with a pass rate of 80.3%. Distributional analysis confirms type diversity and minimal overlap, with strong representation of yes/no, location, and description queries.

Figure 7: Judge score distribution for JL1-QA. QA pairs below 7 are rejected.

Figure 8: Distribution of question types in the 20,060 selected QA pairs of JL1-QA.

Empirical Strengths and Contrasts

JL1-CC&QA distinguishes itself by both scale and annotation granularity compared to prior VLM-grounded remote sensing benchmarks. Unlike existing datasets limited to either binary masks, isolated captions, or VQA-style queries, JL1-CC&QA uniquely enables multi-task learning under congruent visual context, substantially increasing the potential for robust cross-modal, cross-task generalization—for example, enabling joint segmentation, captioning, and interactive dialogue.

Statistically, the LLM judging and human verification protocol yields a strong filtration effect; only 17.9% of captions and approximately 20% of QA pairs are formally rejected for factual or linguistic inadequacy—suggesting high LLM annotation precision under the pipeline, especially when context-enriched. Notably, the QA pass rate outperforms captioning, likely due to the grounding effect of CC-derived context during question/answer generation. This benchmark therefore provides a strong foundation for future studies in reducing hallucination and evaluating factual consistency in remote sensing VL tasks.

Practical and Theoretical Implications

Practically, JL1-CC&QA provides a standardized reference for:

• Evaluation of multi-modal and multi-task architectures: Enabling rigorous cross-task benchmarking (segmentation, captioning, QA) with shared underlying inputs.
• Study of annotation pipeline efficiency: Demonstrating the value of LLM-multiplexed annotation with downstream expert oversight compared to traditional fully manual processes.
• Advancement of foundation model adaptation: Supporting transferrable learning from vision-language and agent-based inference paradigms.

Theoretically, the dataset incentivizes exploration of several open research questions:

• Robust multi-task and continual learning where cross-task representations may conflict (e.g., segmentation vs. captioning),
• Factual consistency and reduction of LLM hallucination in remote sensing,
• Interpretability of model decisions bridging mask- and language-level change characterization,
• Grounding of physical change phenomena directly in natural language interaction frameworks.

JL1-CC&QA also serves as a springboard for extension into conversational analysis, temporal predictive querying, and open-vocabulary semantic segmentation, especially as foundation models and multi-agent systems in remote sensing continue to scale (2606.31745).

Conclusion

JL1-CC&QA establishes a new standard in language-grounded remote sensing change detection research by comprehensively unifying mask-, caption-, and QA-oriented annotation for high-resolution bi-temporal satellite image pairs. Its rigorous annotation pipeline, broad semantic coverage, and scale facilitate both robust benchmarking and future progress in multi-task and vision-language remote sensing models. This resource is poised to catalyze advances in both practical deployment and fundamental methodology in AI for earth observation.