JL1-CC&QA: Remote Sensing Change Benchmark

• JL1-CC&QA is a comprehensive remote sensing benchmark that unifies binary change detection, semantic captioning, and QA to assess what changed, where, and why.
• It extends the JL1-CD dataset by adding 17,021 quality-verified change captions and 20,060 QA pairs to 5,000 bi-temporal Jilin-1 image pairs.
• The benchmark employs a three-stage annotation pipeline combining multi-modal LLM generation, vision-grounded judging, and expert verification for robust quality control.

JL1-CC&QA is a multi-task remote sensing benchmark that extends the earlier JL1-CD benchmark with two complementary annotation layers—change captioning and change question answering—while retaining the original pixel-level binary change masks. It is designed to move beyond the traditional “where did change happen?” formulation of change detection and toward “what changed, where, and why?” over the same 5,000 Jilin-1 bi-temporal image pairs (Liu et al., 30 Jun 2026). In the benchmark’s framing, conventional binary change detection outputs a mask that indicates changed versus unchanged pixels but “neither what nor why,” whereas JL1-CC&QA unifies binary change detection, semantic change description, and interactive natural-language querying within one dataset (Liu et al., 30 Jun 2026).

1. Conceptual position in remote sensing change understanding

JL1-CC&QA is proposed in a context where binary change detection (BCD) remains the dominant formulation of remote sensing CD. The benchmark explicitly positions itself against the limits of BCD, which provides only pixel-level binary segmentation. The paper also contrasts JL1-CC&QA with semantic change detection (SCD) and building damage assessment (BDA): these are richer than BCD, but still encode semantics as discrete class labels from a closed taxonomy (Liu et al., 30 Jun 2026).

The benchmark’s central claim is that the field now needs “a shift toward natural-language-grounded change understanding,” especially given the rise of vision-LLMs, change captioning, change-detection VQA, and agentic reasoning systems (Liu et al., 30 Jun 2026). JL1-CC&QA operationalizes that shift by combining masks, captions, and QA in one benchmark so that models can be trained and evaluated across multiple complementary tasks rather than only pixel labeling.

This suggests that JL1-CC&QA is intended not merely as an additional annotation layer over an existing CD corpus, but as a reformulation of what constitutes change understanding in remote sensing. A plausible implication is that evaluation can move from purely spatial localization to joint assessment of localization, semantic abstraction, and interactive reasoning.

2. Source dataset and image-pair characteristics

JL1-CC&QA is built on the JL1-CD source dataset, which contains 5,000 bi-temporal image pairs from the Jilin-1 optical constellation (Liu et al., 30 Jun 2026). The imagery is RGB, at 0.5–0.75 m GSD, and each pair is a pre-event / post-event 512×512 image pair with a professionally annotated pixel-level binary change mask. The geographic coverage spans multiple Chinese provinces, including Shandong, Ningxia, Anhui, Hebei, and Hunan, and the temporal coverage is 2022–2023. The split is 4,000 training pairs and 1,000 test pairs (Liu et al., 30 Jun 2026).

The paper emphasizes two properties of JL1-CD as the basis for language-grounded extension. First, it is described as “all-inclusive” in terms of change types, covering anthropogenic changes such as buildings, roads, hardened surfaces, and photovoltaic panels, as well as natural changes such as woodlands, grasslands, croplands, and water bodies. Second, it exhibits a strongly long-tailed change-area distribution, with CAR spanning nearly zero to 100%, a mean of 9.7%, and a median of 2.9% (Liu et al., 30 Jun 2026).

These properties matter because the benchmark’s language tasks are anchored to the same image pairs and masks. The diversity of land-cover transformations supports varied caption and QA content, while the long-tailed CAR distribution introduces difficulty heterogeneity. This suggests that the benchmark is not restricted to large, visually obvious changes; it also includes small-area and potentially ambiguous transformations that are consequential for both captioning and question answering.

3. Benchmark composition and task schema

JL1-CC&QA comprises three aligned annotation layers over the same image set: the original binary change masks from JL1-CD, JL1-CC for change captioning, and JL1-QA for change-oriented question answering (Liu et al., 30 Jun 2026). The dataset-level composition is summarized below.

Component	Content	Scale
JL1-CD	Binary change masks	5,000 image pairs
JL1-CC	Quality-verified change captions	17,021 captions
JL1-QA	Change-oriented QA pairs	20,060 QA pairs

For JL1-CC, the task definition is: given a bi-temporal pair $(I_A, I_B)$ and a binary change mask $M$, generate a set of natural-language sentences $\{c_1, c_2, \ldots, c_k\}$ that describe the semantic content of the observed surface changes, including type, location, and extent (Liu et al., 30 Jun 2026). For JL1-QA, the task is: given $(I_A, I_B)$ and a natural-language question $Q$, generate a textual answer $A$ based on the visible surface changes (Liu et al., 30 Jun 2026).

JL1-QA contains 20,060 QA pairs across eight question types: YES/NO, WHAT, WHERE, HOW MUCH, BEFORE/AFTER, CAUSE, DETAIL, and COMPARE (Liu et al., 30 Jun 2026). The example questions in the paper define the intended semantics of these categories, including “Has the vegetation in the lower half been removed?” for YES/NO, “What happened to the farmland in the center?” for WHAT, “Where did the most significant change occur?” for WHERE, and “Which area shows the most dramatic change?” for COMPARE (Liu et al., 30 Jun 2026).

The relation among the three layers is integral to the benchmark’s design. The original change masks provide the pixel-level “where” signal. JL1-CC adds natural-language descriptions of “what changed” and often “where/how much” in free-form text. JL1-QA then turns those same image pairs into interactive, queryable examples that test whether a model can answer diverse questions about the same changes, including existence, location, scale, temporal state, causation, detail, and relative comparison (Liu et al., 30 Jun 2026).

4. Annotation pipeline and quality control

A key contribution of JL1-CC&QA is its three-stage annotation pipeline, presented as a scalable way to generate high-quality language annotations (Liu et al., 30 Jun 2026). The stages are multi-modal LLM generation, vision-grounded LLM judging, and human expert verification.

Stage 1 is multi-modal LLM generation. For captions, the model Kimi-K2.6 receives the pre-event image, post-event image, binary change mask, and spatial metadata including CAR and a textual description of the main change region, and is prompted to generate five captions from different perspectives: change type, spatial location, visual appearance, scale, or implication (Liu et al., 30 Jun 2026). For QA, the generation stage uses the same visual inputs plus contextual metadata from JL1-CC: CAR, the spatial change region description, and up to three selected captions. The model generates five QA pairs per image, each sampled from the question taxonomy, and the prompt forbids copying exact numerical metadata such as change percentages into answers because those values cannot be reliably inferred from visual inspection (Liu et al., 30 Jun 2026).

Stage 2 is vision-grounded LLM judging. For JL1-CC, each caption is scored 1–10 on five criteria: accuracy, specificity, spatial correctness, naturalness, and informativeness. The top three captions are retained, with ties at the cutoff preserved. For JL1-QA, each QA pair is scored 1–10 on four criteria: answer accuracy, question quality, answer completeness, and redundancy. QA pairs below 7 are discarded (Liu et al., 30 Jun 2026). The paper identifies common rejection reasons explicitly: hallucinated precise percentages, redundancy with other QA pairs, and vague answers.

Stage 3 is human expert verification, in which domain experts review a subset of generated annotations to verify factual accuracy and expose systematic errors, providing a final quality safeguard (Liu et al., 30 Jun 2026). This design combines automated scale with manual oversight. A plausible implication is that the benchmark attempts to preserve the throughput advantages of LLM-based annotation without treating automated generation or automated judging as sufficient on their own.

5. Statistical profile of captions and QA pairs

JL1-CC provides 17,021 quality-verified captions describing diverse land-cover transformations (Liu et al., 30 Jun 2026). The paper reports that 25,000 candidate captions were generated at five per image pair and then filtered down to 17,021, giving a 68.1% pass rate. The selected captions average 26.2 words in length and contain 7,458 unique tokens. The judge score distribution is reported as 39.8% of captions scoring 9–10, 42.3% scoring 7–8, and 17.9% scoring below 7 and being rejected (Liu et al., 30 Jun 2026).

JL1-QA contains 20,060 QA pairs retained from 24,995 generated QA pairs, corresponding to an 80.3% pass rate, which the paper states is higher than JL1-CC because the QA generation stage is context-enriched using verified captions (Liu et al., 30 Jun 2026). The average question length is 11.1 words and the average answer length is 19.3 words. The selected QA pairs are distributed across all eight types, with YES/NO most frequent at 21.2%, followed by WHERE at 18.3% and WHAT at 16.9% (Liu et al., 30 Jun 2026).

The paper also reports qualitative indicators of lexical and semantic diversity. A word cloud for captions shows prominent terms such as upper, lower, bare soil, agricultural, building, road, and water, which it interprets as reflecting both spatial language and land-cover diversity (Liu et al., 30 Jun 2026). Together with the CAR histogram and QA-type distribution, these analyses are used to argue that the generated annotations are diverse, spatially grounded, and quality-controlled.

Notably, the paper’s emphasis is on construction quality and descriptive statistics rather than broad downstream benchmarking. It presents caption and QA score histograms, the CAR distribution over the 5,000 image pairs, and the distribution of the eight QA types, but does not include a full leaderboard, task-specific benchmark results, or model comparison tables for captioning or QA performance in the excerpt provided (Liu et al., 30 Jun 2026).

6. Significance, use cases, and limitations

The paper states three principal contributions. First, it constructs a change understanding benchmark with binary change masks, natural-language change descriptions, and QA pairs over 5,000 bi-temporal satellite image pairs. Second, it introduces a scalable and reproducible annotation pipeline combining multi-modal LLM generation, vision-grounded LLM judging, and human verification. Third, it provides comprehensive dataset statistics together with public release of the data and code (Liu et al., 30 Jun 2026).

Its broader contribution is to facilitate joint learning and evaluation across CD, CC, and QA for remote sensing (Liu et al., 30 Jun 2026). Because the masks, captions, and questions are aligned on the same image pairs, the benchmark can support cross-task supervision and cross-task evaluation. This suggests a setting in which pixel-level localization, free-form semantic description, and interactive QA are not isolated benchmarks but coordinated views of the same underlying change event.

The paper is correspondingly modest about limitations. JL1-CC&QA is built from a single sensor family and a single modality—RGB Jilin-1 optical imagery—so it does not directly address multispectral, hyperspectral, SAR, or cross-modal change understanding (Liu et al., 30 Jun 2026). The annotation pipeline relies heavily on LLM-generated text and automated judging, which, although grounded and human-checked, may still inherit biases, style artifacts, or coverage limitations from the prompt design and the selected taxonomy. The QA set is constrained to eight question types, which, while broad, does not fully span open-world dialog or complex multi-turn reasoning. Finally, the paper emphasizes construction and statistics more than downstream baselines, so the practical performance ceiling and task difficulty for existing models are not fully quantified in the excerpt (Liu et al., 30 Jun 2026).

In summary, JL1-CC&QA is a carefully designed extension of JL1-CD that transforms a classic binary change detection corpus into a richer benchmark for semantic change captioning and natural-language QA. Its defining characteristic is the unification of masks, captions, and questions on the same 5,000 bi-temporal Jilin-1 pairs, with 17,021 verified captions and 20,060 verified QA pairs produced through a three-stage, quality-controlled annotation pipeline (Liu et al., 30 Jun 2026).