JL1-QA: Remote Sensing Change QA

• JL1-QA is a remote-sensing VQA benchmark that recasts binary change detection into a natural-language task analyzing pre- and post-event satellite images.
• It leverages high-resolution Jilin-1 imagery and a multi-stage, LLM-assisted annotation pipeline to produce quality-verified QA pairs across diverse change types.
• The benchmark integrates change masks, captions, and QA to support multi-task learning and facilitate advanced semantic, spatial, and causal change analysis.

JL1-QA is the question-answering component of the JL1-CC&QA benchmark, an extension of the JL1-CD binary change detection dataset that recasts remote-sensing change analysis from a purely pixel-level localization problem into a natural-language, change-understanding task over bi-temporal satellite imagery. Its central objective is to move beyond identifying where change occurred and toward answering what changed, where, how much, and why, using free-form textual responses grounded in paired pre-event and post-event images from the Jilin-1 satellite (Liu et al., 30 Jun 2026).

1. Benchmark lineage and observational substrate

JL1-QA is built on JL1-CD, a large binary change detection benchmark acquired by the Jilin-1 high-resolution optical satellite. The source imagery has a spatial resolution of 0.5–0.75 m ground sample distance, RGB format, and image size $512 \times 512$ pixels. JL1-CD contains 5,000 bi-temporal image pairs, each with a pre-event image $I_A$, a post-event image $I_B$, and a pixel-level binary change mask $M$ annotated by professional interpreters. The dataset uses a 4,000/1,000 train/test split and covers multiple Chinese provinces, including Shandong, Ningxia, Anhui, Hebei, and Hunan, over 2022–2023 (Liu et al., 30 Jun 2026).

The underlying change phenomena include both anthropogenic and natural processes. The benchmark references buildings, roads, hardened surfaces, photovoltaic panels, forest, grassland, cropland, water bodies, and other natural surfaces. Its change area ratio (CAR) spans near 0–100%, with mean 9.7% and median 2.9%, yielding a long-tailed distribution. This matters for JL1-QA because the QA layer inherits both the semantic heterogeneity and the severe spatial imbalance of the original change detection corpus (Liu et al., 30 Jun 2026).

JL1-CC&QA augments JL1-CD with two semantic annotation layers over the same imagery: JL1-CC for change captioning and JL1-QA for change question answering. JL1-CC contributes 17,021 quality-verified captions, while JL1-QA contributes 20,060 quality-verified question-answer pairs. The resulting benchmark aligns binary masks, captions, and QA on the same 5,000 image pairs, although 4,999 of those pairs have at least one associated QA sample (Liu et al., 30 Jun 2026).

2. Task definition and semantic scope

The task is formulated as follows: given a bi-temporal image pair $(I_A, I_B)$ and a natural-language question $Q$ about the change, produce an answer $A$ grounded in the visible surface changes. Formally, the model learns a mapping

$f:(I_A, I_B, Q)\mapsto A,$

where $A$ is free-form text rather than a member of a fixed answer vocabulary (Liu et al., 30 Jun 2026).

This design places JL1-QA within change-centric remote-sensing VQA rather than multiple-choice VQA. The benchmark is explicitly open-ended: even YES/NO items are expressed as natural-language answers rather than class labels. The question set covers urban expansion, road construction, land clearing, afforestation and deforestation, agricultural changes, waterbody changes, and PV deployment, ranging from small local modifications to almost full-frame transformations (Liu et al., 30 Jun 2026).

JL1-QA defines eight question types intended to capture distinct aspects of change reasoning.

Type	Focus	Typical answer style
YES/NO	Existence or binary condition	Natural-language affirmation or negation
WHAT	Semantic description of change	Open-ended description
WHERE	Spatial localization	Relative location phrase
HOW MUCH	Qualitative magnitude or extent	Localized vs large-scale description
BEFORE/AFTER	Temporal state comparison	Contrastive before/after statement
CAUSE	Plausible causal interpretation	Activity or process explanation
DETAIL	Fine-grained visual identity	Morphological or categorical detail
COMPARE	Relative comparison across regions	Region-level comparative answer

The benchmark operates simultaneously at multiple semantic granularities: region-level references such as “upper half” or “lower-right corner,” object or land-cover level references such as buildings, roads, cropland, forest, water, and PV arrays, and scene-level transitions such as rural-to-urban development. Temporal reasoning is explicit in BEFORE/AFTER questions and implicit in much of the remaining taxonomy. Counting-style questions are not a major focus; magnitude is usually expressed qualitatively rather than through exact counts (Liu et al., 30 Jun 2026).

A common misconception is to treat JL1-QA as merely a textual wrapper over binary change masks. In fact, the task is broader: the mask supports localization and prompt conditioning, but the answers target semantic interpretation, temporal comparison, and limited causal reasoning rather than binary changed-versus-unchanged classification alone (Liu et al., 30 Jun 2026).

3. Annotation workflow and quality control

JL1-QA was produced through a three-stage, LLM-assisted pipeline designed to scale annotation while retaining control over factual grounding. In Stage 1, a context-enriched multi-modal LLM generates candidate QA pairs for each image pair using $I_A$, $I_A$0, $I_A$1, CAR, a textual description of the main change region, and up to three selected captions from JL1-CC. The generator is instructed to produce five QA pairs per image, with question types randomly sampled from the eight-type taxonomy to diversify coverage. The prompt also explicitly prohibits copying precise numeric metadata such as exact CAR or exact pixel counts into answers, because such quantities are not visually observable and would constitute hallucination. This process yields 24,995 candidate QA pairs (Liu et al., 30 Jun 2026).

Stage 2 applies a second LLM as a vision-grounded judge. The judge sees the image pair, change mask, question, answer, CAR, change-region description, and existing captions, and assigns 1–10 integer scores on four criteria: answer accuracy, question quality, answer completeness, and redundancy. QA pairs with judge score below 7 are rejected. The most common rejection causes are precise percentages or numbers that cannot be visually verified, redundant questions, and vague or underspecified answers. After this filtering step, 20,060 QA pairs remain, corresponding to an 80.3% pass rate (Liu et al., 30 Jun 2026).

Stage 3 adds human expert verification on a subset of samples. The review is intended to validate factual grounding, ensure that questions truly require bi-temporal reasoning rather than single-image interpretation, and diagnose systematic failure modes such as consistent confusion between PV arrays and industrial roofs. Acceptance criteria include correct land-cover and change semantics, correct spatial references, no reliance on metadata-only information, and reasonable causal statements grounded in visible evidence. The paper reports this expert review qualitatively; no explicit inter-annotator agreement statistics such as Cohen’s $I_A$2 are given (Liu et al., 30 Jun 2026).

This workflow clarifies another frequent misunderstanding: JL1-QA is not a purely synthetic benchmark. Its annotations originate in LLM generation, but they are constrained by masks and captions, filtered by vision-grounded judging, and reviewed by remote-sensing experts (Liu et al., 30 Jun 2026).

4. Statistical profile and representational characteristics

JL1-QA covers 4,999 image pairs, with 3,999 in train and 1,000 in test. It contains 24,995 generated QA pairs and 20,060 selected QA pairs, distributed as 16,055 train and 4,005 test annotations. The average density is 4.01 selected QA pairs per image pair. Average question length is 11.1 words and average answer length is 19.3 words. Judge scores have mean 7.9 and median 9.0, and 51.4% of selected QA pairs receive scores in the 9–10 range (Liu et al., 30 Jun 2026).

Question-type distribution is broad but not uniform. YES/NO accounts for 21.2%, WHERE for 18.3%, and WHAT for 16.9%; the remainder is distributed among HOW MUCH, BEFORE/AFTER, CAUSE, DETAIL, and COMPARE. The distribution emphasizes binary existence, location, and description queries while still retaining higher-level reasoning types. This suggests that JL1-QA is simultaneously a benchmark for basic change grounding and for more demanding semantic interrogation (Liu et al., 30 Jun 2026).

The dataset also exhibits pronounced semantic and spatial diversity. Questions reference residential expansion, industrial parks, new roads and intersections, new PV farms and roof-mounted solar arrays, cropland expansion, conversion to built-up land, fallow fields, changes in field patterning, forest clearing or planting, grassland changes, water level shifts, and new ponds. Spatial descriptions are usually coarse directional phrases such as “upper left,” “center-right,” or “lower half,” which align with how change regions are summarized for the LLM during annotation (Liu et al., 30 Jun 2026).

Alignment across modalities is a central property of the benchmark. The binary change mask is used to compute CAR, derive textual change-region descriptions, and steer question generation toward changed areas rather than static background. JL1-CC captions serve as contextual priors during QA generation, constraining the LLM to remain semantically consistent with already validated descriptions. Many QA answers paraphrase or refine caption content, while the questions probe specific aspects of those descriptions. The paper attributes the comparatively high QA pass rate partly to this caption-conditioned design, since captions reduce hallucinations at the QA stage (Liu et al., 30 Jun 2026).

5. Evaluation status and methodological issues

JL1-QA is intended as a remote-sensing VQA benchmark with input $I_A$3 and natural-language output $I_A$4, optionally using $I_A$5 if a model chooses to incorporate masks. It inherits the JL1-CD split directly: 3,999 training pairs with 16,055 QA pairs and 1,000 test pairs with 4,005 QA pairs. No special cross-scene or change-type splits are defined in the visible text (Liu et al., 30 Jun 2026).

A notable feature of the released description is the absence of an official experimental section for JL1-QA in the excerpt. No baseline architectures, no per-question-type results, no ablations, and no official metric definitions are provided there. The text states only that exact or normalized answer matching would be reasonable for short factual answers, while token-level F1 or BLEU/ROUGE could be considered for more descriptive answers. It also states explicitly that the excerpt does not list concrete metrics such as BLEU, ROUGE, CIDEr, or VQA-style consensus scores (Liu et al., 30 Jun 2026).

This omission is methodologically important. Open-QA evaluation research has shown that Exact Match can underestimate performance when answers vary in expression or are embedded in longer sentences, and that different automatic evaluators may not preserve the same model rankings as human judgment (Wang et al., 2023). A plausible implication is that JL1-QA, because it combines short factual responses, spatial phrases, and open-ended descriptive answers, will require evaluation protocols that are sensitive to question type and calibrated against human judgments rather than a single lexical metric.

The benchmark’s own construction pipeline reinforces this point. Since the annotation process already uses a vision-grounded LLM judge to assess answer accuracy, question quality, completeness, and redundancy, JL1-QA sits close to a broader trend in grounded QA toward explicit verification and evaluator design. Related work in grounded clinical QA has used modular pipelines with evidence identification, answer generation, and verification stages optimized separately, suggesting a plausible methodological template for future JL1-QA systems, although the JL1-CC&QA paper does not report such baselines (Majeedi et al., 11 May 2026).

6. Role within multi-task change understanding

JL1-QA is tightly coupled to JL1-CC rather than being an independent annotation layer. JL1-CC captions are generated first from $I_A$6, and JL1-QA generation then conditions on those captions together with CAR and change-region descriptions. In this sense, QA is structurally dependent on an existing, verified textual summary of the visible changes. The benchmark is therefore designed not only for stand-alone QA, but also for joint training and evaluation across binary change detection (CD), change captioning (CC), and question answering (QA) (Liu et al., 30 Jun 2026).

This structure makes multi-task learning a natural research direction. The benchmark shares the same image pair and change mask across tasks while exposing complementary outputs: masks identify changed regions, captions describe major changes holistically, and QA supports targeted interrogation of particular semantic, spatial, temporal, and causal aspects. The paper gives a generic multi-task objective,

$I_A$7

but does not report empirical multi-task experiments in the visible excerpt (Liu et al., 30 Jun 2026).

The benchmark’s stated novelty lies in the joint availability of masks, captions, and QA over the same imagery. Prior work is described as typically offering one or two of these layers—for example, BCD or SCD masks only, change captioning only, or change-oriented VQA without pixel-level masks or aligned captions—whereas JL1-CC&QA is presented as the first benchmark family in which all three layers coexist over the same image set (Liu et al., 30 Jun 2026). This unification is significant because it supports cross-task supervision, error analysis across semantic levels, and the study of explainability in change detection systems.

7. Access, practical use, and limitations

JL1-CC&QA, including JL1-QA, is distributed through the JL1-CD GitHub repository at https://github.com/circleLZY/JL1-CD. JL1-CD provides bi-temporal RGB image pairs, binary change masks, and split files, while JL1-CC&QA adds caption annotations and QA annotations. The excerpt does not specify the exact file format, though it describes a typical experimental workflow: obtain the images, masks, and splits; load the QA annotations; feed $I_A$8 into a model; and train to predict $I_A$9 with a language-modeling or sequence-to-sequence loss (Liu et al., 30 Jun 2026).

Several limitations are explicit. First, the visible text does not provide baseline models or official evaluation metrics. Second, human verification is described qualitatively, without inter-annotator agreement statistics. Third, the benchmark emphasizes qualitative magnitude rather than exact counting, so it is not primarily a counting-oriented change QA resource. Fourth, the excerpt does not state an explicit license or usage restrictions; users are directed to check the repository for license terms, citation requirements, and any privacy or security considerations (Liu et al., 30 Jun 2026).

Within those limits, JL1-QA constitutes a change-focused VQA layer over very-high-resolution bi-temporal satellite imagery, with 20,060 quality-verified QA pairs spanning existence, description, location, magnitude, temporal comparison, causation, detail, and comparison. Its main significance lies not in replacing pixel-level change detection, but in extending it into a unified framework for semantic, interactive, and multi-task change understanding in remote sensing (Liu et al., 30 Jun 2026).