JL1-CC: Remote Sensing Change Captioning

• JL1-CC is a remote sensing change captioning layer that enriches JL1-CD by linking bi-temporal Jilin-1 images with detailed natural language descriptions.
• It employs a rigorous three-stage annotation pipeline—multi-modal LLM generation, vision-grounded evaluation, and expert verification—to ensure high-quality, semantically rich captions.
• Its integration with segmentation masks and change-oriented QA fosters multi-task training, improving both spatial grounding and semantic specificity in change understanding.

JL1-CC is the change captioning layer of the JL1-CC&QA benchmark for remote sensing change understanding. It extends the JL1-CD dataset by attaching natural-language descriptions to the same bi-temporal image pairs and binary change masks, thereby shifting the task definition from identifying where change occurred to describing what changed and, where visually supportable, why it changed. In its released form, JL1-CC contains 17,021 quality-verified captions over 5,000 bi-temporal Jilin-1 image pairs, aligned with both pixel-level masks and a complementary change-oriented question answering layer, which makes it a unified resource for multi-task supervision and cross-task evaluation (Liu et al., 30 Jun 2026).

1. Position within the JL1-CD benchmark family

JL1-CC is defined as the captioning layer added on top of JL1-CD, a remote sensing change detection benchmark in which each image pair has a professionally annotated pixel-level binary change mask (Liu et al., 19 Feb 2025). The source imagery consists of RGB bi-temporal pairs from the Jilin-1 high-resolution optical satellite. JL1-CC therefore preserves the spatially explicit supervision of change detection while adding free-form language aligned to the same scenes.

The benchmark design is explicitly tripartite. JL1-CD provides binary masks indicating where change occurred; JL1-CC provides captions that describe the visible transformations; and JL1-QA provides change-oriented question-answer pairs over the same image set. Because all three layers are aligned on the same 5,000 bi-temporal pairs, the resource supports unified training and cross-task evaluation rather than isolated treatment of segmentation, captioning, and QA.

A central methodological distinction is that JL1-CC moves beyond discrete, closed-taxonomy labels. The captions are intended to encode open-vocabulary semantics, including land-cover transformation type, spatial context, scale, and plausible human or natural drivers observable from imagery. In this sense, the benchmark occupies an intermediate position between conventional change detection and more general vision-language modeling for Earth observation.

2. Data composition and semantic coverage

The underlying dataset comprises 5,000 bi-temporal pairs captured by the Jilin-1 satellite, with 0.5–0.75 m ground sample distance and image tiles of 512 × 512 pixels (Liu et al., 30 Jun 2026). The imagery spans multiple provinces in China—Shandong, Ningxia, Anhui, Hebei, and Hunan—and was acquired from 2022 to 2023. Curation removed blur, cloud cover, and extreme illumination artifacts.

Component	Value	Notes
Image pairs	5,000	RGB, 512 × 512, 0.5–0.75 m GSD
Split	4,000 train / 1,000 test	Shared across masks, captions, and QA
Selected captions	17,021	13,616 train, 3,405 test
Captions per pair	3.40 average	Top 3 retained on average
CAR	Mean 9.7%, median 2.9%	Near-zero to 100%, long-tailed

The change area ratio spans near-zero to 100%, with mean 9.7% and median 2.9%, which reflects a long-tailed distribution of change magnitudes. This matters for captioning because many image pairs contain small or localized changes rather than dominant scene-wide transformations.

Caption statistics further characterize the linguistic layer. The mean caption length is 26.2 words. The selected captions contain 7,458 unique tokens overall, with train vocabulary 6,837 and test vocabulary 4,064. The pipeline generates five candidates per image pair and retains the top three on average, producing 17,021 selected captions across the 5,000 pairs.

The semantic coverage includes both anthropogenic and natural transformations. Typical descriptions include farmland to buildings or hardstand/industrial yards, road construction or widening and new intersections, vegetation clearing to bare soil, cropland fallow, planting, harvesting, conversion to greenhouses, water body expansion or contraction with new ponds or canals, and photovoltaic installations on rooftops or open fields. A notable stylistic feature is frequent use of spatial descriptors such as upper-left, lower-right, central band, corner, and edge, which mirror the change mask cues and ground the language in specific regions.

3. Annotation pipeline and quality control

JL1-CC uses a three-stage annotation pipeline designed for scale and quality assurance (Liu et al., 30 Jun 2026). In Stage 1, multi-modal LLM generation takes as input the pre-event image $I_A$, post-event image $I_B$, the change mask $M$, and spatial metadata including CAR and a natural-language description of the primary change region such as “upper-left area.” The model/tool is Kimi-K2.6, which outputs five diverse candidate captions per image pair. These candidates are intended to emphasize different facets, including change type, spatial reference, visual appearance, scale, and implication.

Stage 2 applies vision-grounded LLM judging. For each candidate caption, a second LLM call evaluates the text against $I_A$ and $I_B$ on five dimensions scored on a 1–10 integer scale: accuracy, specificity, spatial correctness, naturalness, and informativeness. Captions scoring below 7 are rejected, and the top-3 are retained, with ties at the cutoff retained. From 25,000 generated captions, 17,021 were retained, corresponding to a 68.1% pass rate. The score distribution is reported as 39.8% achieving 9–10, 42.3% at 7–8, and 17.9% rejected below 7.

Stage 3 adds human expert verification. Domain experts review a subset of selected captions to validate factual accuracy and identify systematic issues such as persistent spatial misreferences or class confusions. The paper does not report inter-annotator agreement or exact verification rates, so the human audit is described qualitatively rather than through a formal reliability coefficient.

The ambiguity-handling policy is explicit. Hallucinations and unwarranted precision are penalized, including claims about exact counts or measurements not supported by imagery. Vague or overly generic statements are down-scored under specificity and informativeness. Incorrect locational references are penalized under spatial correctness, and captions with unclear localization are typically filtered out. Taken together, these rules make the benchmark closer to grounded semantic description than unconstrained image captioning.

4. Evaluation protocol and learning objectives

JL1-CC is intended to be evaluated with standard image captioning metrics that correlate with human judgments, although the paper does not prescribe paper-specific metrics or formulas (Liu et al., 30 Jun 2026). The recommended reporting suite is BLEU-1–4, ROUGE-L, METEOR, CIDEr-D, and SPICE on the official 1,000-pair test split, together with qualitative examples.

The definitions given for these metrics are standard. BLEU-N uses modified $n$-gram precision with clipping and a brevity penalty:

$$\mathrm{BLEU} = BP \cdot \exp\!\left(\sum_{n=1}^{N} w_n \cdot \log p_n\right).$$

ROUGE-L is based on longest common subsequence with

$$P_{LCS} = \frac{LCS(X,Y)}{|X|}, \qquad R_{LCS} = \frac{LCS(X,Y)}{|Y|},$$

and an $F$-measure

$$F_{LCS} = \frac{(1+\beta^2)\,R_{LCS}\,P_{LCS}}{R_{LCS}+\beta^2\,P_{LCS}}.$$

CIDEr-D computes TF-IDF weighted $I_B$0-gram similarity with cosine similarity and length-dependent Gaussian damping. METEOR aligns unigrams via exact, stem, and synonym matches and applies a fragmentation penalty. SPICE parses captions into scene graphs and computes an $I_B$1-score over matching tuples. For a dataset whose labels explicitly encode change type, spatial reference, and contextual semantics, the coexistence of lexical and scene-graph metrics is important: lexical overlap alone does not exhaust the notion of grounded change description.

The paper does not release quantitative captioning baselines or training specifics, but it does list standard training objectives. Cross-entropy teacher forcing is written as

$I_B$2

where $I_B$3 denotes the bi-temporal input and optional metadata. Sequence-level reinforcement learning, for example self-critical sequence training optimizing CIDEr, is written as

$I_B$4

The paper also states that it does not report quantitative captioning baselines or benchmark results. It instead identifies plausible model classes practitioners can evaluate, including transformer-based captioners adapted to bi-temporal inputs and change masks, vision-LLMs adapted for remote sensing with two-image inputs and optional mask channels, and instruction-tuned VLMs with prompts tailored to describing changes between pre- and post-event images.

5. Multi-task integration and model usage

Because JL1-CD, JL1-CC, and JL1-QA share the same image pairs, JL1-CC is naturally embedded in a multi-task formulation rather than functioning only as a stand-alone caption corpus (Liu et al., 30 Jun 2026). The benchmark description proposes joint training with a shared bi-temporal encoder and three task-specific heads: segmentation for binary masks, captioning with a language decoder, and QA with question conditioning. In this setup, masks can guide attention or region-of-interest pooling, which directly ties linguistic generation to spatial evidence.

Cross-task supervision is another design feature. Captions and QA can regularize change semantics, while masks regularize spatial grounding. The QA generation pipeline can be conditioned on top-ranked JL1-CC captions to reduce hallucination. The paper does not quantify multi-task gains; a plausible implication is that complementary supervision may improve both localization fidelity and semantic specificity.

Operational guidance is also provided. Inputs are 512 × 512 RGB tiles at 0.5–0.75 m GSD. Pixel normalization is recommended, and radiometric augmentations such as brightness, contrast, and hue adjustments should be applied with care so as not to alter change cues. Spatial augmentations such as flips and rotations should be applied identically to $I_B$5, $I_B$6, and $I_B$7 to preserve alignment. For bi-temporal modeling, one may concatenate pre- and post-event channels, use separate branches with cross-temporal attention, and add the mask as an extra channel or ROI prior. Explicit encoding of the before/after distinction is recommended because models may otherwise swap temporal roles.

The training and testing protocol follows the official split: train on the 4,000-pair training set, validate on a held-out part of that training set, and evaluate on the 1,000-pair test split. The benchmark notes common strengths and limitations for captioning systems. Robust performance is typically associated with large-scale anthropogenic changes such as roads and buildings and with clear spatial references; failure modes often involve subtle vegetation dynamics, small photovoltaic installations, or confusion arising from shadow and seasonal differences.

6. Limitations, access, and nomenclatural ambiguity

JL1-CC has several stated limitations and biases (Liu et al., 30 Jun 2026). The imagery is geographically concentrated in selected provinces in China and temporally concentrated in 2022–2023, so models trained on it may not generalize globally without adaptation. The long-tailed CAR distribution means that many pairs have small change regions, which can bias systems toward large, salient transformations. Caption subjectivity remains an issue even after LLM judging and expert verification; terminology and granularity can vary, including descriptions such as “industrial” that may not be definitively inferable from imagery alone. The benchmark explicitly advises treating captions as weakly supervised labels and using human-in-the-loop quality control for critical applications.

Access is through the repository at https://github.com/circleLZY/JL1-CD, which hosts JL1-CD images and masks and releases JL1-CC and JL1-QA annotations. Captions are linked to image-pair identifiers and can be matched programmatically to pre/post image paths and the corresponding masks. The paper does not specify a JSON schema, and license terms and usage restrictions are not specified in the paper; the repository is the authoritative source for current file structures and licensing.

The term “JL1-CC” is also ambiguous across arXiv-indexed technical literature. In a high-energy-physics context, “JL1-CC” appears as the label for single-jet production in charged-current deep inelastic scattering within fully differential NNLO QCD predictions (Niehues et al., 2018). In another particle-physics hardware context, the paper does not use or define the name “JL1-CC” and instead identifies the Summary Trigger Unit as the effective central controller for Level-1 jet triggering (Bourrion et al., 2010). For remote sensing and vision-language research, however, JL1-CC denotes the change captioning layer of the JL1-CC&QA benchmark, aligned with JL1-CD masks and JL1-QA pairs on the same Jilin-1 imagery.