---
title: 'Landsat30-AU-Cap: 30-m Landsat Captioning'
url: https://www.emergentmind.com/topics/landsat30-au-cap
type: topic
---

# Landsat30-AU-Cap: 30-m Landsat Captioning

Landsat30-AU-Cap is the captioning component of **Landsat30-AU**, a vision-language dataset built from **30 m resolution Landsat imagery over Australia**. It contains **196,262 image-caption pairs** derived from imagery collected by **Landsat 5, 7, 8, and 9** across **1988 to 2024**, and it was introduced to support training and evaluation of models that can describe, understand, and interact in natural language with Landsat imagery. The dataset is framed as a response to a specific gap in remote-sensing vision-language resources: most existing datasets emphasize higher-resolution imagery, shorter temporal windows, fewer Landsat sensors, or text supervision that is poorly aligned with what is actually visible at Landsat scale [2508.03127].

## 1. Definition and role within Landsat30-AU

Landsat30-AU has two components: **Landsat30-AU-Cap**, with **196,262 image-caption pairs**, and **Landsat30-AU-VQA**, with **17,725 human-verified visual question answering samples**. The two parts are complementary. Landsat30-AU-Cap provides descriptive natural-language supervision for **free-form scene description**, whereas Landsat30-AU-VQA probes **structured question answering and reasoning** across eight remote sensing domains. The paper explicitly states that Landsat30-AU-Cap “supports training and evaluation of captioning models on real-world, multi-sensor, multi-temporal satellite imagery,” while Landsat30-AU-VQA is intended to “capture common reasoning challenges in low-resolution imagery” [2508.03127].

The dataset’s rationale is tightly coupled to the semantics of **30 m Landsat imagery**. The paper argues that object-centric captioning conventions from higher-resolution remote-sensing datasets do not transfer cleanly to Landsat, because many commonly named objects in those resources—such as cars, small rooftops, and lane markings—are not visible at this spatial scale. Landsat30-AU-Cap was therefore designed around **Landsat-native resolution awareness**, **multiple Landsat satellites**, **long temporal coverage**, and **text supervision** appropriate for image-language alignment. A common misconception is that remote-sensing captioning is largely resolution-agnostic; the dataset is explicitly constructed to reject that assumption.

## 2. Data composition, sampling, and geographic-temporal scope

The imagery is sourced from the **Digital Earth Australia (DEA) Analysis Ready Data (ARD)** archive. Samples are represented as **\(256 \times 256\)** RGB tiles at **30-meter ground sample distance**, produced in the main text from **Bands 4/3/2 (Red/Green/Blue)** and, in the appendix, from DEA ARD **nbart_red, nbart_green, nbart_blue** bands as **true-color RGB patches**. At 30 m per pixel, each tile corresponds to a ground footprint of roughly **\(7{,}680 \times 7{,}680\ \mathrm{m}^2\)**. Coverage is restricted to **Australia**, with the paper noting stronger spatial density in **eastern and southwestern Australia**, consistent with the country’s main populated and agriculturally significant regions.

| Aspect | Value | Notes |
|---|---|---|
| Caption corpus size | 196,262 image-caption pairs | Final released Landsat30-AU-Cap |
| Sensor coverage | Landsat 5, 7, 8, 9 | Multi-sensor Landsat archive |
| Spatial representation | \(256 \times 256\) RGB tiles at 30 m GSD | Roughly \(7{,}680 \times 7{,}680\ \mathrm{m}^2\) footprint |
| Geographic scope | Australia only | Density concentrated in eastern and southwestern regions |
| Temporal span | 1988–2024 | Described as 36 years and “more than 36 years” |
| Seed verified caption set | 1,005 pairs | Split into 70% train / 15% validation / 15% test |
| Corpus statistics | Vocabulary 4,405; average caption length 165.4 words; MSTTR 0.82 | Reported for Landsat30-AU caption corpus |

The sampling strategy is deliberate rather than opportunistic. Australia was partitioned into a static grid of approximately **2,500 AOIs**. For each AOI, the authors aimed to select **one image per year**, and to encourage seasonal diversity the search began in a different quarter each year according to **year modulo 4**; the appendix gives **2001**, beginning in **Q2 (April 1st)**, as an example. The **first image passing the initial quality filter** was selected. Before final filtering, Stage 1 produced **over 400,000** candidate RGB tiles; after additional filtering and selection, the dataset was reduced to **196,262 high-quality Landsat images**, which are the same images used for caption generation [2508.03127].

Initial filtering retained only patches marked as at least **\(99.5\%\)** cloud-free by DEA ARD metadata. Manual inspection showed that this criterion alone was insufficient, because many cloudy scenes still passed and many retained scenes were semantically uninformative. A second filtering stage used **Qwen2.5-VL-7B** to classify each patch as **“cloudy”** or **“clear,”** with “cloudy” defined broadly enough to include severe sensor artifacts, striping, missing data, over-exposure, and other conditions that obstruct useful interpretation.

From the construction process, each sample is associated with at least the Landsat image tile, geographic location or **geohash**, capture time or acquisition date, satellite identifier, region-level land-cover labels, mapped OSM land-use categories, and a generated or reviewed caption. The paper explicitly notes that the draft imagery metadata included **geohash, capture time, and satellite metadata**, and that Landsat30-AU retains **precise geographic coordinates and acquisition dates**. It does **not** provide a formal JSON schema or file-format specification, so anything beyond this field set would be speculative.

## 3. Human-in-the-loop construction pipeline

Landsat30-AU-Cap was built with a **three-stage human-in-the-loop bootstrapped pipeline** using **generic VLMs**, **iterative refinement**, and **human verification**. The first stage prepares imagery and auxiliary metadata. After RGB tile extraction and quality filtering, the pipeline extracts **OpenStreetMap tags** within each tile footprint. Because raw OSM labels are noisy, inconsistent, and often too fine-grained for 30 m imagery, they are normalized into a controlled vocabulary of **25 distinct land-use categories** appropriate for Landsat-scale interpretation. The paper provides explicit mappings such as **clinic \(\rightarrow\) urban_fabric**, **river/stream/creek/canal \(\rightarrow\) river_stream**, **farmland/orchard/vineyard \(\rightarrow\) cropland**, **forest/woodland/grassland/shrub \(\rightarrow\) natural_vegetation**, **residential/commercial/industrial/hospital/school \(\rightarrow\) urban_fabric**, and **road/street/highway/railway \(\rightarrow\) road_corridor**.

The same stage also extracts structured land-cover context from the **DEA Land Cover** product, an annual pixel-level classification over Australia. For each image tile, the dominant land-cover class is computed for **top-left**, **top-right**, **bottom-left**, **bottom-right**, **center**, and the **whole tile**. The appendix lists seven broad DEA Land Cover categories: **Cultivated Terrestrial Vegetation**, **Natural Terrestrial Vegetation**, **Natural Aquatic Vegetation**, **Artificial Surfaces**, **Natural Bare Surfaces**, **Water**, and **No Data**. These labels are used as conditioning signals rather than as caption targets.

The second stage fine-tunes three lightweight Landsat-specific modules: **region classification**, **caption generation**, and **caption review**. For region classification, **2,722 tile-region label sets** were manually validated, and **GPT-4o** was fine-tuned with an **80% train / 20% test** split using **3 epochs**, **batch size 4**, **learning rate multiplier 2**, and **seed 42**. On the fine-tuning set, the resulting model achieved **Subset Accuracy 0.278**, **Jaccard 0.630**, **Precision 0.768**, **Recall 0.722**, **F1 0.727**, **LRAP 0.826**, **nDCG 0.917**, **1-Hamming-loss 0.783**, and **1-ranking-loss 0.705**.

For caption generation, the authors manually curated and verified **1,005 image-caption pairs** whose text had to reference objects visible at **30 m** and align with the **acquisition date**. **GPT-4.1** was fine-tuned on this seed set using a **70% train / 15% validation / 15% test** split, **3 epochs**, **batch size 1**, **learning rate multiplier 2**, and **seed 42**, with fine-tuning conducted in **OpenAI Playground**. The captioning prompt conditions on the **image**, **region-level land-cover information**, and **OSM-derived land-use information**, and it explicitly instructs the model to **cross-validate** sources to avoid hallucinations. If, for example, land cover indicates **no water**, the model is instructed to ignore water-related OSM tags. This prompt asks for visible water bodies and their relative sizes, dominant land cover by area, artificial areas such as **“small town”** or **“city”** if visible, road corridors and directions if visible, spatial references such as **top**, **bottom**, **left**, **right**, and **center**, and a summary of the balance between bare and vegetated surfaces. Fine-tuning improved caption quality from **BLEU-4 0.152 / SPIDEr 0.438 / BERT-F1 0.901 / 1-CHAIR-s 0.522 / 1-CHAIR-i 0.864 / average length 140** to **BLEU-4 0.184 / SPIDEr 0.517 / BERT-F1 0.903 / 1-CHAIR-s 0.473 / 1-CHAIR-i 0.853 / average length 161**.

For caption review, the human verification process produced **9,440 image-sentence-decision triplets**, each labeled **keep** or **delete**. **Qwen2.5-VL-7B** was then fine-tuned as a sentence-level reviewer using a **70/15/15** split, **3 epochs**, **batch size 24**, **Adam**, **cosine** schedule, learning rate **\(2 \times 10^{-5}\)**, warm-up over the **first 6% of steps**, weight decay **\(1 \times 10^{-6}\)**, **Adam \(\beta_2 = 0.999\)**, **bfloat16** precision, and **8 \(\times\) NVIDIA L4-24G GPUs**. The model inspects an image and a single caption sentence, returning only **“keep”** or **“delete.”**

The third stage applies these modules at scale. A fine-tuned **GPT-4.1** captioner first produces an **Initial** caption from the image, region labels, and OSM tags. **Qwen2.5-VL-7B** then generates an **Extra** version by adding either one clearly visible missing object or one missing spatial or contextual relationship; the prompts explicitly forbid mention of absent features. The fine-tuned reviewer removes hallucinated or temporally inconsistent sentences to produce the **Reviewed** caption. On a held-out reference set, the authors compare **Initial**, **Extra**, and **Reviewed** captions using **BLEU-4**, **SPIDEr**, **BERTScore-F1**, and **CHAIR-s/i** reported as **1-CHAIR-s** and **1-CHAIR-i**, concluding that the **Reviewed** captions provide the best overall balance, specifically **SPIDEr 0.517** and **1-CHAIR-i 0.853**, and are therefore used throughout the released dataset [2508.03127].

## 4. Caption content, style, and semantic coverage

The captions are characterized in the paper as **detailed**, **visually grounded**, **resolution-aware**, and **semantically rich**. Their intended content includes **land cover**, **land use**, **water bodies**, **vegetation versus bare surface balance**, **artificial surfaces and settlements**, **roads and corridors**, **macro-objects and landscape patterns**, and **spatial relationships**. The emphasis is on semantics that are supportable at Landsat scale rather than on fine-grained object inventories that would be more appropriate for sub-meter or few-meter imagery.

The corpus is notably long-form for remote-sensing captioning. Reported linguistic statistics for Landsat30-AU show a **vocabulary size of 4,405**, **average caption length of 165.4 words**, and **MSTTR of 0.82**. During refinement experiments, **Reviewed** captions averaged around **161 words**. The prompts encourage objective, analytical descriptions, including orientation terms such as **top-left**, **center**, and **bottom-right**, discussion of dominance and extent, and mention of both dominant and minor visual elements. The result is natural language that is semi-standardized by prompt instructions but not reduced to rigid templates [2508.03127].

The paper repeatedly states that captions should mention only what is **visible** or otherwise **supportable at 30 m**. This is a core design principle. Visually ungrounded text and temporally mismatched content are removed during review, which directly addresses a recurrent problem in remote-sensing datasets assembled from OSM or other metadata sources that may mention objects too small to resolve or features no longer present at the image date. The captions are also **temporally grounded**: reviewers checked whether content aligned with the corresponding **acquisition date**, and the review stage was explicitly used to remove **temporally inconsistent** text.

Although the paper does not define a formal caption-domain taxonomy for Landsat30-AU-Cap analogous to the eight VQA categories, its prompts, mappings, and examples indicate broad coverage of **agriculture or cropland**, **natural vegetation**, **water bodies**, **urban fabric or settlements**, **road corridors**, **natural bare surfaces or arid landscapes**, **wetlands**, **rivers and streams**, and some **industrial or extractive features**. Human verification examples include the sentences **“A golf course appears in the image. Decision: Keep.”**, **“A small dam in the lower right. Decision: Keep.”**, and **“Grid of white oil or gas well pads. Decision: Keep.”** The word cloud reported for Landsat30-AU-Cap is dominated by **vegetation**, **bare surface**, and **water**, which is consistent with the dataset’s landscape-scale emphasis.

The paper does not report any caption-domain balancing procedure. A plausible implication is that topic frequency follows the AOI sampling design and the underlying Australian imagery distribution rather than an explicit semantic equalization scheme.

## 5. Benchmark protocol, model performance, and fine-tuning behavior

Landsat30-AU-Cap is also a benchmark for **image captioning on Landsat imagery**. In evaluation, models are given **only the raw image** and **not** the auxiliary land-cover or land-use metadata used during dataset construction. The benchmark prompt states that each pixel is **30 meters** and asks the model to describe land covers, features, surface types, and spatial relationships in detailed, domain-specific language, including both dominant and minor elements and explicit orientation terms, while basing descriptions only on observable features. This makes the benchmark a test of learned Landsat-specific visual understanding rather than metadata exploitation [2508.03127].

The captioning benchmark uses **BLEU-4**, **SPIDEr**, **BERTScore-F1**, **1-CHAIR-s**, **1-CHAIR-i**, and **Average Caption Length**. The paper describes **BLEU-4** as a 4-gram overlap measure, **SPIDEr** as a composite of **SPICE** and **CIDEr** intended to capture semantic alignment and informative phrase overlap, **BERTScore-F1** as contextual-embedding-based semantic similarity, and **CHAIR-s / CHAIR-i** as sentence- and instance-level hallucination rates, reported in complemented form so that higher **1-CHAIR** is better.

Eight base VLMs are evaluated: specialized models **EarthDial (4B)**, **RS-LLaVA (7B)**, **MiMo (7B)**, and **GLM-V (9B)**; general models **Qwen (7B)**, **LLaVA (8B)**, **Llama (11B)**, and **Gemma 3 (12B)**; and fine-tuned variants **Qwen-ft (7B)** and **Llama-ft (11B)**. The reported captioning results are exact. **EarthDial** achieves **BLEU-4 0.0210**, **SPIDEr 0.0726**, **BERTScore-F1 0.8379**, **1-CHAIR-s 0.5920**, **1-CHAIR-i 0.8197**, and average caption length **140**. **RS-LLaVA** reaches **SPIDEr 0.2095**, the highest among the specialized off-the-shelf models. Among general off-the-shelf models, **Llama** performs best on SPIDEr at **0.1695**. These results motivate the paper’s conclusion that off-the-shelf VLMs struggle to understand Landsat imagery.

The strongest gains come from lightweight fine-tuning. The benchmark fine-tunes **Qwen2.5-VL-7B** and the **Llama-3.2 vision-language model** using **LoRA** in the main text and **QLoRA** in the appendix, with **15% of the respective training data for each task**. The appendix specifies a **4-bit NormalFloat** quantized backbone, **rank-64 LoRA adapters**, **\(\alpha = 128\)**, **dropout 0.05**, insertion into all attention projections (**q, k, v, o**), the gated MLP stack (**gate, up, down**), and the cross-modal vision projector, trained for **1 epoch** with **AdamW**, learning rate **\(2 \times 10^{-4}\)**, **cosine annealing**, warm-up fraction **6%**, weight decay **\(10^{-6}\)**, and **8 \(\times\) NVIDIA L4-24G GPUs**.

For **Qwen**, fine-tuning improves **SPIDEr** from **0.1114** to **0.3054**, **BLEU-4** from **0.0350** to **0.1395**, **BERTScore-F1** from **0.8693** to **0.8935**, **1-CHAIR-i** from **0.7959** to **0.8549**, and average caption length from **124** to **157**. For **Llama**, fine-tuning improves **SPIDEr** from **0.1695** to **0.2767** and **BERTScore-F1** from **0.8800** to **0.8914**, but **1-CHAIR-s** declines from **0.5483** to **0.5224** and **1-CHAIR-i** from **0.8296** to **0.8016**, which the paper interprets as a trade-off between semantic quality and hallucination resistance. The abstract rounds the Qwen result as an improvement from **0.11** to **0.31 SPIDEr**. A careful reading suggests that the manually verified **1,005-pair** seed set underlies the captioning benchmark, with evaluation conducted on its held-out portion.

## 6. Limitations, caveats, and practical use

Several caveats are explicit or strongly implied. The most direct is **geographic restriction**: Landsat30-AU-Cap covers **Australia only**. Ecological, land-use, and climatic patterns are therefore region-specific, and the spatial distribution is denser in **eastern and southwestern Australia**, which likely biases the corpus toward populated and agriculturally significant environments. A second caveat is residual quality risk from **cloud contamination** and **sensor artifacts**. The need for a second Qwen-based filtering pass after the **\(99.5\%\)** ARD cloud-free threshold indicates that metadata filtering alone was insufficient.

A third caveat concerns **metadata noise and temporal mismatch**. The pipeline uses **OSM tags** and **DEA Land Cover** as supporting context, but OSM may be spatially or temporally mismatched, may refer to objects smaller than Landsat can resolve, and DEA Land Cover is an **annual summary product** rather than a per-image annotation. The construction procedure attempts to mitigate these problems by cross-validating modalities and deleting unsupported sentences, but it does not eliminate them. A fourth caveat is dependence on **generic VLM bootstrapping**: the caption corpus is not manually authored from scratch, and the final language may therefore retain some stylistic or semantic biases of **GPT-4.1** and **Qwen2.5-VL-7B**.

Human verification is a major part of the quality-control story, but the paper does **not** report the **number of annotators**, their **expertise levels**, **inter-annotator agreement**, **Cohen’s kappa**, or a formal **adjudication process**. Reviewers judged captions **sentence by sentence** with binary **keep/delete** labels, using the criterion of visual verifiability and, when necessary, consulting **pansharpened 15 m imagery** and third-party services such as **Google Earth** or **Google Maps**. This supports the dataset’s claim to visual grounding, but not via formal reliability statistics. The benchmark’s **image-only** setup is also a double-edged design choice: it yields a fairer test of Landsat understanding, but it may understate performance achievable in practical systems that are allowed to use metadata.

The dataset and code are reported as available at **https://github.com/papersubmit1/landsat30-au**. The paper cites upstream licensing for **DEA Land Cover** as **Creative Commons Attribution 4.0 Licence** and for **OpenStreetMap** as **ODbL**, but it does **not** specify the license of the final Landsat30-AU repository itself. That omission matters for downstream reuse. Overall, the dataset’s main significance lies in pairing **multi-sensor**, **multi-decadal**, **30 m Landsat imagery** with **long-form, reviewed, resolution-aware captions**, while its main caveat is that the supervision remains bootstrapped, region-specific, and only partially characterized from an annotation-governance perspective [2508.03127].

Source: https://www.emergentmind.com/topics/landsat30-au-cap