GeoZero-Instruct for Geospatial Adaptation

• GeoZero-Instruct is a non-CoT supervised dataset that grounds geospatial knowledge for remote sensing tasks through instruction–answer pairs.
• It is constructed from GeoZero-Raw by filtering difficult samples and removing image overlaps with GeoZero-Hard to ensure foundational task competence.
• Its role in a two-stage SFT–RL pipeline emphasizes initial geospatial adaptation over explicit reasoning, with RL later activating deep reasoning capabilities.

GeoZero-Instruct is the non-CoT supervised fine-tuning dataset in the GeoZero framework, designed to adapt a general-purpose multimodal LLM to geospatial and remote-sensing vision-language tasks without chain-of-thought supervision. Within GeoZero, it provides image–instruction–answer pairs that supply preliminary geospatial knowledge and instruction-following behavior, while a separate hard subset, GeoZero-Hard, is reserved for reinforcement learning intended to stimulate explicit reasoning. In this design, GeoZero-Instruct is not a reasoning-trace corpus but a domain-grounding corpus whose function is to establish foundational task competence before answer-anchored policy optimization (Wang et al., 27 Nov 2025).

1. Definition and functional position

GeoZero-Instruct is defined as the non-CoT instruction-tuning subset derived from the larger aggregated dataset GeoZero-Raw. It is used in the supervised fine-tuning stage of GeoZero, contains question–answer or instruction–response pairs only, and is intended to help the model acquire preliminary geospatial knowledge. The paper is explicit that both GeoZero-Instruct and GeoZero-Hard contain only question–answer pairs without any CoT data.

Its role is therefore distinct from datasets that teach a model an explicit “think-then-answer” pattern. GeoZero-Instruct teaches task formats, answer styles, and foundational domain knowledge for remote sensing vision-language tasks; GeoZero-Hard is then used to induce deep reasoning during the subsequent reinforcement learning stage. This separation is central to the GeoZero claim that geospatial reasoning can emerge without predefined CoT annotations.

Aspect	GeoZero-Instruct	GeoZero-Hard
Training stage	SFT	RL
Goal	learn geospatial knowledge / instruction following	encourage genuine reasoning on difficult samples
Sample difficulty	remaining non-hard data	top difficult samples selected by filtering
Size	~610K	~20K
CoT supervision	none	none
Image overlap with other set	excludes images used in GeoZero-Hard	selected first

The paper does not report an internal train/validation/test split for GeoZero-Instruct itself. It is used as the SFT set, while evaluation is conducted on the corresponding test sets of the public datasets that contribute training samples to GeoZero-Raw.

2. Construction from GeoZero-Raw

GeoZero-Instruct is produced from GeoZero-Raw, an aggregated remote-sensing instruction corpus containing 754,749 samples. GeoZero-Raw is organized around four representative remote sensing vision-language tasks—Scene Classification, Visual Grounding, Visual Question Answering, and Image Captioning—plus an “Others” category containing counting, honesty evaluation, and multi-turn dialogue samples without explicit task identifiers.

Task group in GeoZero-Raw	Subtotal
Scene Classification	110,649
Visual Grounding	126,620
Visual Question Answering	243,041
Image Captioning	173,639
Others	100,800

The source datasets are public remote sensing datasets that are reformatted into instruction-following examples. Scene classification samples come from VHM-Instruct, AID-train, RESISC45-train, NASC-TG2-train, WHU-RS19-train, EuroSAT-train, and fMoW-RGB-train. Visual grounding samples come from VHM-Instruct, RSVG-train, DIOR-RSVG-train, and VRS-train. Visual question answering samples come from VHM-Instruct, RSVQA-HR-train, RSVQA-LR-train, and VRS-train. Image captioning samples come from SkyEye968k and VRS-train.

GeoZero-Instruct is obtained through a three-step construction process. First, a data filtering model is trained on GeoZero-Raw. Second, that model is used to identify difficult samples and construct GeoZero-Hard. Third, all samples sharing the same image with GeoZero-Hard are removed from GeoZero-Raw; the remainder becomes GeoZero-Instruct.

The data filtering model is Qwen2.5-VL-7B-Instruct, trained on GeoZero-Raw for 3 epochs with global batch size 32 on 8 NVIDIA A100 GPUs, learning rate $1\times10^{-4}$, weight decay 0.1, frozen vision encoder, LoRA rank 8 with $\alpha=32$, bfloat16, FlashAttention, and DeepSpeed. Hard-sample selection uses a coarse-to-fine two-stage filtering strategy. In the first stage, all Scene Classification, Visual Grounding, Visual Question Answering, and Image Captioning samples are evaluated once, and incorrectly predicted samples are retained as candidate hard samples. Correctness is defined as normalized exact label match for Scene Classification, IoU $\ge 0.5$ for Visual Grounding, normalized exact match or substring containment for Visual Question Answering, and word-level $F_1 \ge 0.6$ for Image Captioning. In the second stage, each candidate is reevaluated 3 times with different random seeds; per-sample accuracy is the mean correctness over 3 trials, difficulty is $1-Acc$, the candidates are ranked by difficulty, the hardest samples are selected, balanced sampling ensures an equal number from each of the four primary tasks, and deduplication removes redundant samples.

This construction makes GeoZero-Instruct a residual corpus by design: it is not merely a random training split, but the complement of a difficulty-filtered RL subset with image-level separation.

3. Supervision format and instruction design

GeoZero-Instruct provides image plus instruction plus answer. It does not provide explicit reasoning traces, chain-of-thought annotations, manually curated rationales, or cold-start “thinking–answering” demonstrations. The supervision is therefore direct task supervision rather than rationale supervision.

The dataset inherits the breadth of GeoZero-Raw except for samples excluded because their images overlap with GeoZero-Hard. It includes supervision for scene classification, visual grounding, visual question answering, image captioning, and likely some auxiliary instruction-following behavior from the “Others” category. The paper highlights diversity across resolutions, sensor modalities, and scene categories, with the stated purpose of improving generalization.

A key construction step is instruction formatting. Each example is converted into an instruction-following format, and task descriptors are prepended: [cls], [grounding], [vqa], and [caption]. Task-specific textual hints are added to help the model understand output format, but these hints are included with 50% probability to prevent over-reliance. For each task, the authors designed more than 20 variations of textual hints, and one is randomly selected. The supplementary material gives a representative grounding prompt: “Please provide the location of the object in the image: {original grounding text}.”

Output forms are task-specific: scene classification uses a textual class label, visual grounding uses a bounding box location, visual question answering uses answer text, and image captioning uses caption text. The paper does not fully specify the serialized label syntax for every task in the main text, but it states that formatting details and converted sample examples appear in supplementary figures.

GeoZero-Raw may contain distinct entries that share the same underlying imagery across source datasets. The paper nevertheless treats these as distinct entries at the raw stage because textual instructions are regenerated according to the formatting rules. This makes the instruction layer itself part of the dataset construction, not merely a wrapper around preexisting annotations.

4. Role in the GeoZero training pipeline

GeoZero uses a two-stage SFT–RL pipeline: supervised fine-tuning on GeoZero-Instruct, followed by reinforcement learning on GeoZero-Hard using Answer-Anchored Group Relative Policy Optimization. Optional downstream RFT is then used for some reported results. The functional logic is explicit: the SFT stage equips the model with a preliminary understanding of geospatial scenes, while the RL stage is responsible for activating reasoning behavior.

The SFT stage is described conceptually as standard supervised fine-tuning on instruction–response pairs; the supplied paper text does not provide a standalone SFT loss equation. What it does provide is the downstream RL formulation that GeoZero-Instruct helps motivate. The overall A$^2$GRPO reward is

$r = r_a + \lambda \cdot r_t,$

and the Answer-Modulated Thinking Reward is

$$r_t = \mathcal{G}(r_a) \cdot r_a \cdot s_t, \qquad \mathcal{G}(r_a) = \frac{1}{1 + e^{-k(r_a - \tau)}}.$$

This design ties thinking reward to answer quality rather than rewarding unconstrained reasoning.

The interaction with GeoZero-Instruct is especially important at the objective level. Because the SFT reference model trained on GeoZero-Instruct does not naturally emit explicit reasoning, GeoZero applies KL alignment only to answer tokens through the answer-token mask $m_{i,t}\in\{0,1\}$. The paper states that $m_{i,t}=1$ if token $\alpha=32$0 is inside the answer tags and $\alpha=32$1 otherwise. This is why the method is “answer-anchored”: it regularizes only the answer portion against the SFT reference, while allowing reasoning tokens to explore more freely.

The paper also reports that the original DeepSeek-R1-style format, > reasoning <answer> answer </answer>, did not work well because the SFT model was not accustomed to starting with <think>. GeoZero therefore simplifies the RL prompt format by retaining only "<answer> ... </answer>" markers and removing explicit think-tag constraints. This suggests that the response distribution learned from GeoZero-Instruct materially shapes the feasible RL interface.

5. Empirical interpretation

The most direct evidence for the contribution of GeoZero-Instruct comes from the training-paradigm ablation. With SFT only on GeoZero-Instruct, the model reaches UCM 94.52% with no reasoning and RSVG 47.76% with no reasoning. By contrast, RL only yields UCM 76.90% with reasoning and RSVG 0.16% with reasoning. GeoZero without RFT, which combines GeoZero-Instruct SFT and GeoZero-Hard RL, reports UCM 93.81% with reasoning and RSVG 37.16% with reasoning, while GeoZero with RFT reaches UCM 95.48% with reasoning and RSVG 50.04% with reasoning (Wang et al., 27 Nov 2025).

These results support a specific interpretation. GeoZero-Instruct produces the largest jump in basic task competence, particularly for scene classification and visual grounding, but it does not by itself activate explicit reasoning. Supplementary reasoning-activation analysis strengthens this point: after SFT only on GeoZero-Instruct, the thinking activation rate is 0.00% on both UCM and RSVG, even though task accuracy improves substantially. The authors interpret this as SFT making the model memorize efficient question–answer patterns rather than think explicitly.

The complementary role of GeoZero-Hard further clarifies the status of GeoZero-Instruct. When random RL data are used instead of GeoZero-Hard, GeoZero without RFT reports UCM 94.05% with no reasoning and RSVG 18.34% with no reasoning; using GeoZero-Hard instead yields UCM 93.81% with reasoning and RSVG 37.16% with reasoning. This indicates that difficult RL data, not GeoZero-Instruct alone, are what push the model toward overt reasoning behavior.

This suggests that GeoZero-Instruct is best understood as a foundational geospatial adaptation set rather than as a reasoning-induction set. Its contribution is necessary for stable RL, but it is not sufficient for reasoning-from-scratch behavior.

6. Limitations, scope, and naming ambiguities

Several caveats follow directly from the dataset design. GeoZero-Instruct does not itself trigger reasoning; its supervision contains no CoT, no rationale supervision, and no manually refined reasoning paths. Coverage depends on existing public remote sensing datasets, so the corpus inherits the task distribution, geographic biases, and modality biases of those datasets. The authors also note that source datasets may share imagery, which required explicit removal of all samples sharing the same image with GeoZero-Hard. Task imbalance was substantial enough that VHM-Instruct was supplemented with additional samples from existing datasets. The datasheet says “No” to errors, noise, and redundancies, but the method section still discusses overlap, deduplication, and filtering, indicating that practical curation remained necessary (Wang et al., 27 Nov 2025).

GeoZero-Instruct is also easy to misidentify because several contemporaneous systems use superficially similar names. “GeometryZero: Improving Geometry Solving for LLM with Group Contrastive Policy Optimization” does not mention “GeoZero-Instruct”; its official family name is GeometryZero, built from Qwen2.5-Instruct backbones, and the paper explicitly states that “GeoZero-Instruct” is not an official model name there (Wang et al., 8 Jun 2025). “G-Zero: Self-Play for Open-Ended Generation from Zero Data” likewise does not mention GeoZero-Instruct and should be read as a verifier-free self-evolution framework rather than a geospatial dataset (Huang et al., 11 May 2026). “RemoteZero: Geospatial Reasoning with Zero Human Annotations” does not mention GeoZero-Instruct either, but it is conceptually adjacent because it studies query-conditioned geospatial localization with minimal external supervision and no box annotations during policy optimization (Yao et al., 6 May 2026).

The practical distribution plan is more straightforward. The GeoZero paper states that code, data, and models will be publicly available at the GeoZero GitHub, that distribution will occur via GitHub and HuggingFace, and that the license is CC BY-NC-SA 4.0 (Wang et al., 27 Nov 2025). Within that release structure, GeoZero-Instruct is the component that teaches the model what to know and how to answer in geospatial tasks, while leaving the emergence of explicit reasoning to the later RL stage.