RSThinker: Faithful Remote Sensing VLM

• RSThinker is a remote-sensing vision-language model that employs a Perceptually-Grounded Geospatial Chain-of-Thought to generate both a structured analytical trace and final answer.
• It decomposes complex tasks like object counting, visual grounding, and detection into clear, verifiable steps, overcoming limitations of end-to-end mappings.
• Leveraging the large-scale Geo-CoT380k dataset and a robust two-stage alignment (SFT and GRPO), RSThinker achieves significant improvements on evidence-heavy remote sensing tasks.

Searching arXiv for the specified paper and closely related work on RSThinker and Geo-CoT. Tool unavailable in this environment, so proceeding with the arXiv records already supplied in the prompt, centering the article on "Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-LLMs" (Liu et al., 26 Sep 2025) and using only claims present in the provided data. RSThinker is a remote-sensing vision-LLM (VLM) proposed to perform faithful, verifiable, perceptually grounded reasoning over Earth observation imagery by generating both a final answer and a structured analytical trace. It is built by aligning GLM-4.1V-9B-Base so that remote sensing analysis is expressed not as a single end-to-end prediction, but as a Perceptually-Grounded Geospatial Chain-of-Thought (Geo-CoT) organized around Planning–Grounding–Synthesize. The resulting model is positioned as a pathway from “opaque perception” toward “structured, verifiable reasoning for Earth Observation” (Liu et al., 26 Sep 2025).

1. Concept and problem formulation

RSThinker is introduced against a specific diagnosis of prior remote sensing VLMs: they are typically trained end-to-end to map image-text inputs directly to answers, which leads to failures on complex analytical tasks, weak explicit reasoning, and outputs that are difficult to verify against the image evidence. The paper targets tasks such as object counting, visual grounding with spatial relations, detection in dense scenes, and compositional VQA, arguing that these tasks require decomposition, explicit evidence gathering, and falsifiable intermediate reasoning rather than one-shot recognition (Liu et al., 26 Sep 2025).

The model’s central terms are defined operationally. Faithful reasoning is tied to explicit grounding in visual evidence and alignment toward factually correct outcomes; verifiable reasoning means that major intermediate claims can be checked against the image or task ground truth; and perceptually grounded reasoning means that the model’s intermediate statements are tied to observable image content such as objects, regions, positions, relations, counts, and layouts. The paper’s framing is that remote sensing scenes are large-scale, cluttered, relation-heavy, and often less object-salient than natural images, so abstract or purely semantic rationales are inadequate (Liu et al., 26 Sep 2025).

This suggests that RSThinker should be understood less as a new backbone architecture than as a reasoning-oriented alignment strategy for Earth observation. The underlying claim is that explicit analytical structure is itself a major source of performance on evidence-heavy remote sensing tasks (Liu et al., 26 Sep 2025).

2. Geo-CoT and the Planning–Grounding–Synthesize structure

The model’s core reasoning framework is the Perceptually-Grounded Geospatial Chain-of-Thought (Geo-CoT). Geo-CoT treats remote sensing analysis as a multi-step, verifiable reasoning process rather than a monolithic prediction. Its recurring operations are: task planning and decomposition, iterative evidence gathering, perceptual grounding, intermediate inference, and final synthesis. In practical terms, Geo-CoT is intended to let the model identify likely regions, search systematically, ground claims in visible entities and spatial structure, and synthesize a conclusion only after the relevant evidence has been collected (Liu et al., 26 Sep 2025).

The canonical organization is Planning–Grounding–Synthesize. In the Planning phase, the model states how it will approach the task, for example by identifying likely target regions or defining spatial partitions such as left and right. In the Grounding phase, it makes concrete observations about the image, including object presence, region-level evidence, relative position, object signatures, and local counts. In the Synthesize phase, it aggregates those grounded observations into a final answer, often with an explicit consistency check such as ensuring no missed objects or duplicate counts. The appendix examples described in the paper include region-wise counting, anchor-object identification for grounding, and coarse-to-fine target search in detection (Liu et al., 26 Sep 2025).

The rationale is serialized during training in the format:

$$\texttt{<think> ... </think><answer> ... </answer>}$$

This does not define a symbolic reasoning engine. Instead, the reasoning process is operationalized through the structured output format and the training objectives that enforce it (Liu et al., 26 Sep 2025).

The perceptual and geospatial character of Geo-CoT is further reinforced by the model’s visual backbone adaptation. The base model uses Aimv2-Huge as vision encoder, with positional adaptation:

$$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$

where $g = (w,h)$ is a patch coordinate, $W_p, H_p$ are patch-grid width and height, $P_{orig}$ is the original pretrained position table, and $\mathcal{I}_{bicubic}$ denotes bicubic interpolation. The decoder uses 3D-RoPE for improved spatial awareness (Liu et al., 26 Sep 2025).

3. Geo-CoT380k dataset and rationale construction

RSThinker’s supervision is built around Geo-CoT380k, described as the first large-scale dataset of structured Geo-CoT rationales for remote sensing. The dataset contains 384,591 structured rationales spanning VQA, image captioning, scene classification, visual grounding, object counting, and object detection (Liu et al., 26 Sep 2025).

Its composition is explicitly enumerated in the paper. The training sources include VRSBench-train-VQA with 85,813 samples; captioning data from VRSBench-train-cap with 20,264 and FIT-RS-cap with 65,197; scene classification data from NWPU-RESISC45-train with 31,500 and AID-train with 10,000; visual grounding data from DIOR-RSVG-train with 34,744 and VRSBench-train-VG with 35,967; object counting data from DOTAv2-train with 25,769 and HRRSD-train with 24,784; and object detection data from DOTAv2-train with 25,769 and HRRSD-train with 24,784 (Liu et al., 26 Sep 2025).

The rationales are generated with a scalable annotation pipeline using GPT-4V, but not through unconstrained free-form explanation. The generator receives the image, the question, the correct answer, auxiliary information derived from ground truth, task-specific exemplars, and a shared prompt specifying the desired reasoning style. The paper describes this as retrofitting a verifiable reasoning path onto ground-truth data. Auxiliary information can include bounding boxes, referring phrases, spatial attributes, normalized coordinates, object counts, captions, and relative positions, but the prompt explicitly states that this auxiliary information is used only to scaffold rationale construction and should not appear in the final CoT text (Liu et al., 26 Sep 2025).

The prompting protocol is two-tiered. A shared base prompt standardizes task intent, input-output format, the Planning–Grounding–Synthesize structure, and the prohibition of unsupported assertions. A task-specific prompt then adds curated few-shot exemplars spanning diverse scene types and difficulty levels. The generator is instructed that the CoT should infer the answer naturally, should include thinking, deduction, and reflection, and should be divided into several segments (Liu et al., 26 Sep 2025).

This suggests that Geo-CoT380k is not simply a rationale corpus, but a dataset that attempts to impose a common cognitive architecture across heterogeneous Earth observation tasks. The paper nevertheless notes that generated rationales may inherit stylistic biases from the generative process itself, identifying this as an explicit limitation and future direction (Liu et al., 26 Sep 2025).

4. Two-stage alignment: SFT followed by GRPO

RSThinker is trained in two stages. Stage I uses supervised fine-tuning on Geo-CoT380k to instill the foundational reasoning scaffold. Stage II uses Group Reward Policy Optimization (GRPO) to refine the policy toward factually correct final outcomes while preserving the learned rationale structure (Liu et al., 26 Sep 2025).

The SFT stage uses standard autoregressive maximum likelihood on the structured rationale-plus-answer output. The objective is:

$\mathcal{L}_{\text{SFT}(\theta) = - \sum_{t=1}^{|o_i|} \log p(o_{i, t} \mid o_{i, <t}, I, Q; \theta),$

where $o_i$ is the target structured output, $I$ is the image, and $Q$ is the question. The paper’s interpretation is that SFT teaches not only answer production but also the decomposition, grounding, and synthesis workflow embedded in Geo-CoT (Liu et al., 26 Sep 2025).

The GRPO stage uses the original rationale-free instances from Geo-CoT380k plus additional data: RSVQA-HR-train with 67,228, NWPU-Captions-train with 28,350, RSICD-train with 10,921, and RSTMD-train with 4,291. For each input $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$0, the policy samples a group of $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$1 outputs $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$2, assigns raw rewards $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$3, and normalizes them within the group to obtain relative advantages (Liu et al., 26 Sep 2025).

The clipped GRPO objective is given as:

$$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$4

with

$$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$5

The reference policy $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$6 is initialized from the SFT checkpoint, and the paper states that KL regularization is crucial, with Figure 1 showing that removing KL causes catastrophic collapse of the learned reasoning format (Liu et al., 26 Sep 2025).

Rewards are task-specific. For VQA and scene classification, the reward is $$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$7 for correct, partially correct, and otherwise. For visual grounding, reward is IoU. For object counting, reward is

$$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$8

where the intent is a normalized count-error penalty. For object detection, reward is mAP@0.5. For image captioning, reward is

$$\begin{aligned} &g_{norm} = (w_{norm}, h_{norm}) = 2 \cdot \left( \frac{w + 0.5}{W_p}, \frac{h + 0.5}{H_p} \right) - 1, \ &P_{adapted}(g) = \mathcal{I}_{bicubic}(P_{orig}, g_{norm}), \end{aligned}$$9

with $g = (w,h)$0 (Liu et al., 26 Sep 2025).

The reported training settings are: base checkpoint GLM-4.1V-9B-Base, hardware 8 NVIDIA A100 GPUs, SFT for 3 epochs with batch size 32, AdamW, and learning rate $g = (w,h)$1, followed by GRPO for 970 steps (Liu et al., 26 Sep 2025).

5. Outputs, benchmarks, and empirical behavior

At inference time, RSThinker generates both a reasoning trace and a final answer, with the answer possibly taking the form of a class label, count, short phrase, caption, or localization output. The trace typically follows the Geo-CoT pattern of identifying task objective, describing scene layout, searching likely regions, grounding observations, aggregating them, and then producing the final answer (Liu et al., 26 Sep 2025).

The evaluation covers six task families: object counting, object detection, visual grounding, scene classification, image captioning, and VQA. Datasets include HRRSD, RSOD, DOTAv2-val, NWPU-VHR, VRSBench-test-VG, DIOR-RSVG, RRSIS-D, RSVG, NWPU-RESISC45-test, AID-test, WHU-RS19, SIRI-WHU, UCMerced, UCM-Captions, RSICD, RSITMD, NWPU-Captions, Sydney-Captions, VRSBench-test-cap, VRSBench-test-VQA, and RSVQA-HR-test (Liu et al., 26 Sep 2025).

The baseline set includes commercial VLMs Claude-sonnet-4, Gemini-2.0-flash, and ChatGPT-5; open-source general VLMs MiniGPT-v2 and Qwen2.5-VL; reasoning VLMs GLM-4.1V-Thinking and Kimi-VL-Thinking; and remote sensing VLMs GeoChat, VHM, SkySenseGPT, and EarthDial (Liu et al., 26 Sep 2025).

The strongest gains are reported on evidence-heavy tasks. On visual grounding, RSThinker reaches 90.4 / 77.2 / 80.79 on VRSBench-VG, 93.1 / 90.2 / 89.02 on DIOR-RSVG, 94.0 / 90.5 / 89.59 on RRSIS-D, and 64.0 / 54.5 / 59.74 on RSVG for $g = (w,h)$2 (Liu et al., 26 Sep 2025). On object counting, it reports 43.93 Acc / 2.728 MAE on DOTAv2-val, 85.26 / 0.242 on HRRSD, and 80.0 / 0.465 on NWPU-VHR (Liu et al., 26 Sep 2025). On object detection, the appendix numbers are 79.56 / 77.06 / 69.88 on DOTA and 98.54 / 97.99 / 94.59 on HRRSD for $g = (w,h)$3 (Liu et al., 26 Sep 2025).

Scene classification results are also strong: 96.89 on RESISC45, 98.17 on AID, 99.74 on RS19, 77.67 on SIRI, and 92.14 on UCM. In VQA, the reported scores on VRSBench-VQA include 82.84 for Category, 92.36 for Existence, 68.97 for Position, 56.67 for Quantity, 73.24 for Scene, 64.33 for Color, and 92.87 for Image; on RSVQA-HR, 66.95 for Presence and 78.98 for Comparison are listed (Liu et al., 26 Sep 2025).

The ablation study isolates the role of process supervision and RL refinement. Starting from the base model, performance rises sharply with SFT and rises further when CoT-style supervision is included. The reported numbers are:

Model	VG (mIoU)	OC (MAE)	Det ([email protected])	IC (BLEU-4)	SC (Acc)	VQA (Acc)
Base	56.26	10.81	3.56	10.99	69.78	8.16
+ SFT w/o CoT	81.80	3.272	49.36	31.14	93.33	63.57
+ SFT w/ CoT	87.70	2.932	74.03	33.31	96.67	74.20
+ SFT w/o CoT + GRPO	86.47	4.510	56.77	30.87	97.56	74.09
+ SFT w/ CoT + GRPO	89.02	2.728	77.06	33.96	96.89	77.24

These results support the paper’s thesis that structure first, policy refinement second is more effective than reward optimization without a reasoning scaffold (Liu et al., 26 Sep 2025).

6. Interpretation, significance, and limitations

RSThinker’s significance lies in the claim that, for remote sensing, reasoning format and supervision strategy are major determinants of performance. The paper repeatedly contrasts Geo-CoT with generic chain-of-thought and with end-to-end VLM pipelines, arguing that remote sensing tasks often require region-wise search, relational grounding, and explicit evidence aggregation. The especially large gains on visual grounding, detection, counting, and complex VQA are consistent with that argument (Liu et al., 26 Sep 2025).

At the same time, the paper does not present a formal proof of faithfulness in the strict causal sense. The support for faithfulness comes from three sources: the rationale-construction protocol grounded in known answers and auxiliary annotations, the structured Geo-CoT format, and qualitative examples showing region-wise or anchor-based reasoning. This suggests that the work is best read as a strong step toward faithful remote sensing reasoning rather than as a complete solution to formal faithfulness verification (Liu et al., 26 Sep 2025).

The clearest explicit limitation stated in the paper is that generated rationales may inherit stylistic biases from the generative process itself. A further implication, stated in the technical discussion, is that GRPO optimizes outcome-level correctness more directly than step-level faithfulness, with step quality preserved through the SFT scaffold and KL anchoring rather than through explicit step verification (Liu et al., 26 Sep 2025).

The work nevertheless establishes a concrete research program for Earth observation reasoning: use a domain-shaped analytical trace, align a strong VLM to emit it, and evaluate not only end-task performance but the model’s ability to externalize a checkable evidential path. The paper also states that the Geo-CoT380k dataset and RSThinker model will be publicly released, with code and model pages at https://github.com/minglangL/RSThinker and https://huggingface.co/minglanga/RSThinker (Liu et al., 26 Sep 2025).