Perceptually-Grounded Geo-CoT

• Perceptually-Grounded Geo-CoT is a framework that transforms geospatial analysis into a verifiable, multi-step reasoning process anchored in spatial evidence.
• It decomposes complex visual tasks into planning, iterative grounding with observable cues, and synthesis, ensuring each inference can be audited against spatial data.
• Empirical results demonstrate significant improvements in remote sensing tasks by reducing semantic ambiguity and enhancing accuracy with structured chain-of-thought methods.

Searching arXiv for the specified paper and closely related Geo-CoT work to ground the article in current literature. Perceptually-Grounded Geospatial Chain-of-Thought (Geo-CoT) denotes a class of multimodal reasoning frameworks in which geospatial analysis is externalized as a verifiable, multi-step process whose intermediate claims are tied to observable spatial evidence rather than left implicit in an end-to-end prediction. In remote sensing, the term is explicitly formalized as a framework that models analysis as a verifiable, multi-step process and requires that each step of analytical reasoning be explicitly and verifiably grounded in the visual input (Liu et al., 26 Sep 2025). Closely related formulations include the Geospatial Reasoning Chain-of-Thought (GR-CoT) for open-vocabulary semantic segmentation (Zhou et al., 9 Feb 2026), the Geographical Chain-of-Thought (GeoCoT) framework for geolocation (Song et al., 19 Feb 2025), and Grounding CoT formats in which intermediate steps are expressed through spatial coordinates, points, lines, or regions (Du et al., 27 Nov 2025).

1. Conceptual foundations

Geo-CoT emerged from dissatisfaction with implicit or purely appearance-based inference. In remote sensing open-vocabulary semantic segmentation, existing methods are described as predominantly relying on the passive mapping of visual features and textual embeddings, an appearance-based paradigm that lacks geospatial contextual awareness and leads to semantic ambiguity when land-cover classes have similar spectral features but distinct semantic attributes (Zhou et al., 9 Feb 2026). In remote sensing VLMs more broadly, the underlying criticism is that end-to-end training bypasses crucial reasoning steps and produces outputs that are difficult to verify (Liu et al., 26 Sep 2025).

The emphasis on grounding also has a deeper semantic background. Earlier work on geospatial semantics argued that ontology-based approaches suffer from the symbol grounding problem, are decontextualised, and decontextualise experience; it proposed an enactive view in which meaning is considered an emergent feature arising context-dependently in action (0901.4224). This suggests that Geo-CoT is not only a prompting or alignment strategy, but also a response to a longstanding concern in geospatial reasoning: symbolic or latent decisions alone are insufficient when semantics must be reconstructed from context, perception, and action.

A recurring motivation across the literature is hallucination control. GeoSolver frames the central bottleneck as the lack of visual faithfulness in intermediate steps and targets spatial hallucination through token-level process supervision (Sun et al., 10 Mar 2026). PointCoT makes an analogous argument for 3D point cloud understanding, stating that implicit mapping bypasses intermediate logical steps and consequently suffers from geometric hallucinations (Zhang et al., 27 Feb 2026). Across these settings, Geo-CoT is defined less by any single architecture than by the requirement that intermediate reasoning remain auditable against the underlying spatial evidence.

2. Analytical structure and grounding mechanisms

In its most explicit remote-sensing formulation, Geo-CoT decomposes reasoning into three phases: task planning, iterative evidence gathering (“Grounding”), and synthesis (Liu et al., 26 Sep 2025). The planning stage decomposes a complex visual analysis into sequential subgoals; the grounding stage requires intermediate conclusions to be justified with observable spatial evidence such as bounding boxes, coordinates, or region attributes; the synthesis stage integrates these grounded findings into a final answer. The resulting trace is designed to be auditable, so that each inference step can be checked against specified parts of the visual data.

GR-CoT provides a concrete realization of this logic for open-vocabulary segmentation through a dual-stream architecture. Its offline knowledge distillation stream constructs a Category Interpretation Standard,

$$S = \{ (c_{i}, D_{i}) \mid c_{i} \in C, D_{i} = \text{MLLMs}(c_{i}, \text{priors}) \},$$

while the online instance reasoning stream proceeds through macro-scenario anchoring, visual feature decoupling, and knowledge-driven decision synthesis (Zhou et al., 9 Feb 2026). These stages are formalized as

$G = f_{\text{anchor}(I),$

$A = f_{\text{decouple}(I, G),$

and

$V_{\text{adaptive} = \left\{ c_{i} \mid \text{verify}(c_{i}, G, A, S) = \text{true} \right\},$

so that pixel-to-text alignment is restricted to an image-adaptive vocabulary:

$M(x, y) = \arg\max_{c_j \in V_{\text{adaptive}} \langle \mathbf{F}_v(x, y), \mathbf{E}_t(c_j) \rangle.$

Within the paper’s interpretation, this filtering reduces semantically illogical assignments by constraining the candidate label space to classes that are compatible with global context, image attributes, and distilled category standards.

A more general VLM formulation appears in Grounding CoT. There, intermediate reasoning is represented with explicit spatial reference, such as coordinate trajectories or point/line/region descriptions, with each grounded step sampled as

$$(r_t^{(g)}, g_t) \sim \pi_{\theta}(\cdot \mid Q, I, R_{t-1}^{\text{grd}})$$

and grounding coordinates defined by

$$C_k = \begin{cases} (x, y) & G_k=\text{point} \ \{(x_1, y_1), (x_2, y_2)\} & G_k=\text{line} \ \{(x_i, y_i)\}_{i=1}^n & G_k=\text{region}. \end{cases}$$

This formulation makes the grounding token itself part of the reasoning trace rather than a post hoc explanation (Du et al., 27 Nov 2025).

3. Data regimes and supervision resources

The maturation of Geo-CoT has been tied to large-scale datasets that encode intermediate reasoning rather than only final labels. The principal resources differ in supervision granularity, modality, and task scope.

Resource	Scale	Role
Geo-CoT380k	384,591 samples	Structured, step-by-step, visually grounded rationales across VQA, image captioning, scene classification, visual grounding, object counting, and detection
Geo-PRM-2M	~2 million token-level process supervision samples	Training a token-level process reward model for faithfulness verification
GeoChain	1.46 million street-level images; over 30 million Q&A pairs	21-step multimodal CoT benchmark for geographic reasoning
GeoComp	25 million entries of metadata; 3 million geo-tagged locations; 740K users	Large-scale geolocation dataset with human gameplay records

Geo-CoT380k is described as the first large-scale dataset of structured Geo-CoT rationales and is central to the RSThinker alignment pipeline (Liu et al., 26 Sep 2025). Its task distribution includes 85,813 VQA samples, 85,461 image captioning samples, 41,500 scene classification samples, 70,711 visual grounding samples, and 50,553 samples each for object counting and object detection. The annotation pipeline uses GPT-4V under strict, multi-tiered prompts, with auxiliary ground-truth information used as scaffolds for reconstructing natural-language, step-by-step, visually referable reasoning traces.

Geo-PRM-2M serves a different purpose. It provides token-level process supervision synthesized via entropy-guided Monte Carlo Tree Search and targeted visual hallucination injection, and is used to train GeoPRM, a process reward model that detects both logical and perceptual flaws in reasoning (Sun et al., 10 Mar 2026). This shift from sequence-level to token-level supervision is significant because it enables error localization at the level of individual reasoning steps.

GeoChain is a diagnostic benchmark rather than an alignment corpus. Each of its 1.46 million Mapillary street-level images is paired with a 21-step chain-of-thought question sequence, yielding over 30 million question-answer pairs distributed across visual, spatial, cultural, and precise geolocation categories (Yerramilli et al., 1 Jun 2025). Images are enriched with semantic segmentation for 150 classes and a visual locatability score, allowing researchers to evaluate whether models genuinely ground their geographic reasoning in perceptual cues.

GeoComp couples scale with human reasoning behavior. It comprises 25 million entries of metadata and 3 million geo-tagged locations collected from 740K users over two years, with each location annotated thousands to tens of thousands of times by human users (Song et al., 19 Feb 2025). In that framework, GeoEval assesses reasoning quality through completeness of feature extraction, accuracy of feature extraction, accuracy of reasoning/cue correspondence, logical coherence, and hallucination categories.

4. Alignment, reinforcement learning, and test-time verification

A characteristic pattern in the Geo-CoT literature is two-stage or multi-stage alignment. RSThinker first undergoes supervised fine-tuning on Geo-CoT380k to learn a structured output of the form > ... <answer> ... </answer>, optimized with the standard autoregressive objective

$\mathcal{L}_{\text{SFT}(\theta) = - \sum_{t=1}^{|o_i|} \log p(o_{i, t} \mid o_{i,<t}, I, Q; \theta),$

and is then refined with Group Reward Policy Optimization (GRPO), using task-aligned rewards such as IoU, MAE, or mAP to correct sequence-level failures not addressed by token-level likelihood training (Liu et al., 26 Sep 2025).

Geo-R1 applies an analogous scaffolding-and-elevating strategy in cross-view geospatial reasoning. Its scaffolding stage uses supervised fine-tuning on synthetic CoT exemplars to instill a four-stage geospatial thinking paradigm: Visual Cue Identification, Knowledge Association, Evidence Corroboration, and Conclusion Formulation. The elevating stage then applies GRPO-based reinforcement learning to a cross-view pairing proxy task with a reward that combines accuracy, format, length, and repetition terms:

$$r = \lambda_{\text{acc}} r_{\text{acc}} + \lambda_{\text{fmt}} r_{\text{fmt}} + \lambda_{\text{len}} r_{\text{len}} + \lambda_{\text{rep}} r_{\text{rep}}.$$

The stated goal is to bridge structured reasoning and perceptual generalization without costly human reasoning annotations (Xu et al., 29 Sep 2025).

GeoSolver extends this trend from outcome supervision toward process supervision. GeoPRM is trained with a token-level binary classification objective,

$$\mathcal{L}(\theta) = - \frac{1}{L} \sum_{t=1}^L m_t \left[ y_t \log \hat{y}_t + (1-y_t)\log(1-\hat{y}_t) \right],$$

and the resulting verifier is integrated into Process-Aware Tree-GRPO, which combines tree-structured exploration with faithfulness-weighted credit assignment (Sun et al., 10 Mar 2026). The paper explicitly argues that outcome-signal-only reinforcement learning can reward entire trajectories even when intermediate steps are perceptually unfaithful, and that direct aggregation of scalar process scores introduces a bias against longer legitimate reasoning.

The VQA literature on satellite imagery introduces a related but distinct alignment method. There, CoT supervision is followed by Direct Preference Optimization (DPO), with preference pairs formed from correct and incorrect rationale-answer chains. The DPO loss is given as

$G = f_{\text{anchor}(I),$0

Here the objective is not only correctness but also interpretability, robustness, and preference alignment of intermediate rationales (Shanker et al., 14 Nov 2025).

5. Empirical performance and diagnostic findings

The strongest empirical claims for Geo-CoT in remote sensing come from RSThinker. On DIOR-RSVG visual grounding it reports an mIoU of 89.02; on HRRSD object counting it reports 85.26 accuracy and 0.242 MAE; on DOTAv2 object detection it reports 77.06 [email protected]; on NWPU-RESISC45 scene classification it reports 96.89; on RSICD image captioning it reports a CIDEr score of 99.83; and on VRSBench-Existence VQA it reports 92.36 (Liu et al., 26 Sep 2025). The paper’s central interpretation is that the combination of SFT with structured CoT and GRPO with task-aligned rewards is required for the highest and most stable performance.

GR-CoT reports improvements on two remote-sensing segmentation benchmarks. On LoveDA, CAT-Seg obtains 34.23 mIoU and 51.75 OA, RSKT-Seg obtains 40.71 mIoU and 57.35 OA, and GR-CoT obtains 41.39 mIoU and 59.93 OA; on GID5, CAT-Seg obtains 40.23 mIoU and 59.12 OA, RSKT-Seg obtains 42.20 mIoU and 61.40 OA, and GR-CoT obtains 45.34 mIoU and 63.34 OA (Zhou et al., 9 Feb 2026). Its ablation reports 11.19% category accuracy for a vanilla OVSeg baseline, 45.12% with MLLM expert knowledge, and 64.85% with the full GR-CoT configuration. These results are used to support the claim that explicit geospatial reasoning reduces semantic ambiguity in visually confusable land-cover categories.

In geolocation, GeoCoT reports a city-level accuracy of 0.118, recall of 0.089, and F1 of 0.086 on GeoComp, compared with 0.094, 0.052, and 0.042 for GPT-4o (CoT), while its prediction-within-1km score is 0.073 (Song et al., 19 Feb 2025). GeoEval further reports the highest scores across completeness of feature extraction, accuracy of feature extraction, accuracy of reasoning/cue correspondence, and logical coherence, together with reduced object, fact, and attribution hallucination.

For satellite-image VQA, CoT supervision improves accuracy by 34.9% over zero-shot baseline and by 18.19% versus direct-only SFT, while DPO yields a further 5.67% improvement above SFT-CoT alone; the best model reaches 82.77% overall accuracy on RSVQA when all weights are unfrozen (Shanker et al., 14 Nov 2025). This result is presented as evidence that intermediate rationales are useful not only for interpretability but also for final answer quality.

Diagnostic evaluation nevertheless shows that the problem is not solved. GeoChain benchmarks contemporary MLLMs on a 2,088-image subset and finds that models frequently exhibit weaknesses in visual grounding, display erratic reasoning, and struggle to achieve accurate localization, especially as reasoning complexity escalates (Yerramilli et al., 1 Jun 2025). The benchmark therefore functions as evidence that good object-level perception does not automatically produce reliable multi-step geographic reasoning.

6. Variants, misconceptions, and broader generalization

A central misconception in the wider CoT literature is that longer reasoning traces are intrinsically better. A controlled study of vision-centric reasoning compares Language CoT, Grounding CoT, and Visual CoT, and reports that visual and longer CoT mainly accelerate convergence but do not lift the final performance ceiling; concise CoT containing only essential grounding steps outperforms longer traces; and CoT retaining only the minimal grounding generalizes best across different maze sizes (Du et al., 27 Nov 2025). In that work, Grounding CoT is as effective as explicit visual manipulation for generalization, and the authors characterize the result as a “short is long” effect. A plausible implication is that perceptual grounding and reasoning faithfulness are not equivalent to verbosity.

Related spatial-reasoning systems extend Geo-CoT principles beyond remote sensing imagery. EagleVision formulates spatial CoT as BEV-grounded pose querying, in which an agent iteratively predicts poses on a BEV plane, retrieves the nearest real frames, and is trained purely by reinforcement learning with a spatial grounding reward; on VSI-Bench it reports a 63.5 average score and ranks first among open-source vision-LLMs (Wan et al., 17 Dec 2025). Spatial-Agent instead formalizes geo-analytical question answering as a concept transformation problem represented by GeoFlow Graphs, directed acyclic graphs whose nodes correspond to spatial concepts and whose edges represent transformations (Bao et al., 23 Jan 2026). These systems indicate that explicit evidence chains can be constructed not only as natural-language rationales but also as executable workflows or tool-mediated spatial hypotheses.

Three-dimensional reasoning offers further corroboration. PointCoT introduces a “Look, Think, then Answer” paradigm for point clouds and reports that explicit CoT reasoning cuts geometric hallucinations from 25.4% in a black-box model to 5.1%, while its overall score reaches 78.5 with 82.3 on geometry, 76.4 on spatial reasoning, and 75.1 on functionality (Zhang et al., 27 Feb 2026). CoT3DRef similarly formulates 3D visual grounding as a sequence-to-sequence task that first predicts a chain of anchors and then the final target, and reports that on Sr3D, when trained only on 10% of the data, it matches the state-of-the-art performance trained on the entire dataset (Abdelrahman et al., 2023). Although these papers do not use the remote-sensing Geo-CoT label directly, they reinforce the same design principle: intermediate, inspectable anchors can improve both interpretability and data efficiency.

At the language level, GeoLM grounds textual geo-entities in geospatial context by linking natural-language mentions to OpenStreetMap-derived pseudo-sentences and spatial coordinate embeddings, improving toponym recognition, toponym linking, relation extraction, and geo-entity typing (Li et al., 2023). This suggests that perceptual grounding in Geo-CoT is part of a broader movement toward geospatially grounded representations in which spatial relations, coordinates, and context are treated as first-class components of reasoning rather than auxiliary metadata.