GeoVista GeoBench: Geospatial AI Benchmarks

• GeoVista GeoBench is a benchmark framework that evaluates complex agentic geolocalization tasks and multi-domain Earth observation segmentation with globally distributed, high-resolution images.
• It offers two distinct evaluation regimes—one emphasizing web-augmented, tool-based visual reasoning and another focused on semantic segmentation for diverse remote sensing applications.
• The framework drives methodological innovations by standardizing evaluation protocols and inspiring adaptive model architectures to address class imbalance and multi-modal challenges in geospatial AI.

GeoVista GeoBench encompasses two benchmark frameworks used in the evaluation of geospatial artificial intelligence: (1) the GeoBench/GeoVista benchmark for geolocalization-focused, web-augmented agentic reasoning, and (2) the GeoBench multi-task segmentation benchmark for geospatial foundation model (GeoFM) adaptation and evaluation. The term "GeoBench" has thus been independently adopted as the name of benchmarks in both agentic geolocalization reasoning and multi-domain Earth observation segmentation, each serving a crucial role in the rigorous, reproducible evaluation of geospatial models.

1. Benchmark Definitions and Scope

Agentic Geolocalization: GeoVista GeoBench

GeoBench, as described in connection with GeoVista, is a curated benchmark for evaluating models on the real-world geolocalization task. The dataset contains 1,142 high-resolution images: 512 photographs, 512 planar panoramas, and 108 satellite image crops. These images span 66 countries and 108 cities, with explicit constraints on image quality (e.g., photos ≥1 MP, panoramas 4096×2048 pixels, satellite scenes ≈2000×2000 pixels). The benchmark purposefully excludes "non-localizable" and "obvious landmark" scenes, focusing on scenes that challenge high-level agentic visual reasoning and web-augmented inference. This GeoBench is designed to evaluate vision-language agents on complex reasoning involving tool use (image zoom, web search) (Wang et al., 19 Nov 2025).

Multi-Task Earth Observation Segmentation: GeoBench

A distinct “GeoBench” benchmark introduced by Lacoste et al. (2023) and used as the core performance testbed in advanced adaptation frameworks (e.g., DARN) consists of six diverse remote-sensing semantic segmentation tasks. These span applications such as livestock monitoring, solar panel detection, forest canopy delineation, multi-class land-cover mapping, plantation mapping, and crop-type classification, each with its own spectral and spatial characteristics. All are semantic segmentation rather than classification or detection, with substantial class imbalance and varying input modalities (e.g., Sentinel-2 L2A multispectral, RGB). This GeoBench is used to drive the development and fair comparison of foundation model adaptation techniques for geospatial imagery (Yadav et al., 6 Nov 2025).

2. GeoVista GeoBench: Agentic Geolocalization Task Structure

Dataset Composition

• Photographs: 512 standard, high-resolution photos.
• Panoramas: 512 stitched planar 360° street-view images.
• Satellite Views: 108 high-resolution crops.

Images are globally distributed across six continents, designed to demand geographic reasoning rather than trivial landmark recognition or pixel-level comparison.

Evaluation Protocols

Two evaluation regimes are employed:

• Hierarchical Classification: Country, province/state, and city-level accuracy metrics, as well as per-modality city accuracy.
• Continuous Geolocation Error: Geocoded predictions are compared to ground truth using the haversine formula

$$d = 2 R_e \arcsin\left(\sqrt{v}\right),\quad v = \sin^2\left(\frac{\phi_2-\phi_1}{2}\right) + \cos\phi_1 \cos\phi_2 \sin^2\left(\frac{\lambda_2 - \lambda_1}{2}\right)$$

where $(\phi_1, \lambda_1)$ and $(\phi_2, \lambda_2)$ are the predicted and ground truth locations, respectively.

Reported metrics include:

• % of predictions with $d<3$ km.
• Median distance $d$.

Baseline and SOTA Results

GeoVista-7B, a web-augmented, tool-using agent trained on this benchmark, achieves 92.64% country, 79.60% state/province, and 72.68% city accuracy—comparable to proprietary models such as Gemini-2.5-pro and GPT-5. Distance-based assessments confirm strong localization ability (52.83% within 3 km; median distance 2.35 km) (Wang et al., 19 Nov 2025).

3. GeoBench: Multi-Task Segmentation Benchmark for Geospatial Foundation Models

Dataset Summary

The segmentation-focused GeoBench dataset contains six tasks:

Task	Classes	Train	Val	Test	Modality	Application Domain
m-nz-cattle	2	2.4K	800	800	S2L2A	Livestock Monitoring
m-pv4ger-seg	2	1.2K	400	400	S2L2A	Solar Panels
m-NeonTree	2	270	94	93	RGB	Forest Canopy
m-chesapeake	7	4.8K	1.2K	1.2K	S2L2A	Land Cover Mapping
m-cashew-plant	2	1.8K	600	600	S2L2A	Plantation Mapping
m-SA-crop-type	10	3.0K	1.0K	1.0K	S2L2A	Crop Classification

Each reflects distinct challenges in spectral modality, spatial scale, object geometry, and class distribution (Yadav et al., 6 Nov 2025).

Evaluation Metric and Protocol

• Primary metric: mean Intersection-over-Union (mIoU)

$$\mathrm{mIoU} = \frac{1}{C}\sum_{i=1}^{C} \frac{\mathrm{TP}_i}{\mathrm{TP}_i + \mathrm{FP}_i + \mathrm{FN}_i}$$

Overall GeoBench performance is reported as the mean of the six task-level mIoUs.

• Baselines: U-Net, DeepLabV3+, UPerNet, and strong foundation models (e.g., TerraMind-L, Prithvi-EO-V2).
• Challenges: pronounced class imbalance (notably for rare crop types), heterogeneity in noise and resolution, and varying object scales.

4. Methodological Innovations and Benchmark Role

GeoVista: Multiturn Reasoning and Tool Use

GeoVista introduces agentic reasoning through an iterative Thought/Action/Observation loop. Built-in tools include:

• CropZoom (image region magnification)
• WebSearch (external retrieval)

Training utilizes a cold-start supervised fine-tuning stage (to teach tool invocation patterns) followed by reinforcement learning with a hierarchical, multi-level reward structure, yielding substantial performance improvements in city-level localization (Wang et al., 19 Nov 2025).

GeoBench (Segmentation): Model Adaptation Testbed

This benchmark provides a demanding environment for model adaptation research. The DARN architecture, for example, uses GeoBench to demonstrate adaptive regularization:

• Task Complexity Predictor (TCP): per-sample complexity estimation $c\in[0,1]$.
• Adaptive Dropout Modulation (ADM): dropout $p(c) = p_{\max} - (p_{\max}-p_{\min}) \cdot c$, yielding p from 0.1 to 0.5.
• Dynamic Capacity Gating (DCG): channel scaling linear in $c$.

DARN achieves 86.66% mean mIoU, a +5.56 percentage point improvement over previous state-of-the-art (TerraMind-L) (Yadav et al., 6 Nov 2025).

5. Benchmark Impact, Extensions, and Insights

GeoVista GeoBench has established a rigorous framework for evaluating agentic geolocalization on complex real-world imagery and has enabled the development of web-augmented vision-language agents that narrow the performance gap with closed-source models. Complementarily, the segmentation-focused GeoBench is a foundational testbed for general-purpose model adaptation in remote sensing. Its mixture of tasks, challenging modalities, and severe class imbalance catalyzes innovations in decoder design, regularization, and adaptation.

Both benchmarks emphasize:

• Standardized, reproducible evaluation protocols (e.g., prescribed splits, normalization, bootstrapped Interquartile Mean, confidence intervals).
• The necessity of moving beyond fixed-architecture or non-adaptive decoders for state-of-the-art performance in geospatial domains.
• Rich opportunities for future work in domain adaptation, minor class recognition, multi-modal fusion, and robustness.

6. Context within the Broader Field

GeoVista GeoBench and the segmentation-focused GeoBench share the goal of advancing geospatial AI via transparent, reproducible, and challenging evaluation. These efforts complement larger initiatives such as GEO-Bench-2, which unifies classification, segmentation, regression, detection, and instance segmentation across 19 datasets and introduces task "capability groups" to enable fine-grained model diagnostics (Simumba et al., 19 Nov 2025). However, both "GeoVista GeoBench" and the original GeoBench remain distinctive in their respective roles as agentic reasoning and multi-domain adaptation benchmarks rather than as generic leaderboard suites. Their adoption in cutting-edge model development underscores their significance in catalyzing research for robust, adaptable, and semantically rich geospatial foundation models.