GeoRover Evaluation Framework

• GeoRover is a framework that assesses tool-augmented LLMs in dynamic, multi-objective geo-spatial exploration by navigating dual constraints like EV charging and co-located activities.
• It employs four atomic geo-spatial tools—local search, POI retrieval, coordinate shifting, and distance calculation—integrated with few-shot and chain-of-thought prompting for structured reasoning.
• Empirical results reveal that long-range exploration reduces accuracy, while explicit reasoning modes and trajectory summarization enhance performance in complex urban scenarios.

GeoRover is the evaluation framework introduced in Section 4 of "EVGeoQA: Benchmarking LLMs on Dynamic, Multi-Objective Geo-Spatial Exploration" for systematically assessing LLMs as tool-augmented agents in dynamic, multi-objective geo-spatial exploration (Wu et al., 8 Apr 2026). It is designed to operate over EVGeoQA, a benchmark of location-anchored electric-vehicle charging queries that couple a primary charging need with a secondary co-located activity preference, such as "charge and grab coffee." In the materials considered alongside that paper, the same name also appears in planetary-robotics deployment contexts, where GeoRover denotes the rover platform to which methods for exploration, mineral mapping, and immersive teleoperation are adapted. The formal definition, however, is the EVGeoQA evaluation framework.

1. Definition, motivation, and conceptual scope

GeoRover targets a problem class that traditional geo-spatial question answering benchmarks largely do not: dynamic, purpose-driven exploration anchored to a user’s real-time coordinate under compound constraints (Wu et al., 8 Apr 2026). Earlier GSQA settings are described as emphasizing static retrieval over offline maps or facts, such as distances between landmarks. GeoRover instead enforces partial observability, multi-step reasoning, active search, navigation, and termination decisions. The evaluation problem is not merely whether a model can retrieve a fact from a map, but whether it can search locally, move its anchor point, summarize prior exploration, and stop only after satisfying a dual objective.

The benchmark task supplied to GeoRover is characterized in the source material as "Go to one place to do two things." In EVGeoQA, each query is explicitly bound to a user’s real-time coordinate and combines two objectives: charging necessity and a co-located activity preference. This framing makes the framework sensitive to spatial reachability, local context, and sequential tool use rather than to static factual recall alone. A plausible implication is that GeoRover sits closer to agentic planning evaluation than to classical retrieval-oriented GSQA, even though it remains benchmark-driven rather than embodied.

A frequent source of confusion is the relation between EVGeoQA and GeoRover. EVGeoQA is the benchmark: it defines the dataset, query generation pipeline, and ground truth. GeoRover is the evaluation framework: it supplies the agentic loop, tool APIs, and scoring protocol used to assess whether an LLM can solve EVGeoQA tasks through active exploration.

2. Tool-augmented agent architecture

GeoRover instantiates an LLM agent equipped with four atomic geo-spatial tools with local scope, built atop Gaode APIs (Wu et al., 8 Apr 2026). The agent autonomously decides which tool to call, how often, and when to terminate. Few-shot and Chain-of-Thought prompting are used to stabilize reasoning and improve task comprehension.

The four tools partition the search problem into local retrieval, movement, and quantitative comparison. SearchStations takes a coordinate and returns charging stations within a $5$ km radius, providing the local charging landscape around the current anchor. SearchPOIs takes a coordinate and returns points of interest within a $1$ km radius, enabling verification of the secondary activity constraint. ChangeLocation moves the current coordinate in one of four cardinal directions by a chosen distance and returns the new coordinate; it is the core exploration primitive for long-range search. CalculateDistance computes vehicle driving distance between two coordinates and is used to rank feasible candidates.

Beyond the explicit tools, the reported behavior implies four functional roles inside the framework. A planner proposes the next action and its parameters from current observations and task goals. A tool-caller interfaces with Gaode map search, POI retrieval, and routing APIs. A memory or trajectory summarizer synthesizes historical exploration steps to avoid redundant coverage and to select new anchor points that maximize spatial coverage. A critic or evaluator checks whether the charging and activity constraints are simultaneously satisfied and decides when to terminate with a final recommendation.

The interaction loop is sequential. GeoRover first reads the EVGeoQA query and initializes the current coordinate. It then searches for stations around that coordinate. For each promising station, it searches for nearby POIs to test activity compatibility. If multiple stations satisfy both constraints, it calculates driving distances and selects the optimal candidate. If no feasible station exists in the currently observable region, it invokes ChangeLocation, updates the anchor, and repeats the cycle. Historical trajectories are summarized to guide future moves and reduce repeated coverage. Termination occurs when a valid station satisfying both constraints is found.

The I/O contract is correspondingly explicit. Tool inputs include latitude-longitude coordinates, radii or step distances, direction flags, optional POI category filters, and coordinate pairs for distance calculation. Tool outputs include lists of stations with names and coordinates, lists of POIs with category, name, and coordinate, numeric distances, and updated coordinates after ChangeLocation.

3. Environment formalization, dataset coupling, and scoring

GeoRover is coupled tightly to EVGeoQA’s formalization of dynamic user location and dual-objective search (Wu et al., 8 Apr 2026). Each query $Q$ is bound to a real-time coordinate $L_u$, and the dynamic location is denoted

$x_t = (lat_t, lon_t).$

The target is a charging station $S$ that satisfies two conditions. First, $S$ must provide EV charging services. Second, within walkable distance, operationalized as $1$ km, there must exist at least one POI $P$ compatible with the user’s secondary intent. GeoRover’s local observability model is asymmetric: SearchStations exposes only stations within $5$ km of the current anchor, whereas SearchPOIs exposes only POIs within $1$0 km of a coordinate. This makes the environment partially observable and forces multi-step search.

Ground-truth construction in EVGeoQA uses exhaustive station matching and POI alignment verified by cosine similarity between the query’s activity slot and station-surrounding POIs using the CoNAN embedding model, with a threshold of $1$1. For each query, the retained ground-truth set contains up to five stations ranked by vehicle driving distance from the query coordinate. Manual verification of approximately $1$2 QA pairs is reported for naturalness and correctness. The benchmark spans three cities—Hangzhou, Qingdao, and Linyi—with the following statistics: stations $1$3, $1$4, and $1$5; user-anchor locations $1$6, $1$7, and $1$8; POI categories $1$9, $Q$0, and $Q$1; and QA pairs $Q$2, $Q$3, and $Q$4, respectively.

The primary accuracy metric is Hits@$Q$5 for $Q$6:

$Q$7

Here, $Q$8 is the predicted station for query $Q$9, and $L_u$0 is the set of up to $L_u$1 best ground-truth stations ranked by distance. Evaluation is stratified by three driving-distance tiers between user location and optimal station: $L_u$2 km, $L_u$3 km, and No Limit. The first tier is twice the SearchStations radius; the second is four times that radius. GeoRover also logs the average ChangeLocation call frequency per episode per tier as a measure of exploration depth.

The user anchors in EVGeoQA are synthesized by K-Means clustering of population or road heatmaps and Softmax sampling over semantic cluster weights. The reproduced formulas are

$L_u$4

and

$L_u$5

These formulas govern data generation rather than GeoRover’s online scoring. Likewise, the source material provides conceptual dual-objective and path-cost expressions as analytical aids, but explicitly notes that evaluation relies on Hits@$L_u$6 and distance tiers rather than explicit cost functions.

4. Empirical behavior, emergent capabilities, and error taxonomy

The main empirical finding is that accuracy deteriorates as exploration distance increases (Wu et al., 8 Apr 2026). In Hangzhou, an example reported in the source material shows average Hits@$L_u$7 dropping from $L_u$8 in the $L_u$9 km tier to $x_t = (lat_t, lon_t).$0 in the No Limit tier. This establishes long-range exploration, rather than local tool execution, as the principal difficulty. The paper also reports a "laziness" phenomenon: agents often terminate early and return plausible but incorrect answers without sufficient exploration.

Explicit reasoning modes improve performance. The reported "Thinking" variants consistently outperform their non-thinking counterparts. One cited example is the Hangzhou No Limit tier, where Qwen3-8B* reaches Hits@$x_t = (lat_t, lon_t).$1 while Qwen3-8B reaches $x_t = (lat_t, lon_t).$2. Larger models such as Qwen2.5-72B and Gemini-2.5-Pro generally perform better across tiers, which the paper describes as a scaling law pattern.

An especially notable observation is emergent trajectory summarization. The framework reports that LLMs, particularly "Thinking" models, can summarize historical exploration trajectories without explicit instruction and use those summaries to choose new anchors that maximize coverage and avoid redundancy. GeoRover detects this through tool-use logs and qualitative trajectory visualizations. In Linyi, for example, Gemini-2.5-Pro* shows average ChangeLocation calls of $x_t = (lat_t, lon_t).$3 in the $x_t = (lat_t, lon_t).$4 km tier, $x_t = (lat_t, lon_t).$5 in the $x_t = (lat_t, lon_t).$6 km tier, and $x_t = (lat_t, lon_t).$7 in the No Limit tier, with higher tool-use frequency correlated with improved accuracy on harder tasks.

The reported error taxonomy separates failures into four categories.

• Insufficient Exploration: premature stopping without satisfying dual constraints.
• Max Tool Call: logical loops or inefficient strategies that exhaust the tool budget, cited as $x_t = (lat_t, lon_t).$8 calls in the Appendix.
• Argument Error: incorrect tool parameter formatting.
• Factual Conflation: hallucination or Lost-in-the-Middle behavior that mixes stations, POIs, and trajectory content.

These failure modes indicate that GeoRover stresses not only map reasoning but also long-horizon control, structured API use, and large-context synthesis under dense urban ambiguity.

5. Alternate usage in planetary rover research

In adjacent papers included with the topic, "GeoRover" is used not as the EVGeoQA evaluation framework but as the rover platform or deployment target for perception, exploration, and teleoperation methods. This produces a broader, polysemous usage of the term within the supplied literature.

Context	GeoRover role	Associated method
EVGeoQA	Evaluation framework	Tool-augmented LLM exploration
Planetary survey	Deployment target	Bi-level image-guided ergodic exploration
Mineral mapping	Deployment target	GyPSUM hyperspectral clustering
Lunar teleoperation	Deployment target	XR+AI immersive control

The paper "Bi-Level Image-Guided Ergodic Exploration with Applications to Planetary Rovers" describes how a GeoRover can use image-guided ergodic exploration for geological survey and localization of rock formations (Wittemyer et al., 2023). The method decomposes planning into a coarse body-level ergodic planner over the rover workspace and a fine sensor-level planner over camera yaw and pitch. The standard ergodic objective is written as

$x_t = (lat_t, lon_t).$9

with a coarse occupancy grid of $S$0 over approximately $S$1, a coarse horizon $S$2, and a fine horizon $S$3 per coarse pose. In the reported Mars-analog simulation, bi-level optimized camera control found on average $S$4 of rocks over $S$5 minutes with mean path length $S$6 m, compared with $S$7 and $S$8 m for ergodic trajectory optimization with a fixed camera and $S$9 and $S$0 m for a random camera path. In this literature, GeoRover functions as the vehicle on which the exploration method is operationalized.

The paper "Generalized Unsupervised Clustering of Hyperspectral Images of Geological Targets in the Near Infrared" describes how the GyPSUM pipeline can be used by a geologic rover, GeoRover, to identify and map mineralogical units from near-infrared hyperspectral imagery (Gao et al., 2021). GyPSUM consists of preprocessing, a lightweight autoencoder, Gaussian mixture modeling, optional post-processing, and mapping clusters back to image space. It operates on the $S$1-$S$2 nm spectral range, uses spectral-angle reconstruction loss,

$S$3

and clusters latent representations with a GMM of the form

$S$4

Reported runtimes are approximately $S$5 minutes for the Oman core image and $S$6 minutes for the Jezero scene. The method is described as consistently identifying and separating major mineral classes in both submillimeter laboratory imagery and meter-scale orbital imagery, with ratioing improving cluster separability on CRISM data.

The paper "Immersive Rover Control and Obstacle Detection based on Extended Reality and Artificial Intelligence" presents an XR+AI system for lunar rover teleoperation and frames its extensions in terms of GeoRover-style deployment (Coloma et al., 2024). The system uses an Intel RealSense D455 RGB-D camera, YOLOv5 for 2D rock detection, RTAB-Map for SLAM and 3D reconstruction, a ROS–Unity bridge between Ubuntu/ROS and Unity3D on Windows, and an HTC Vive Cosmos Elite VR system. Validation in the University of Luxembourg’s LunaLab involved five participants performing two runs each: a 2D monitor-based condition and an XR condition with a 3D mesh visualization. The paper reports qualitative outcomes rather than numerical workload scores, but participants reported reduced cognitive load and better spatial perception with XR.

Taken together, these papers do not redefine GeoRover’s formal benchmark meaning in EVGeoQA. They instead demonstrate that the name also circulates as a planetary rover platform concept linking geo-spatial reasoning, image-guided exploration, hyperspectral mineral mapping, and immersive teleoperation.

6. Limitations, misconceptions, and prospective extensions

The most important misconception is to collapse GeoRover and EVGeoQA into a single object. The source material explicitly separates them: EVGeoQA is the benchmark and query-generation pipeline, whereas GeoRover is the evaluation framework that assesses LLMs by means of an agentic tool loop, trajectory logging, and Hits@$S$7 scoring (Wu et al., 8 Apr 2026). A second misconception is to treat the conceptual dual-objective score or path-cost expressions as official scoring functions. The material states that these are analytical aids; actual evaluation uses Hits@$S$8 and distance tiers.

GeoRover’s current scope is constrained by the benchmark on which it is built. EVGeoQA covers three Chinese cities—Hangzhou, Qingdao, and Linyi—and is described as predominantly Chinese language, creating potential linguistic and geographic bias. Long-horizon search and large-context synthesis remain challenging, and dense urban environments induce attribute conflation across stations, POIs, and historical trajectories. These limitations are visible both in declining long-range accuracy and in the documented error categories.

At the same time, the framework is explicitly positioned as extensible. The source material states that the same tool-augmented exploration paradigm generalizes to tasks such as "park and buy groceries," "visit hospital with nearby pharmacy," "tourist sightseeing with lunch," and logistics planning under compound constraints. Future directions noted by the authors include multilingual datasets, more global cities, and investigation of domain-specific fine-tuning and adaptation techniques such as SFT and LoRA to strengthen long-horizon planning, reduce "laziness," and improve synthesis of heterogeneous geo-spatial information.

The adjacent rover-oriented literature suggests a broader interpretation of GeoRover as a bridge term between geospatial benchmark evaluation and physically situated rover autonomy. This suggests continuity rather than identity: the EVGeoQA GeoRover is a formal evaluation framework for tool-augmented LLMs, while the planetary uses treat GeoRover as a rover platform to which exploration, sensing, and teleoperation methods can be attached.