EVGeoQA: Dynamic Geo-Spatial EV Charging Benchmark

• EVGeoQA is a benchmark for geo-spatial exploration that integrates dynamic, location-anchored queries to satisfy both EV charging requirements and nearby activity preferences.
• It employs a sequential decision process using hard POI constraints and driving distance minimization, facilitated by tool-augmented agents like GeoRover.
• The framework exposes emergent behaviors in tool use and context management, highlighting challenges in multi-step geospatial reasoning and long-horizon exploration.

EVGeoQA is a benchmark for dynamic, multi-objective geo-spatial exploration built around electric-vehicle charging scenarios. Its defining properties are a location-anchored query model, in which each query is explicitly tied to a user’s real-time coordinate, and a dual-objective task design, in which a valid answer must simultaneously satisfy charging necessity and a co-located activity preference. The benchmark is paired with GeoRover, a tool-augmented agent framework for evaluating whether LLMs can conduct sequential exploration under these constraints rather than performing static retrieval alone (Wu et al., 8 Apr 2026).

1. Formal task structure

EVGeoQA formulates the exploration problem at discrete step $t$ with a user location

$L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$

a set of charging stations $S=\{S_i\}$ with coordinates $(x_i,y_i)$ and associated POIs $P(S_i)$, and a natural-language query $Q$ issued at $t=0$ and anchored to $L_u(0)$. The benchmark seeks a station $S^*$ that satisfies two conditions: it must meet the EV charging requirement, and it must lie within walking distance of at least one POI matching the user’s secondary intent. The station-ranking criterion is minimum driving distance from the anchored user location (Wu et al., 8 Apr 2026).

The paper defines the co-located activity component through

$$A(S; \text{intent}) = \max_{p \in P(S)} \operatorname{sim}(p.category, intent),$$

and then solves for stations satisfying the POI constraint $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$0, ranking them by

$L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$1

An equivalent multi-objective view is also given as minimizing

$L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$2

under $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$3, or using hard constraints $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$4 and $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$5. In the operational benchmark design, however, no explicit $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$6-weighting is used; instead, the objectives are solved sequentially by hard filtering on the co-located activity condition and then minimizing driving distance (Wu et al., 8 Apr 2026).

This formulation distinguishes EVGeoQA from question answering settings in which the target is fixed independently of user state. In EVGeoQA, the correct answer depends jointly on the current user coordinate, the station set, the nearby POI configuration, and the semantic match between POI category and user intent.

2. Dataset construction and annotation

The dataset is built from three external sources. Charging-station records come from the State Grid Corporation of China. POIs within $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$7 of each station are retrieved via the Gaode (Amap) API. Population-density and road-network heatmaps are obtained from Baidu map heatmap imagery (Wu et al., 8 Apr 2026).

User locations are sampled from the heatmap space by treating heatmap pixels $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$8 as inputs to $L_u(t) = (x_t, y_t) \in \mathbb{R}^2,$9-means clustering with objective

$S=\{S_i\}$0

Each cluster receives a weight $S=\{S_i\}$1 reflecting population or road density, and sampling is then performed with

$S=\{S_i\}$2

This procedure ties the location distribution to semantically meaningful regions rather than uniform geographic sampling (Wu et al., 8 Apr 2026).

Dual-objective queries are generated in three stages. First, a small set of templates such as “Help me find a charging station near {POI}.” is instantiated using approximately 20 POI categories. Second, a strong LLM, qwen2.5-72B, paraphrases these into more realistic intents using Few-Shot+CoT prompting, turning static categories into formulations such as “doing errands,” “getting coffee,” or “working out.” Third, all candidate stations are exhaustively matched, and POI–intent alignment is enforced by cosine similarity at least $S=\{S_i\}$3 under CoNAN embeddings. Valid stations are ranked by driving distance, and up to five are retained as the ground-truth set. The authors additionally report manual verification of approximately 1,000 random QA pairs (Wu et al., 8 Apr 2026).

The benchmark is organized by city. Table 1 of the paper reports Hangzhou with 258 stations, 997 seed-locs, 25 POI-cats, and 19,940 QA-pairs; Qingdao with 165 stations, 995 seed-locs, 23 POI-cats, and 14,162 QA-pairs; and Linyi with 157 stations, 997 seed-locs, 21 POI-cats, and 14,416 QA-pairs (Wu et al., 8 Apr 2026). The city-wise construction matters because it couples exploration difficulty to actual station density, POI coverage, and road-network structure.

3. GeoRover and tool-augmented exploration

GeoRover is the evaluation framework used to measure model behavior on EVGeoQA. It is described as a general evaluation framework based on a tool-augmented agent architecture, with an LLM “brain” prompted by Few-Shot + Chain-of-Thought, a history buffer that stores prior prompts, thoughts, tool calls, and tool outputs, and four atomic tools exposed through simple APIs: SearchStations(coord, radius), SearchPOIs(coord, radius), ChangeLocation(direction, distance), and CalculateDistance(coord₁, coord₂) (Wu et al., 8 Apr 2026).

The dual objectives are operationalized through tool use. For charging necessity, the agent invokes SearchStations(center, 5 km) to retrieve candidate stations in a $S=\{S_i\}$4 radius. For the co-located activity preference, it queries SearchPOIs(S_i, 1 km) for each candidate station and checks whether there exists a POI satisfying $S=\{S_i\}$5 (Wu et al., 8 Apr 2026).

The exploration loop is iterative. The current location is initialized to $S=\{S_i\}$6, the history is initialized with the query, and the agent repeatedly builds a prompt from query plus history, emits either a tool call or a final answer, appends tool results to history, and updates the current location after ChangeLocation. The episode terminates when the model outputs “Answer” or when the number of calls reaches 20. The framework does not impose a hard-coded invocation order; the model decides when and which tool to call (Wu et al., 8 Apr 2026).

This autonomy is central to the benchmark. EVGeoQA is not merely an answer-selection task over precomputed candidates. It evaluates whether a model can decompose the problem into sub-tasks, manage a growing interaction history, decide when local evidence is insufficient, and extend the search spatially through explicit movement.

4. Metrics and empirical performance

The primary evaluation metric is Hits@K. For each test query $S=\{S_i\}$7, the agent outputs an ordered list of candidate stations $S=\{S_i\}$8, and the benchmark provides a ground-truth set $S=\{S_i\}$9. The metric is

$(x_i,y_i)$0

The benchmark also stratifies episodes by distance tier: Tier 1 for under $(x_i,y_i)$1, Tier 2 for under $(x_i,y_i)$2, and Tier 3 with no limit, using geodesic distance between the user location and the nearest ground-truth station. A tool-usage statistic is also reported as the average number of ChangeLocation calls per episode, $(x_i,y_i)$3 (Wu et al., 8 Apr 2026).

The evaluated baselines include Qwen3-8B and 30B, Qwen2.5-72B, GPT-OSS-20B and 120B, and Gemini-2.5 Flash and Pro, with “thinking” variants marked by explicit CoT prompting. The reported pattern is consistent across cities and tiers: short-range performance is materially better than long-range performance. Specifically, Hits@2 in the under-$(x_i,y_i)$4 regime is approximately $(x_i,y_i)$5–$(x_i,y_i)$6, but drops to $(x_i,y_i)$7–$(x_i,y_i)$8 in the no-limit regime; averaged across cities, Hits@2 is about $(x_i,y_i)$9 for under $P(S_i)$0, about $P(S_i)$1 for under $P(S_i)$2, and about $P(S_i)$3 with no distance limit (Wu et al., 8 Apr 2026).

The paper attributes part of the gain from “thinking” models to more persistent exploration. In Linyi under No-Limit, Qwen3-8B* makes about $P(S_i)$4 ChangeLocation calls versus $P(S_i)$5 for the vanilla variant, with a Hits@2 increase of approximately 5–6 points. At the same time, the experiments show that reliable tool invocation on local sub-tasks does not by itself solve long-range exploration. The dominant performance degradation occurs when relevant stations are not discoverable through shallow local search (Wu et al., 8 Apr 2026).

5. Failure modes and emergent behavior

The paper identifies several characteristic weaknesses. The first is “laziness”: premature termination and insufficient exploration, which produce false positives especially in long-range tiers. The second is “Lost in the Middle,” where long CoT contexts lead the model to conflate POI attributes with station attributes and to hallucinate unsupported conclusions. The third class includes argument errors and max-tool-call failures, which appear when agents loop, exceed limits, or violate API formats (Wu et al., 8 Apr 2026).

These failure modes indicate that the benchmark stresses more than semantic matching. The agent must maintain state across a long sequence of heterogeneous tool outputs, distinguish station-level from POI-level evidence, and continue exploration when current evidence is only locally satisfactory. This suggests that context management, action budgeting, and memory compression are structurally important to benchmark performance.

Alongside these weaknesses, the paper reports an emergent capability: high-performing agents can summarize historical exploration trajectories to improve exploration efficiency. In those cases, the model sketches previously explored anchor points and avoids redundant moves (Wu et al., 8 Apr 2026). The result is notable because the framework does not hard-code such summarization; it emerges from interaction between tool use, history accumulation, and the search objective.

The authors accordingly propose several future directions: reinforcement learning or supervised fine-tuning to penalize premature termination and encourage deeper exploration; enhanced context management through sliding windows or vector-database-backed retrieval over prior tool calls; explicit multi-objective optimization layers, including learnable $P(S_i)$6 in $P(S_i)$7; and cross-city as well as multilingual data augmentation to reduce geographical and linguistic bias (Wu et al., 8 Apr 2026).

6. Position within GeoQA and adjacent benchmarks

EVGeoQA belongs to a broader family of benchmarks for spatial, visual, and multimodal reasoning, but it occupies a distinct evaluation regime. GS-QA, for example, is an extensible geospatial QA benchmark with 2,800 question-answer pairs across 28 templates on top of OpenStreetMap and Wikipedia, covering a wide range of spatial predicates, answer types, and some multi-source reasoning; its focus is deterministic QA generation and evaluation over a spatial database rather than sequential exploration from a dynamic user location (Saeedan et al., 21 May 2026). GeoX-Bench evaluates cross-view geo-localization and pose estimation with 10,859 panoramic–satellite image pairs and 755,976 QA pairs, emphasizing localization and heading rather than tool-mediated search over charging and POI constraints (Zheng et al., 17 Nov 2025).

Other neighboring work lies in remote sensing and embodied spatial understanding. EarthVL introduces a progressive Earth vision-language framework with 10.9k sub-meter images and 761.5k textual pairs for object-relational reasoning in city-planning-oriented remote sensing (Wang et al., 6 Jan 2026). GeoEyes studies on-demand zooming for evidence-grounded understanding of ultra-high-resolution remote sensing imagery and identifies the failure mode of “Tool Usage Homogenization,” in which zoom-enabled MLLMs collapse into task-agnostic tool patterns (Wang et al., 15 Feb 2026). A plausible implication is that EVGeoQA’s long-range exploration failures and GeoEyes’ tool homogenization diagnose related issues in agentic multimodal systems: effective tool use requires task-adaptive control rather than mere access to tools.

A common terminological ambiguity arises from the label “GeoQA.” In mathematical diagram reasoning, GeoQA refers to a plane-geometry benchmark rather than geo-spatial exploration. EAGLE reports 67.1% zero-shot accuracy on the GeoQA benchmark for geometric problem solving (Li et al., 2024), and VeriGeo reports 82.74% on GeoQA after supervised fine-tuning on 8.7K verified synthetic examples (Duan et al., 12 Jun 2026). These works concern multimodal geometry problems with diagrams, proofs, and numerical verification, not EV charging or dynamic geospatial search. This distinction is important because EVGeoQA evaluates geo-spatial exploration under dynamic coordinates and compound constraints, whereas those “GeoQA” systems evaluate geometric reasoning in the Euclidean-diagram sense.

Taken together, these comparisons suggest that EVGeoQA serves as a specialized testbed for dynamic, location-anchored, dual-objective exploration. Its significance lies less in raw fact retrieval than in evaluating whether an LLM agent can integrate semantic intent matching, road-distance ranking, movement decisions, and long-horizon memory into a coherent exploratory policy.