SpatialAgent: Multi-Modal Spatial Reasoning

• SpatialAgent is a term for systems that combine language-model planning with explicit spatial reasoning over geographic databases, images, 3D scenes, and XR environments.
• These frameworks typically employ multi-agent pipelines that decompose tasks into specialized roles such as entity extraction, SQL generation, and result verification, integrating predetermined toolsets.
• SpatialAgent systems have demonstrated significant accuracy gains in geospatial querying, multimodal scene understanding, and benchmark evaluations through iterative self-correction and deterministic computations.

Searching arXiv for papers using or defining “SpatialAgent” and closely related spatial-agent frameworks. SpatialAgent is a term used in recent arXiv literature for agentic systems whose defining capability is spatial reasoning over geographic databases, multimodal images, embodied 3D scenes, or hybrid XR-IoT environments. In these works, the term does not denote a single canonical architecture; instead, it refers to a family of systems that combine language-model planning with spatial representations, external tools, memory, and verification loops in order to answer spatial questions, generate spatial SQL, localize places, manipulate layouts, or coordinate action in space (Kazazi et al., 23 Oct 2025, Huang et al., 14 Jul 2025, Wu et al., 22 May 2025, Bao et al., 23 Jan 2026).

1. Uses of the term in contemporary literature

Recent papers use “SpatialAgent” in several closely related senses. In database-centered work, it denotes a multi-agent stack that translates natural-language geospatial questions into executable PostGIS queries (Kazazi et al., 23 Oct 2025). In multimodal warehouse reasoning, it denotes an LLM-based agent that orchestrates distance, inclusion, and centroid tools over RGB images and object masks (Huang et al., 14 Jul 2025). In multimodal benchmark research, it denotes a tool-augmented reasoning system that coordinates specialized vision-geometry modules under Plan-Execute and ReAct paradigms (Wu et al., 22 May 2025). In geo-analytical question answering, the corresponding formulation appears as “Spatial-Agent,” where geo-analytical QA is formalized as concept transformation through GeoFlow Graphs (Bao et al., 23 Jan 2026).

Paper	Environment	Primary function
(Kazazi et al., 23 Oct 2025)	PostGIS-enabled databases	Natural language to spatial SQL
(Huang et al., 14 Jul 2025)	Warehouse RGB-D scenes with masks	Spatial question answering
(Wu et al., 22 May 2025)	Multimodal spatial benchmarks	Tool-augmented spatial reasoning
(Bao et al., 23 Jan 2026)	Map APIs and geospatial tools	Geo-analytical workflow construction
(Guan et al., 2024)	XR-IoT smart spaces	Zone-based spatial agents

This diversity suggests that the term is best understood as a research umbrella for systems in which spatial state, spatial transformation, or spatial computation is the primary substrate of agency. Closely related formulations extend the same idea to travel planning and last-100-meter navigation, compute-grounded industrial QA, UAV-satellite reasoning, geo-localization, GeoJSON execution pipelines, and neuroscience-inspired embodied cognition (Deng et al., 9 Jul 2025, Sharma, 13 Apr 2026, Zhang et al., 30 Jun 2026, Jia et al., 10 Feb 2026, Luo et al., 10 Sep 2025, Manh et al., 11 Sep 2025).

2. Recurrent architectural pattern

Across these systems, a recurring pattern is decomposition into specialized roles rather than reliance on a monolithic model. “From Questions to Queries” implements a controller-mediated multi-agent pipeline with entity extraction, metadata retrieval, query logic formulation, SQL generation, and a Review Agent for self-verification; the Review Agent executes SQL in a sandbox, parses manifests, checks schema compatibility, and repairs errors before returning validated output (Kazazi et al., 23 Oct 2025). “GeoJSON Agents” adopts a simpler Planner/Worker split in which the Planner decomposes a natural-language GIS task and the Worker either invokes a function library or generates Python code in a sandbox (Luo et al., 10 Sep 2025). “SpatialScore” instantiates the same principle in two variants: a Plan-Execute configuration with Planner, Executor, and Summarizer, and a ReAct configuration with Observer, Executor, and Summarizer, both operating over a toolbox of specialized spatial modules (Wu et al., 22 May 2025).

A second recurrent feature is tool use. The warehouse SpatialAgent does not rely on large-scale MLLM finetuning; instead, it uses Gemini 2.5-Flash in function-calling mode together with a small set of learned perception tools and simple geometric functions, including a ResNet-50 distance regressor, a ResNet-50 inclusion classifier, and centroid-based relation operators (Huang et al., 14 Jul 2025). The spatio-temporal text-to-SQL pipeline in “From Queries to Insights” similarly attributes its gains not to a stronger SQL generator alone, but to Mistral-based ReAct orchestration over schema inspection, SQL generation, execution, visualization, and file-reading tools (Redd et al., 29 Oct 2025). “Spatial Atlas” generalizes this into compute-grounded reasoning, in which every answerable sub-problem is resolved by deterministic computation before an LLM is asked to generate, with routing across a three-tier frontier model stack and structured intermediate representations (Sharma, 13 Apr 2026).

A third pattern is explicit self-correction. SpatialAgent systems often execute intermediate actions, inspect outputs, and replan. This is explicit in the SQL review loop (Kazazi et al., 23 Oct 2025), in the code-generation debugging cycle of GeoJSON Agents (Luo et al., 10 Sep 2025), in the retry mechanism of the warehouse agent when tool syntax fails (Huang et al., 14 Jul 2025), and in score-driven iterative refinement for machine-learning tasks in Spatial Atlas (Sharma, 13 Apr 2026).

3. Spatial representations and reasoning substrates

The systems differ most sharply in their internal representations of space. In spatial Text-to-SQL, representation is schema- and function-centric. The database-oriented SpatialAgent maintains a knowledge base with column-level and table-level metadata, including geometry subtype, SRID, ST_IsValid, spatial extent, and temporal coverage; embeddings and cosine similarity are used to retrieve trimmed schema subsets and relevant PostGIS operators such as ST_Intersects, ST_DWithin, ST_Area, ST_Perimeter, and ST_Transform (Kazazi et al., 23 Oct 2025).

In visual spatial QA, the representation is often image-anchored. The warehouse SpatialAgent treats reasoning as functions over images and masks, with centroid computation for left-right relations and learned models $D = F(I, M_1, M_2)$ and $y = G(I, M_1, M_2)$ for distance regression and inclusion classification (Huang et al., 14 Jul 2025). SpatialScore expands this to a toolbox of open-vocabulary recognition, localization, segmentation, optical flow, SIFT matching, homography estimation, object depth, region depth, camera parameter estimation, and orientation prediction; the overall spatial QA task is formalized as $$\mathbf{r} = \mathcal{A}(\mathbf{q}, \mathbf{v}; \mathcal{T})$$ (Wu et al., 22 May 2025).

Other work makes the representation itself the central contribution. Spatial-Agent for geo-analytical reasoning formalizes question answering as a concept transformation problem and represents workflows as GeoFlow Graphs $G = (V, E, \lambda, \rho)$, where nodes are spatial concepts and edges are transformations constrained by role ordering, type compatibility, acyclicity, executability, and connectivity (Bao et al., 23 Jan 2026). Spatial Atlas constructs a structured spatial scene graph whose nodes are spatial entities with positions, attributes, and zones, and whose edges encode spatial relations and distances before generation is attempted (Sharma, 13 Apr 2026). SatAgent lifts UAV imagery into explicit 3D spatial structure through a Geometric-Aware 3D Reconstruction Encoder, aligns UAV and satellite views in a common BEV frame, and reasons over topology-semantic fused tokens (Zhang et al., 30 Jun 2026). The neuroscience-inspired framework in “Mind Meets Space” argues for a still richer substrate consisting of multimodal sensing, egocentric-allocentric conversion, artificial cognitive maps, spatial neural memory, and predictive world models (Manh et al., 11 Sep 2025).

These choices correspond to different notions of “space”: database geometry, masked image regions, topological scene graphs, map APIs, BEV reconstructions, and cognitive maps. The literature therefore treats SpatialAgent less as a single model class than as an interface between a planner and an explicitly spatial computational substrate.

4. Major application domains

One major domain is geospatial database interaction. “From Questions to Queries” targets non-expert GIS users, domain experts, and autonomous GIS pipelines that need to translate high-level questions into correct PostGIS over points, polylines, polygons, CRS transformations, and spatial joins (Kazazi et al., 23 Oct 2025). “From Queries to Insights” extends this toward spatio-temporal analytics over check-in data, where the agent must align vague user phrasing with schema-specific categories, handle temporal reasoning, and choose maps, plots, or summaries (Redd et al., 29 Oct 2025). “GeoJSON Agents” moves from SQL to file-based GIS automation, transforming natural-language tasks into GeoJSON operation commands through either Function Calling or Code Generation (Luo et al., 10 Sep 2025). “Spatial-Agent: Agentic Geo-spatial Reasoning with Scientific Core Concepts” further shifts the emphasis from API calling to theory-grounded workflow synthesis (Bao et al., 23 Jan 2026).

A second domain is multimodal scene understanding. The warehouse SpatialAgent answers questions about object retrieval, counting, distance, and inclusion in complex indoor scenarios using image-and-mask tools rather than monolithic multimodal finetuning (Huang et al., 14 Jul 2025). SpatialScore’s SpatialAgent is a benchmark-facing system for 3D spatial perception and understanding across 28K multimodal samples and a challenging hard subset, addressing depth, distance, camera motion, object properties, and transformation reasoning (Wu et al., 22 May 2025). SpatialAct is not itself called SpatialAgent, but it directly probes the same class of capability in 3D layout-editing scenarios, where agents must maintain coherent spatial beliefs under action-induced scene changes (Liu et al., 29 May 2026).

A third domain is spatial mobility and geo-experience. The user-centric geo-experience framework uses three cooperating agents—a Travel Planning Agent, a Destination Assistant Agent, and a Local Discovery Agent—to cover multimodal trip planning, last-100-meter navigation, and disruption-aware local discovery via grids, bearings, geospatial APIs, and image embeddings (Deng et al., 9 Jul 2025). SpotAgent recasts geo-localization as a tool-using POMDP in which an LVLM alternates between visual interpretation, web search, geocoding, and zoom actions to produce verifiable coordinates (Jia et al., 10 Feb 2026). SatAgent extends spatial agency to aerospace embodied intelligence by jointly leveraging satellite and UAV perspectives for spatial reasoning in urban environments (Zhang et al., 30 Jun 2026).

A fourth domain is smart space and embodied systems. The XRI zone-agent framework treats space itself as an agentic entity: zones in mixed physical-virtual environments become smart, social, scalable, expressive controllers over IoT and XR content (Guan et al., 2024). More abstractly, the neuroscience-grounded perspective argues that robust agentic spatial intelligence requires explicit modules for multisensory integration, reference-frame transformation, memory, and predictive reasoning, especially for robotics and unstructured 3D environments (Manh et al., 11 Sep 2025).

5. Empirical performance and evaluation regimes

Evaluation protocols vary widely by domain, but nearly all reported studies show large gains from explicit spatial tooling and orchestration. In spatial Text-to-SQL, the multi-agent system in “From Questions to Queries” achieved 81.2% accuracy on KaggleDBQA after review, compared with 68.7% before review, and 87.7% on the 90-question SpatialQueryQA benchmark, compared with 76.7% without the Review Agent; accuracies after review were 93.3% for basic, 90.0% for intermediate, and 80.0% for advanced spatial queries (Kazazi et al., 23 Oct 2025). In spatio-temporal Text-to-SQL, the agentic orchestration pipeline achieved 91.4% accuracy on 35 queries over NYC and Tokyo check-in data, versus 28.6% for the naive baseline using the same SQL generator (Redd et al., 29 Oct 2025).

In warehouse spatial QA, the LLM-agent system achieved 95.8638% accuracy and 1st place on Track 3 of the 2025 AI City Challenge (Huang et al., 14 Jul 2025). In multimodal spatial benchmark evaluation, SpatialAgent-Intern-PE achieved 46.08 overall on SpatialScore-Hard, markedly above the underlying open-source MLLM and above several frontier proprietary baselines reported in the same table (Wu et al., 22 May 2025). In geo-analytical QA, Spatial-Agent reached 45.15 overall on MapEval-API with GPT‑4o-mini and 71.88 with GPT‑5; on MapQA, reported overall accuracies reached 61.45 with GPT‑4o-mini and 62.45 with LLaMA‑70B (Bao et al., 23 Jan 2026). In GeoJSON-based GIS automation, the Function Calling system achieved 85.71% and the Code Generation system 97.14%, both above the best-performing general-purpose model at 48.57% (Luo et al., 10 Sep 2025).

Benchmark methodology itself is a live issue. The SpatialQueryQA study explicitly uses manual, rationale-based evaluation because many spatial questions admit multiple correct SQL formulations and some benchmark “gold” queries are incomplete or suboptimal (Kazazi et al., 23 Oct 2025). SpatialAct, by contrast, emphasizes decomposed diagnostics and shows a reasoning-to-action gap: on multi-turn interactive refinement, the best tested model achieved Repair Rate 0.411 and Scene Success Rate 0.206, versus human baselines of 0.911 and 0.763 (Liu et al., 29 May 2026). Spatial Atlas reports 0.72, 0.68, and 0.74 on factory, warehouse, and retail splits of FieldWorkArena with its full compute-grounded pipeline, and 0.82 valid submission rate with 0.32 medal-level rate on MLE-Bench (Sharma, 13 Apr 2026). Taken together, these results indicate that “spatial intelligence” is being measured across at least three distinct axes: accurate spatial computation, robust perception-grounded reasoning, and action-conditioned spatial control.

6. Open problems, failure modes, and theoretical directions

A persistent problem is that spatial competence remains brittle when underlying representations are incomplete or when models must maintain state across turns. The spatial Text-to-SQL system still exhibits failure modes in geometric reasoning, such as incorrect planar versus geodesic choices and misapplied geometry functions in advanced aggregations (Kazazi et al., 23 Oct 2025). The warehouse SpatialAgent depends on accurate masks and a fixed toolset, and it does not maintain a persistent global map across questions (Huang et al., 14 Jul 2025). SpatialScore notes that tool inaccuracies propagate upward and that the toolbox remains mostly “2.5D,” lacking direct point-cloud, mesh, or full 3D grounding support (Wu et al., 22 May 2025). SpotAgent improves verifiability but remains vulnerable in sparse-cue or misleading-semantic scenes, such as indoor ambiguity or strong but deceptive branding cues (Jia et al., 10 Feb 2026).

A second open problem concerns the gap between static reasoning and spatial action. SpatialAct shows that strong VLMs can score in the $0.7$–$0.8$ range on several isolated spatial reasoning tasks yet fail to maintain coherent beliefs under iterative move/rotate/scale operations, with high premature-stop rates and low effective repair ratios (Liu et al., 29 May 2026). Spatial Atlas responds by insisting on deterministic computation for answerable sub-problems before language generation, while SatAgent responds by constructing explicit BEV-aligned world models rather than relying on 2D appearance priors (Sharma, 13 Apr 2026, Zhang et al., 30 Jun 2026).

A third issue is conceptual fragmentation. Current papers use “SpatialAgent” for spatial SQL systems, warehouse QA agents, benchmark toolchains, XR zone agents, geo-analytical planners, and neuroscience-inspired embodied architectures. This suggests that the field is organized more by shared design commitments than by a stable object class. A plausible implication is that future standardization will turn on internal representations—scene graphs, GeoFlow Graphs, BEV maps, cognitive maps, memory structures—rather than on any single planner or model family.

The strongest forward-looking proposal is the neuroscience-inspired one: a SpatialAgent should not merely call tools over static spatial inputs, but should integrate multimodal sensing, multi-sensory integration, egocentric-allocentric conversion, artificial cognitive maps, spatial memory, and predictive spatial reasoning into a continuous perception-cognition-action loop (Manh et al., 11 Sep 2025). Whether that broader program can be reconciled with the successful but narrower tool-augmented systems in Text-to-SQL, GIS automation, and benchmark-oriented spatial QA remains an open question.