---
title: Spatial Reasoning Benchmarks
url: https://www.emergentmind.com/topics/spatial-reasoning-benchmarks
type: topic
---

# Spatial Reasoning Benchmarks

Spatial Reasoning Benchmarks

Spatial reasoning benchmarks provide systematic, multi-task evaluations of computational agents’ abilities to perceive, represent, and manipulate spatial relations, structure, transformations, and trajectories in 2D, 3D, and 4D settings. These benchmarks are designed to probe distinct facets of spatial intelligence—ranging from low-level perceptual grounding to high-level causal inference and planning—using controlled datasets, diverse task taxonomies, and rigorously defined performance metrics. They are vital for revealing persistent limitations in contemporary large language models (LLMs), vision-language models (VLMs), and multimodal large language models (MLLMs), and serve as the basis for progress in embodied AI, robotics, scene understanding, and agentic systems.

## 1. Taxonomies of Spatial Reasoning Abilities

Benchmarks have formalized a range of taxonomic frameworks to partition spatial cognition hierarchically or quadrantically.

- **Hierarchical Cognitive Levels**
  - *Observation*: Object enumeration, attribute extraction ([2511.21471]).
  - *Topology & Relations*: Adjacency, containment, relative position, and temporal ordering ([2511.21471], [2503.19707]).
  - *Symbolic Reasoning*: Mapping spatial cues to abstract rules, multi-hop inference ([2511.21471]).
  - *Causality*: Predicting outcomes under hypothetical movement or interaction ([2511.21471]).
  - *Planning*: Synthesizing sequences to achieve spatial goals ([2511.21471], [2606.09669]).

- **DISE Quadrants (Spatial-DISE)**
  - *Intrinsic-Static*: Reasoning over internal object structure (e.g., which face painted blue).
  - *Intrinsic-Dynamic*: Predicting effects of intra-object transformations (folding, rotating).
  - *Extrinsic-Static*: External relations among objects (projection, view-based correspondence).
  - *Extrinsic-Dynamic*: Multi-object, transformation-changing relations (assembly, multi-step manipulation) ([2510.13394]).

Benchmarks such as SpatialBench [2511.21471], Spatial-DISE [2510.13394], and GamiBench [2512.22207] explicitly operationalize these taxonomies in their dataset construction and task design, facilitating both fine-grained skill attribution and unified capability metrics.

## 2. Benchmark Construction Methodologies

Spatial reasoning benchmarks employ a range of rigorous methodologies optimized for reproducibility, coverage, and diagnostic power.

- **Automated Procedural Generation**: Synthetic scene generation with controlled object placement, geometry, and distractor crafting, using engines like Blender (Spatial457 [2502.08636], Spatial-DISE [2510.13394], SIRI-Bench [2506.14512]).
- **Expert-Annotated Reasoning Chains**: Canonical solution steps annotated for logical dependency tracking, supporting process-level evaluation (DynaSolidGeo [2510.22340]).
- **Human-Validated Naturalistic Data**: Real images or videos from datasets such as LSUN, COCO, GQA, AI2-THOR, ScanNet, and field data, annotated for reference frames and hierarchical phenomena (SpatialText [2603.03002], SpatialWorld [2606.09669], CityCube [2601.14339]).
- **Simulation Environments**: Integration of multiple backends and agent interfaces to probe interactive, sequential, or agentic tasks under partial observability (SpatialWorld [2606.09669], EvoEmpirBench [2509.12718]).

These methods enable large-scale, balanced, and robust datasets supporting both passive and interactive task formats. Controlled distractor generation and multi-view or multi-step setups are standard to preclude superficial pattern matching.

## 3. Task Typologies and Diagnostic Sub-abilities

Benchmarks comprehensively span sub-skills to dissect spatial competence:

- **Perceptual and Relational Tasks**: Primitive object identification, 2D/3D localization, spatial relation extraction (GSR-Bench [2406.13246], Spatial457 [2502.08636], Spatial Reasoning in Foundation Models [2509.21922]).
- **Geometric Manipulation**: Mental rotation, spatial visualization, cross-sectional inference, origami folding, and shape reconstruction (SpatialViz-Bench [2507.07610], SpinBench [2509.25390], GamiBench [2512.22207]).
- **Physical Constraints and Dynamics**: Reasoning under extrinsic and intrinsic-dynamic transformations; enforcing occlusion, support, and contact constraints (SSI-Bench [2602.07864], DynaSolidGeo [2510.22340]).
- **Sequential and Agentic Planning**: Multi-step pathfinding, navigation, and active exploration under partial observability (SpatialWorld [2606.09669], GRASP [2407.01892], EvoEmpirBench [2509.12718]).
- **4D Spatiotemporal Cognition**: Memory, action recognition, state-change detection, and prediction in video (Spatial4D-Bench [2601.00092]).
- **Pure-Text Spatial Reasoning**: Mental modeling from text alone, disentangling visual pattern-matching (SpatialText [2603.03002]).

Benchmarks have instantiated specialized metrics beyond raw accuracy, e.g., Viewpoint Consistency (VC) and Impossible Fold Selection Rate (IFSR) in GamiBench [2512.22207], Relative Performance Dropping Rate (RPDR) in Spatial457 [2502.08636], and process-qualified accuracy in DynaSolidGeo [2510.22340].

## 4. Empirical Findings and Performance Stratification

Systematic evaluation across dozens of state-of-the-art open-source and proprietary models reveals consistent trends and bottlenecks:

| Benchmark           | Human Best         | Top Model    | Open Model   | Random    | Notable Failure Modes                      |
|---------------------|-------------------|-------------|--------------|-----------|--------------------------------------------|
| SpatialBench        | 96.4% overall     | Gemini-2.5-p| Qwen3-VL-235B| n/a       | Symbolic (L3), Causal (L4), Planning (L5)  |
| GSR-Bench           | >90% Subset A     | LLaVA-NeXT  | Qwen1.5-110B | ~25%      | Behind/in-front relation, small objects    |
| CityCube            | 88.3%             | Doubao-1.6  | GLM-4.1V-9B  | 22.8%     | Cross-view, scale, egocentric rotation     |
| SpinBench           | 91.2%             | InternVL3-38| InternVL-14B | n/a       | Mental/persp. rotation, viewpoint change   |
| SpatialViz-Bench    | ~95%              | Gemini-2.5-p| LLama-4-Scout| 25–27%    | 3D folding, animation, formulaic bias      |
| DynaSolidGeo        | n/a               | GPT-5       | Qwen3-VL-30B | n/a       | Visual perception, logic, hallucination    |
| Spatial457          | n/a               | GPT-4o      | InternVL2 8B | n/a       | 3D pose, depth, collision (6D)             |
| Spatial-DISE        | 76.8%             | Doubao1.5VL | InternVL-3   | 25%       | Multi-step, multi-view reasoning           |
| EarthSpatialBench   | F1≈ 0.91 (Within) | Gemini-2.5-p| Qwen3-VL-T-30| n/a       | Visual grounding, composite geometry       |
| SIRI-Bench          | ~70% (<60% error) | Doubao-1.5-p| Qwen2.5-VL-72| n/a       | Parameter extraction from video            |
| SpatialWorld        | n/a               | GPT-5 (17.4%)| Qwen-3.5 (14.1%) | n/a   | Partial observability, long-horizon plan   |
| EvoEmpirBench       | n/a               | -           | -            | n/a       | Local memory, dynamic state update         |

Across all benchmarks, performance decays markedly as tasks move from static, single-image perception (object detection, simple relations) to high-dimensional, dynamic, multi-step, and cross-perspective reasoning (mental rotation, planning, causal inference, spatiotemporal prediction). Even top proprietary models routinely trail human accuracy by 20–50 percentage points on composite tasks ([2511.21471], [2512.22207], [2601.14339], [2509.12718]).

## 5. Failure Modes and Diagnostic Insights

Benchmarks systematically expose persistent model deficiencies, which cluster into:

- **Egocentric/Reference Frame Bias**: Overcommitment to observer-centric frames; failures under allocentric or perspective-shifted queries ([2509.25390], [2603.03002]).
- **Rotational and Mental Simulation Gaps**: Chance-level performance on dynamic/mental rotation, folding, cross-sectional reasoning, animation, and long-range transformations ([2509.25390], [2512.22207], [2507.07610]).
- **Visual Plausibility Bias & Shortcutting**: Preference for coherent yet incorrect visual patterns; overreliance on surface cues and object priors ([2512.22207], [2601.14339]).
- **Degraded Planning under Partial Observability**: Inefficient exploration and lack of recovery from errors in agentic tasks; trial-and-error search rather than strategic execution ([2606.09669], [2407.01892], [2509.12718]).
- **Combinatorial and Dynamic Complexity Collapse**: Near-random performance on multi-view, multi-step, and sim-to-real generalization tasks involving several spatial factors or real-world grounded transformations ([2502.08636], [2510.13394], [2510.22340], [2601.14339]).
- **Hallucination and Logical Inconsistency**: In process-evaluation setups, frequent invalid inferences, unjustified leaps, or contradictions in reasoning chains ([2510.22340], [2603.03002]).

Scaling model size, instruction tuning, and chain-of-thought prompting yield only modest improvements for high-complexity tasks; numeric gains are predominantly in perception and simple relational categories ([2511.21471], [2512.22207], [2503.19707]).

## 6. Directions for Benchmark and Model Innovation

Identified gaps motivate several directions:

- **Explicit Geometric and Physics Modules**: Integration of 3D pose/orientation estimators, physics engines, and spatial graph architectures ([2502.08636], [2510.22340], [2602.07864]).
- **Programmatic, Multi-modal Testing**: Composition of numeric/geometric input (bboxes, polylines) with raster and natural-language contexts for precise, multifaceted evaluation ([2602.15918], [2511.21471]).
- **Process-oriented Metrics**: Judgement of reasoning chains, process-tracing, and actionable sequence synthesis rather than only answer accuracy ([2510.22340], [2512.20595]).
- **Interactive, Task-driven Evaluation**: Shift from passive VQA paradigms to agentic, partially observable, iterative-action tasks coupled to real-world simulators ([2606.09669], [2509.12718]).
- **Sim-to-Real and Multi-environment Generalization**: Extension of task variety, environment complexity, and observability structure to stress sim-to-real robustness ([2606.09669], [2510.13394]).
- **Cognitive and Processual Grounding**: Taxonomies and constructions explicitly grounded in human cognitive science and psychometric frameworks ([2511.21471], [2507.07610], [2603.03002]).

Collectively, these future-oriented approaches aim for models that can “think in space”—not just recognize spatial features, but robustly simulate, plan, and reason about geometric, topological, and physical constraints across modalities, views, and horizons.

## 7. Representative Benchmarks: Summary Table

| Benchmark             | Modalities   | Categories/Taxonomy    | Key Diagnostic Features                    | arXiv ID       |
|-----------------------|-------------|------------------------|--------------------------------------------|---------------|
| SpatialBench          | Multi-modal | 5-level cognitive      | Unified metric, 15 tasks, L₅ planning      | 2511.21471    |
| GamiBench             | Visual      | 2D→3D planning         | Origami: cross-view, physical feasibility  | 2512.22207    |
| GSR-Bench             | Visual      | Relations, grounding   | CircularEval, mask/depth, scaling laws     | 2406.13246    |
| Spatial457            | Visual      | 6D spatial             | Level-by-level, unbiased attribute, RPDR   | 2502.08636    |
| SpinBench             | Visual      | Perspective/rotation   | Egocentric/allocentric, 51 subtypes        | 2509.25390    |
| SpatialViz-Bench      | Visual      | 4 visualization skills | 12 tasks: rotation, folding, animation     | 2507.07610    |
| DynaSolidGeo          | Multimodal  | Solid geometry         | Dynamic instance gen, process evaluation   | 2510.22340    |
| EarthSpatialBench     | Geo-visual  | Distance, topology     | Polygons, polylines, quantitative tasks    | 2602.15918    |
| CityCube              | Visual      | Cross-view             | Urban, rotation/orbit, 5 cognitive dims    | 2601.14339    |
| SIRI-Bench            | Video       | 3D math, perception    | Video-based, multi-step, automatic gen     | 2506.14512    |
| Spatial-DISE          | Visual      | DISE quadrants         | Multi-view, multi-step reasoning           | 2510.13394    |
| SpatialWorld          | Interactive | POMDP, planning        | Text-action, 8 env backends, TSR/effic.    | 2606.09669    |
| EvoEmpirBench         | Agent       | Dynamic, experience    | Long-horizon, local obs, experience mem    | 2509.12718    |
| SpatialText           | Text-only   | 5-level, dual source   | Human+synthetic, mental modeling           | 2603.03002    |
| SSI-Bench             | Visual      | Constrained manifold   | Real 3D structures, ranking, physics       | 2602.07864    |
| Spatial4D-Bench       | Video+img   | 6 cognitive domains    | ~40k QA, spatiotemporal, physical law      | 2601.00092    |

These benchmarks collectively constitute the state-of-the-art in evaluating and dissecting spatial reasoning in AI systems and inform the specification of next-generation, spatially-aware models and agentic architectures.

Source: https://www.emergentmind.com/topics/spatial-reasoning-benchmarks