---
title: 'Maze Test: Benchmark for Spatial Reasoning'
url: https://www.emergentmind.com/topics/maze-test
type: topic
---

# Maze Test: Benchmark for Spatial Reasoning

Maze Test denotes a family of maze-solving evaluations in which an agent must infer, remember, or plan over spatial structure under controlled conditions. In current AI research, the term is used most explicitly for first-person textual navigation tasks intended to probe spatial awareness, perspective-taking, goal-directed behavior, and temporal sequencing in large language models [2508.16705]. Closely related benchmarks span pixel-art visual mazes [2603.26839], coordinate-only navigation [2507.20395], randomized 3D first-person environments for long-term memory [2210.13383], locally observable dynamic grids [2509.12718], and stateful textual worlds designed for world-modeling agents [2607.00627]. *This suggests* that the Maze Test is better understood as a benchmark paradigm than as a single fixed task: the common core is a maze-structured environment whose solution requires sequential spatial reasoning, while the observation model, action interface, and evaluation target vary substantially.

## 1. Historical and formal foundations

Classical maze research framed the problem as finding the shortest distance to the goal from any position in a 2D maze represented as a square matrix. One influential neural approach used the Cellular Simultaneous Recurrent Neural Network (CSRN), with obstacle and goal indicators as external inputs, neighbor connectivity in the four cardinal directions, recurrent nodes, and Extended Kalman Filter training; clustering maze cells and feeding the cluster number as a third external input improved correctness from 67.7% to 82.9% and goodness from 42.9% to 53.2% on random \(12 \times 12\) mazes [0908.0939]. In this formulation, the maze test is primarily a supervised learning problem over shortest-distance and direction labels.

Other formalisms alter the target computation itself. Maze Dash, presented as a map exploration problem, requires the agent to visit all non-obstacle cells exactly once while moving only in cardinal directions and continuing straight until forced to turn. The movement path is written as \( \pi : \mathcal{N} \rightarrow \mathcal{N} \times \mathcal{N} \), and the resulting task is treated as a variant of the Hamiltonian Path problem [2005.14156]. A quantum formulation, restricted to two-dimensional perfect mazes with no closed loops, converts the task into a Quantum Search Problem by encoding all length-\(n\) paths in \(2n\) qubits, applying a fitness operator, and using iterative Grover search with a thresholding oracle to approach the optimum solution [1312.4116].

The maze test has also been realized in physical and biological substrates. A survey of twelve laboratory maze solvers argues that they all implement the 1961 Lee algorithm by first developing a gradient of values showing distance from each site to the destination and then tracing a path along that gradient through electrical, chemical, fluidic, thermal, or biological processes [1601.04672]. At the smallest scale, engineered bacteria have been used to solve chemically generated \(2 \times 2\) maze problems: four chemical inputs encode the maze, six genetic logic circuits distributed across six cell populations compute the solution, and fluorescent proteins indicate the valid path, yielding three solvable and thirteen unsolvable cases among the sixteen possible mazes [2106.09882]. These variants show that “maze test” can refer not only to AI benchmarking but also to a broader class of structured pathfinding evaluations across computational paradigms.

## 2. Task formulations and sensory interfaces

Modern maze tests differ most sharply in how the maze is presented and how an answer must be expressed. MazeBench uses 110 procedurally generated pixel-art maze images across nine controlled groups. The model must return a strict JSON containing grid size, detected start and goal positions, reachability, shortest path length, and the actual sequence of moves \(U/D/L/R\), and a trial is correct only if all components are correct [2603.26839]. By contrast, the consciousness-oriented Maze Test presents a bird’s-eye textual rendering with entrances, exits, wall positions, and numbered zones, but requires a first-person navigation plan composed of “walk forward” and “turn left/right” actions [2508.16705].

MazeEval removes visual input entirely. Models receive only the current \((x,y)\) position, goal position, distance-to-wall information for the four cardinal directions, and the full travel history, and they must act through a function-calling interface `move(direction)` with directions in \(\{\text{north, south, east, west}\}\) [2507.20395]. Memory Maze moves in the opposite direction, using randomized 3D mazes viewed from a first-person perspective, where the agent must repeatedly find prompted colored objects while remembering room connectivity, object locations, and its own pose over episodes of 1000 to 4000 timesteps [2210.13383]. AGI Maze uses low-bandwidth local textual observations under partial observability, exposing only local consequences of actions and forcing the agent to infer latent structure such as walls, keys, rivers, pits, and hidden mechanics [2607.00627].

| Benchmark or test | Observation regime | Required output or interaction |
|---|---|---|
| MazeBench | Pixel-art image | Strict JSON with grid, start/goal, reachability, shortest path |
| Maze Test | Textual bird’s-eye maze | First-person sequence of turns and forward moves |
| MazeEval | Coordinates, wall distances, history | `move(direction)` function calls |
| Memory Maze | First-person 3D images | Repeated target-finding behavior |
| AGI Maze | Local textual observations | Stepwise actions under a step budget |

Additional variants adapt maze tests to more situated settings. Memory-Maze for Visual Language Navigation uses a maze-like virtual environment in CARLA and natural-language route instructions given from human memory, including stutters, errors, omissions, and ambiguous landmark descriptions; the task is to execute the instruction with robot-control APIs rather than to emit a symbolic path [2405.07060]. EvoEmpirBench uses a \(9 \times 9\) locally observable grid in which the agent sees only local surroundings, while monsters, coins, and tool use dynamically modify the environment after each action [2509.12718]. *This suggests* that the representational interface is not ancillary: it is itself a major experimental variable in maze-based evaluation.

## 3. Metrics and diagnostic decomposition

Maze tests have gradually shifted from single success indicators toward multi-axis diagnostics. MazeBench already enforces a stringent all-or-nothing criterion: a model receives no partial credit if any element of the required JSON is wrong [2603.26839]. The consciousness-oriented Maze Test explicitly separates fully coherent solutions from early competence by defining Complete Path Accuracy as the fraction of test cases with a fully correct solution and Partial Path Accuracy as the average fraction of correct consecutive steps before the first error. In the reported results, Gemini 2.0 Pro reached 52.9% Complete Path Accuracy, while DeepSeek-R1 reached 80.5% Partial Path Accuracy, and the gap between these metrics was interpreted as difficulty in maintaining a coherent self-model throughout the full trajectory [2508.16705].

Other benchmarks decompose failure much more finely. “Lost in Aggregation” organizes maze navigation into Fine, Meso, and Macro levels corresponding to local passability, junction topology, and global goal direction, and reports Success Rate, Valid-Move Ratio, First Error Step, wall-collision rate, teleport rates, junction accuracy, dead-end entry, backtrack-success rate, progress rate, and direction drift rate [2606.22219]. EvoEmpirBench evaluates maze navigation with eight main metrics—Suc.Rate, A.Score, A.Steps, A.Explor, A.Gold, Rem.HP, A.kills, and A.Barr.—because the task couples exploration, resource management, survival, and navigation in a dynamic partially observable world [2509.12718]. Memory Maze augments online return with offline probing: frozen representations are tested for global wall-layout prediction accuracy and object-location mean squared error, so that memory can be measured separately from overt behavior [2210.13383].

The proliferation of metrics reflects a methodological shift. *A plausible implication is* that a maze test is increasingly treated as a structured probe of multiple latent competences—perception, state estimation, memory, sequential consistency, and action selection—rather than as a single pathfinding score.

## 4. Observed strategies and failure modes in current models

A central contemporary result is that high maze accuracy does not necessarily imply human-like spatial understanding. MazeBench reports that GPT-5.4 (medium) solves 91% of the core set and Gemini 3.1 Pro 79%, yet qualitative traces show a common two-stage strategy across models: image-to-grid translation followed by serial path enumeration, effectively breadth-first search in prose. Token consumption ranges from about 1,700 to over 22,000 tokens per maze, all configurations without increased reasoning budget fall to 2–12% accuracy, and on ultra-hard \(20 \times 20\) mazes models often fail by exceeding token limits rather than by misperceiving the maze [2603.26839]. The same study shows that Claude Sonnet 4.6 rises from 6/100 on images to 80/100 when given the correct text grid, isolating weak pixel-to-grid extraction from downstream search.

The diagnostic picture sharpens when navigation is decomposed across scales. “Lost in Aggregation” finds that end-to-end one-shot navigation collapses to near zero by \(10 \times 10\) for GPT-4o, DeepSeek-V3, and Llama-3.3-70B, even though isolated Fine, Meso, and Macro probes remain at 30–75% beyond that size. A multi-hot first-error analysis attributes 59% of first errors to Meso junction choices, 39% to Fine perception, and about 1% to Macro direction, indicating that the main barrier is cross-scale aggregation rather than global goal orientation [2606.22219]. Hierarchical route planning, in which a deterministic walker handles corridor execution and the LLM is queried only at junctions with explicit cell-type prompts, lifts GPT-4o success by up to 92 points at mid sizes, although the scaling wall reappears by \(30 \times 30\).

MazeEval isolates another recurrent failure mode: looping. In its coordinate-only function-calling setting, OpenAI’s O3 achieves perfect navigation up to \(30 \times 30\) in both English and Icelandic, while other models fail catastrophically beyond roughly \(9 \times 9\), with 100% of failures attributed to excessive looping behavior in which a cell is revisited at least 10 times [2507.20395]. The same benchmark reports significant degradation in Icelandic, with models solving mazes 3–4 sizes smaller than in English, which was taken as evidence that spatial performance depends on linguistic training patterns rather than on a language-agnostic mechanism. AGI Maze reaches a related conclusion from partially observable textual environments: several vanilla LLMs fail to represent mazes internally at inference time, and a baseline planning agent that uses its message history as explicit working memory improves performance but still does not reliably solve even small mazes within a human-adequate step budget [2607.00627].

These results jointly challenge a common assumption that strong top-line performance on a maze benchmark straightforwardly demonstrates planning competence. In several settings, the dominant mechanisms are token-level search, brittle execution, poor junction integration, or looping, rather than stable world modeling or holistic spatial planning.

## 5. Memory, partial observability, and dynamic environments

Maze tests have become a standard instrument for studying memory under partial observability. Memory Maze was designed specifically to evaluate long-term memory in agents that navigate randomized 3D mazes from first-person sensory input. Agents must remember the positions of relevant objects, localize themselves by integrating information over time, and learn room connectivity; humans establish a strong baseline, current algorithms benefit from truncated backpropagation through time, and agents succeed on small mazes but fall short of human performance on large mazes [2210.13383]. The benchmark also includes an offline dataset of 30,000 trajectories each for \(9 \times 9\) and \(15 \times 15\) mazes and an offline probing protocol, making it possible to separate behavioral success from representational retention.

Dynamic variants intensify the requirement for state tracking. EvoEmpirBench’s maze navigation task takes place in a \(9 \times 9\) grid where only local surroundings are visible, the map begins largely unknown, and difficulty scales from a static coin-collection setting to moving monsters and four interactive items—Pickaxe, Iron Sword, Magnet, and Key—whose use changes the environment [2509.12718]. The associated Agent-ExpVer framework records trajectories \(\mathcal{H}_{0:T}\), summarizes them into subjective experience memory, validates them through later episodes, and promotes successful abstractions into a truth memory. Reported gains such as +5.56% success rate and +243 average score for GPT-4.1 indicate that iterative experience verification can improve adaptive planning, although a gap to human performance remains.

AGI Maze explicitly targets world-modeling rather than static reasoning. Its tutorial, training, classic, extended, and hidden regimes combine partial observability with items, rivers, pits, and unknown mechanics, and the framework is designed so that the agent never sees the full map except in tutorials [2607.00627]. *This suggests* that, within maze-based evaluation, statefulness and hidden-state inference are increasingly treated as the decisive features separating pattern completion from agentic reasoning. A related result appears in CogNGen, a cognitive architecture combining predictive processing with hyperdimensional memory: across Mini-GridWorld maze-learning tasks, CogNGen matched deep reinforcement learning on navigation and planning tasks and exceeded deep RL on the memory task, supporting the view that explicit memory machinery can be decisive in maze settings that require recalling earlier cues [2204.00619].

Maze tests have also been adapted to real-world assistive scenarios. Memory-Maze for guiding blind people collects route instructions both from sighted passersby onsite and from online annotators, finding that onsite instructions are longer and use more varied wording. The proposed Visual Language Navigation model parses instructions with an LLM, generates Python code for robot control, and outperforms NavGPT on this benchmark, whereas the state-of-the-art baseline records zero success across reported conditions [2405.07060]. Here the maze test is not merely a graph-search problem but a model of instruction following under human memory imperfections.

## 6. Benchmark generation, generalization, and broader implications

The growth of maze testing has been supported by explicit dataset infrastructure. The `maze-dataset` library provides configurable generation algorithms including randomized depth-first search, Wilson’s algorithm, percolation, and DFS-plus-percolation, along with filtering by path length or start-end distance, invertible rasterized and tokenized outputs, metadata retention, and A\(^*\)-based shortest-path ground truth [2309.10498]. This enables systematic studies of distributional shift by varying generation algorithm, grid size, wall structure, or output modality. AMaze extends this idea toward human-guided generalization studies by letting experimenters control clues, lures, traps, and maze complexity, and by quantifying “surprisingness” and “deceptiveness”; in its proof-of-concept experiments, scaffolding and interactive training outperform one-shot training, with median gains ranging from 50% to 100% depending on the training regime, algorithm, and generalization metric [2411.13072].

Recent work has also used mazes to probe reasoning beyond standard language models. VFScale, a verifier-free test-time scalable diffusion model, trains on maze sizes up to \(6 \times 6\) and solves 88% of much larger \(15 \times 15\) mazes when combined with hybrid Monte Carlo Tree Search at inference, whereas a standard diffusion model drops to about 6% on the same setting [2502.01989]. VR-Bench uses 7,920 procedurally generated videos across five maze types to evaluate whether video models can reason through video generation; the study reports that supervised fine-tuning elicits reasoning ability efficiently and that diverse test-time sampling improves reasoning reliability by 10–20% [2511.15065]. These results place maze tests within a broader landscape of test-time scaling, multimodal reasoning, and search-guided inference.

A recurring conclusion across the literature is that maze scores are highly sensitive to representation and evaluation protocol. Structured coordinate text can far outperform images for language models [2606.22219]; correct text grids can transform apparent reasoning deficits into strong downstream performance [2603.26839]; and language choice can shift maximal solvable maze size by several steps [2507.20395]. Another recurring conclusion is that success on a maze test does not by itself resolve larger claims about cognition. The consciousness-oriented Maze Test argues that reasoning-enhanced LLMs exhibit progress in spatial awareness, perspective-taking, and goal-directed behavior, yet still lack the integrated, persistent self-awareness implied by a coherent self-model across long solutions [2508.16705]. In that sense, the maze test functions not as a final verdict on intelligence or consciousness, but as a tightly controlled instrument for locating where sequential spatial competence begins, where it breaks, and what auxiliary machinery—memory, search, delegation, or explicit world models—is required to extend it.

Source: https://www.emergentmind.com/topics/maze-test