Consistent Spatial Imagination

• Consistent spatial imagination is the capacity to accurately manipulate spatial structures using operations such as folds, rotations, and reflections while maintaining coherence.
• Benchmark evaluations reveal significant challenges in state tracking and transformation consistency, with current models showing varied performance across video, image, and text modalities.
• Emerging architectures leverage intermediate state externalization, 3D-centered interfaces, and generative feedback, offering promising directions for enhancing spatial reasoning in complex tasks.

Consistent spatial imagination denotes the ability to imagine, transform, and manipulate spatial structure while preserving physical and mathematical coherence across intermediate mental states. In its most explicit benchmark definition, it is the capability to apply folds, reflections, rotations, and hole-punches so that imagined unfolding returns the correct hole configuration on the original sheet. Closely related work operationalizes the same general capacity as imaginative perception, implicit spatial world modeling, orthographic mental simulation, egocentric cognitive mapping, and active mental imagery for unseen viewpoints, path tracing, multi-view counting, navigation, and dynamic spatial reasoning (Yilmaz et al., 22 Feb 2026, Bigverdi et al., 2 Jun 2026, Zhan et al., 8 Mar 2026, Cao et al., 1 Dec 2025).

1. Conceptual scope

In the benchmark literature, spatial visualization is defined as the human-level ability “to imagine, transform, and manipulate the spatial characteristics of objects and actions.” “MentalBlackboard” sharpens this into consistent spatial imagination: a sequence-sensitive competence in which every geometric operation remains coherent with every previous one, so that the final imagined state is not merely plausible but transformation-consistent under reversal and composition (Yilmaz et al., 22 Feb 2026).

Adjacent papers broaden the same idea beyond paper-folding. “Imaginative Perception Tokens” treats imagination as an intermediate perceptual representation for what a model would perceive under alternative spatial configurations; “OmniView-Space” frames consistency as query-aligned re-anchoring into camera-, object-, or direction-centric ego frames; “SpatialDreamer” defines active mental imagery as a closed loop of exploration, world-model rendering, and evidence-grounded reasoning; and “3ViewSense” identifies a “view-consistent spatial interface” as the missing bridge between 2D perception and 3D logic (Bigverdi et al., 2 Jun 2026, Li et al., 1 Jul 2026, Cao et al., 8 Dec 2025, Zhan et al., 8 Mar 2026).

A recurrent distinction in this literature is between static recognition and state tracking. Several systems argue that object recognition and high-level planning can be delegated to textual reasoning, while geometry-sensitive state transformation must remain grounded in visual latents, orthographic projections, cognitive maps, or simulator-generated views. This suggests that consistent spatial imagination is not identical to generic multimodal reasoning; it is a constraint on how internal representations evolve under transformations rather than a synonym for answering spatial questions correctly (Li et al., 19 Apr 2026, Chen et al., 21 Oct 2025).

2. Formal structure of consistency

The clearest formalization appears in paper-folding and hole-punching. “MentalBlackboard” models the sheet as a planar domain $\Omega \subset \mathbb{R}^2$, with holes as points $h \in \Omega$ augmented by attributes such as size $s$, shape $\sigma$, and orientation $\alpha$. Representative operators include a horizontal fold $T_{H1}([x,y])=[x,-y]$, a vertical fold $T_{V1}([x,y])=[-x,y]$, a diagonal fold $T_{D1}([x,y])=[y,x]$, and a global rotation

$$T_R(\theta)= \begin{pmatrix} \cos\theta & -\sin\theta\ \sin\theta & \cos\theta \end{pmatrix}.$$

Unfolding is defined as inverse folding, and compositions are explicit: $T_{\rm composed}=T_R(\theta)\circ T_F$ and $h \in \Omega$0. In this setting, consistency means choosing each inverse operator in the correct reverse order and in the correct rotated reference frame (Yilmaz et al., 22 Feb 2026).

Other work generalizes consistency from planar transforms to viewpoint and frame transforms. “OmniView-Space” defines a query-aligned spatial state $h \in \Omega$1, where $h \in \Omega$2 is a rendered BEV cognitive map and $h \in \Omega$3 is a textual spatial graph in the ego frame. Re-anchoring is performed by

$h \in \Omega$4

or equivalently through a $h \in \Omega$5 ego transform $h \in \Omega$6. “MILO” pursues a related objective through Relative Positional Encoding, adding flattened relative camera-pose transforms to 2D patch embeddings so that inter-frame geometry is encoded relationally rather than through absolute coordinates (Li et al., 1 Jul 2026, Cao et al., 1 Dec 2025).

A more abstract formulation comes from hippocampal replay. Dabaghian represents place-cell assemblies as a simplicial complex and imposes path invariance through zero holonomy: for any closed simplicial loop $h \in \Omega$7, the transfer matrix must satisfy

$h \in \Omega$8

The corresponding discrete curvatures $h \in \Omega$9 must vanish. Here, spatial consistency is not answer accuracy but the requirement that a closed replay trajectory returns the network to exactly the same population-activity state. This is a different domain, but it supplies a mathematically precise notion of why internally generated spatial trajectories may drift unless explicit flatness constraints are imposed (Dabaghian, 2015).

3. Benchmarks and empirical diagnosis

The current benchmark record indicates that consistent spatial imagination remains weak in contemporary VLMs. On MentalBlackboard prediction, using 900 video, 315 image, and 315 text queries, the best model o3 achieved 8.0% exact-match on video, 10.5% on image, and 25.1% on text. Claude Opus 4.1 reached 1.2%, 2.5%, and 17.5% respectively. On planning, Claude Opus 4.1 obtained 10% exact-match and o3 9.5%, while missing-hole errors accounted for more than 70% of errors. On generalization, which “does not require spatial visualization but transfers spatial data,” o3 reached 71.6%, while size and shape transferred reliably but location and direction remained difficult. Reported failure modes included incorrect folding depth leading to extra holes, symmetry miscalculation when layers occlude, and rotational reorientation being ignored, with true unfolding accuracy dropping from approximately 90% without rotation to under 40% with 90°–270° rotations (Yilmaz et al., 22 Feb 2026).

“3ViewSense” diagnoses a related “spatial intelligence gap.” A 4-layer probe trained on frozen image features reached 55.8% accuracy on the Block-Counting subset of OrthoMind-3D, while providing explicit natural-language three-view hints boosted absolute accuracy by over 30% for models such as Gemini-3-Pro. After Stage I+II SFT, 3ViewSense-4B-sft reached 33.4% on in-domain block counting, and with RL refinement under strict reward, 3ViewSense-4B-rl-strict reached 95.0% on block counting and 98.7% on object counting. On transfer tasks, accuracy on ViewSpatial rose from 33.5% to 72.9%, and the generated traces were reported to be 10–20× shorter (Zhan et al., 8 Mar 2026).

The same literature also shows that more imagination is not always better. “AVIC” reports that 54% of SAT questions are answered correctly without any imagination, only approximately 14% genuinely need imagination, and excessive fixed imagination of more than 4 views degrades accuracy. On SAT-Real, AVIC improved average accuracy from 66.6% for always-on MindJourney to 74.6% while reducing average world-model calls from 12.3 to 0.64. On the GPT-4.1 setting, GPT-4.1+AVIC reached 79.3% with 0.73 world-model calls, versus 77.3% and 12.3 calls for GPT-4.1+MindJourney. A complementary diagnosis from the LoopNav benchmark shows that short-context world models collapse on loop closure: Oasis and DIAMOND often “fade” into blur after approximately 30 frames, and even a 32-frame context does not suffice for loops beyond 50 steps (Yu et al., 9 Feb 2026, Lian et al., 29 May 2025).

4. Architectures for inducing consistent imagination

One major design line externalizes intermediate spatial states. “Imaginative Perception Tokens” introduces a two-stage decomposition

$s$0

implemented in BAGEL with semantic $s$1 tokens and generative $s$2 tokens. Training uses $s$3. The generated imagination is decoded, re-encoded, appended to context, and then attended to when answering. SpatialImaginer follows a comparable principle but splits reasoning into text planning $s$4, optional visual latent generation $s$5, and textual deduction, optimized with $s$6, with $s$7. It also uses difficulty-aware routing and a two-stage verification process, with overall retention of approximately 48.2% for synthesized visual states (Bigverdi et al., 2 Jun 2026, Li et al., 19 Apr 2026).

A second design line imposes explicit scene-centered or 3D-centered interfaces. OmniView-Space constructs Multi-Perspective Spatial Mapping, returning both a BEV cognitive map and a textual spatial graph in the relevant ego frame. It then trains a tool-use policy with GRPO and distills approximately 26 K MPSM-guided trajectories into a standalone model; map-alignment reward adds 2.3 points in average accuracy. 3DThinker instead inserts a bank of $s$8 learnable “3D mental imagery” tokens into chain-of-thought, aligns them to VGGT features with $s$9, and then refines them with GRPO. On MindCube-Tiny with a Qwen2.5-VL-3B backbone, performance rose from 33.2% to 62.7% after Stage 1 and to 75.2% after Stage 1+2 (Li et al., 1 Jul 2026, Chen et al., 21 Oct 2025).

A third line uses generative teachers or generative feedback to internalize imagination. MILO combines Relative Positional Encoding with a diffusion loss

$\sigma$0

and trains on GeoGen, which contains 2,241 videos and 267,827 observation–action–outcome triplets. World2VLM uses a view-consistent world model as a training-time teacher, optimizing

$\sigma$1

with 100 K total SFT samples and a subsequent task-aware GRPO stage. On the four-benchmark comparison, World2VLM-GRPO with SVC reached 72.67 on SAT-Real, 59.20 on SAT-Synth, 41.55 on VSI-Bench, and 34.86 on MindCube, all above the base Qwen2.5-VL-7B model (Cao et al., 1 Dec 2025, Zhang et al., 29 Apr 2026).

5. Agentic, embodied, and generative instantiations

In agentic systems, consistent spatial imagination becomes a controlled resource. AVIC gates world-model use through a sufficiency score $\sigma$2 and a scaling parameter $\sigma$3, explicitly deciding whether to skip imagination or how much to invoke. SpatialDreamer turns imagination into a closed-loop policy, using Stable Virtual Camera for novel views and GeoPO for tree-structured sampling with step-level rewards and a geometric penalty on redundant or conflicting actions; it reached 93.9 on SAT-Real, 92.5 on SAT-Synth, and 84.9 on MindCube-Tiny. Astra couples Astra-VL with Astra-WM, where view consistency tuning improved pose-consistency from 9% to 70% and content-consistency from 0.35 to 0.56, and the full agentic pipeline improved Qwen3-VL-8B from 29.8 to 38.8 on MMSI-All and from 36.8 to 42.7 on MindCube (Yu et al., 9 Feb 2026, Cao et al., 8 Dec 2025, Zhu et al., 4 Jun 2026).

Embodied navigation work adopts similar principles but under action constraints. ImagineNav++ predicts six candidate future poses from panoramic subviews, synthesizes six imagined future images, and asks a VLM to perform best-view selection while maintaining a hierarchical keyframe memory that contains on average only approximately 20 images even over a 500-step trajectory. I-MP defines imagined spatial states as fixed points of an energy-gradient contraction mapping and then approaches them through real-time torque control. In simulation and hardware, it reported mean absolute error below 2 mm over 50 Hz planning, and over 43,200 simulations plus 216 hardware runs it outperformed probability-based planners by +69.5% success, model-based by +88.3%, and simulation-based by +86.8% (Wang et al., 19 Dec 2025, Wang et al., 21 Sep 2025).

The same theme also appears in image generation. SmartSpatial makes a diffusion UNet 3D-aware through depth information injection and cross-attention control, and evaluates spatial fidelity with Object Recognition, Object Proximity, and Spatial-Relation metrics. On SpatialPrompts, SmartSpatial reached IoU=0.434 and OP+OR=0.604, compared with SD+AG at IoU=0.223 and OP+OR=0.563; on VISOR, IoU rose from 0.150 to 0.324. This does not address reasoning in the same sense as spatial QA or navigation, but it shows that consistent spatial imagination can also be formalized as faithful 3D arrangement under generation-time constraints (Huang et al., 1 Jan 2025).

6. Failure modes, debates, and research directions

Across papers, the dominant diagnosis is that text-dominant reasoning is poorly matched to geometry-sensitive state tracking. MentalBlackboard attributes failures to a sequential reasoning bottleneck, insufficient visuospatial working memory, lack of physical orientation awareness, and selection-bias in multiple-choice training. IPT reports that textual chain-of-thought often underperforms label-only supervision and can substantially degrade performance, which it interprets as a modality mismatch when spatial computation is forced through language. SpatiaLite reaches a similar conclusion, arguing that advanced VLMs predominantly rely on linguistic representations and show rapid token growth as transformation complexity increases; on Qwen-2.5-VL-7B, the original model scored 4.3 on Mental Rotation, 12.5 on Cube Rolling, and 20.1 on Rubik’s Cube, while the two-stage ID + RD setting reached 7.5, 42.3, and 44.7 respectively (Yilmaz et al., 22 Feb 2026, Bigverdi et al., 2 Jun 2026, Lian et al., 16 Nov 2025).

A second recurring limitation is drift under long horizons or noisy teachers. LoopNav shows that short-window baselines lose coherence on long return segments, suggesting that a true memory with explicit read/write and coordinate indexing is still missing. World2VLM notes that teacher errors such as warping, drift, and hallucinated objects upper-bound student performance, especially under large motions and in cluttered scenes. Astra likewise identifies simulator hallucinations, policy mis-queries, and sparse reward as ongoing limits (Lian et al., 29 May 2025, Zhang et al., 29 Apr 2026, Zhu et al., 4 Jun 2026).

The main proposed remedies are structurally convergent. MentalBlackboard recommends group-equivariant architectures, explicit transform modules, memory-augmented attention, hybrid neuro-symbolic pipelines, and curriculum pretraining on multi-step fold tasks. SpatialImaginer proposes end-to-end differentiable routing and explicit 3D latent modules. OmniView-Space emphasizes query-aligned ego-frame evidence and cognitive-map distillation. MILO favors relative rather than absolute positional encodings, and SmartSpatial shows that depth-grounded conditioning can improve spatial fidelity in a generative setting. Taken together, these proposals suggest that future progress is likely to depend less on longer verbal chains and more on stable intermediate spatial states, explicit frame transforms, and memory mechanisms that preserve consistency across composition, re-anchoring, and long-horizon rollout (Yilmaz et al., 22 Feb 2026, Li et al., 19 Apr 2026, Li et al., 1 Jul 2026, Cao et al., 1 Dec 2025, Huang et al., 1 Jan 2025).