Thought Space in Spatial Communication

• Thought Space is a representational workspace where agents reconcile individual, egocentric observations into a shared, stable allocentric model.
• It involves establishing anchor correspondences, integrating relational constraints, and validating global geometric consistency during communication.
• Benchmarks like COSMIC highlight the challenges of maintaining spatial thought space, motivating improvements such as explicit spatial memory and repair protocols.

Searching arXiv for the primary paper and a few related “thought space” papers to ground the article. Thought space denotes a representational workspace in which intermediate reasoning is organized, updated, and made actionable. In collaborative spatial communication, it is the allocentric, jointly maintained mental model that two agents construct by integrating their partial, egocentric observations through language (Sikarwar et al., 28 Mar 2026). In this formulation, thought space is not a metaphor for generic reasoning alone: it is the concrete process of anchoring references across views, integrating relational constraints, and validating global geometric consistency. Recent arXiv literature extends the same term to tree-structured response spaces, continuous latent manifolds, multimodal reasoning substrates, and latent multiagent communication channels, but the spatial case is distinctive because it couples dialogue, perspective-taking, and map building within a single benchmarked setting (Wang et al., 9 Apr 2025).

1. Definition and theoretical core

In the spatial setting, thought space is the shared allocentric model that emerges when agents reconcile viewpoint-dependent observations into a scene-centered layout (Sikarwar et al., 28 Mar 2026). The paper distinguishes egocentric frames, which are agent-centered and expressed as front, left, right, and behind relative to each agent’s camera, from allocentric frames, which are scene-centered top-down layouts. Shared anchor objects are landmarks visible in both views; they enable perspective alignment and cross-view reference. Relational reasoning then extends beyond anchoring to compare distances and directions of objects that may be visible to only one agent. Cognitive mapping is the further requirement that both views be reconciled into a coherent allocentric layout that remains precise and stable across dialogue turns.

This decomposition gives thought space a staged structure. First, agents must establish anchor correspondences across views. Second, they must integrate relational constraints such as distances and directions. Third, they must test whether the resulting configuration is globally consistent. The paper therefore treats thought space not as free-form deliberation, but as a constrained spatial inference process whose success depends on reference grounding, perspective-taking, and consistency maintenance throughout the dialogue (Sikarwar et al., 28 Mar 2026).

The benchmark’s implied geometry makes this explicit. Egocentric-to-world mapping is written as

$p_{world} = R_i p_{ego,i} + t_i,$

where $R_i$ and $t_i$ are the rotation and translation of agent $i$’s camera pose. Anchor alignment is described through attribute similarity, and perspective transfer for direction judgments is expressed as

$\theta_m^A \approx \theta_m^H + \Delta \psi,$

where $\Delta \psi$ is the yaw difference inferred from anchored correspondences. At the global level, allocentric integration is captured by the consistency objective

$$\mathcal{L}_{consistency} = \sum_{(i,j) \in E} \| (R_i p_i + t_i) - (R_j p_j + t_j) \|^2,$$

which formalizes the requirement that local descriptions cohere into a single scene-level layout (Sikarwar et al., 28 Mar 2026).

2. COSMIC benchmark and experimental setting

To study language-mediated spatial integration systematically, the paper introduces COSMIC, a benchmark for Collaborative Spatial Communication (Sikarwar et al., 28 Mar 2026). COSMIC contains 899 procedurally generated, photorealistic indoor scenes from Infinigen Indoors and 1,250 question-answer instances. The scenes span living rooms, bedrooms, bathrooms, kitchens, and dining rooms, include more than 23 distinct object categories, and vary in clutter, with a mean of 17.71 objects per scene and a range of 6–31. Each of the five tasks contains 250 multiple-choice questions, each drawn from a unique scene. Questions are paraphrased for linguistic diversity and filtered to ensure cross-view necessity.

The environment contains two static MLLM agents, an Answerer and a Helper, that observe complementary egocentric RGB images of the same indoor room. The Answerer receives the question and options, and in the cognitive mapping task also receives the candidate map image. The Helper sees only its own image. The visibility sets are designed so that $O_A \cap O_H \neq \emptyset$ for anchors and $O_A \cup O_H \approx O$ for global coverage. Dialogue proceeds in alternating turns for up to 10 rounds, the Answerer may terminate early by sending “TERMINATE,” and after dialogue the Answerer outputs a single final option (Sikarwar et al., 28 Mar 2026).

The five tasks are summarized below.

Task	Core requirement	Evaluated ability
Anchor Recognition	Identify which object appears in both views	Cross-view anchor grounding
Global Counting	Count total instances across both views	Deduplication and inclusion of exclusive objects
Relative Distance	Find closest or farthest object to a target anchor	Cross-view metric comparison
Relative Direction	Infer egocentric direction of an unseen target object	Allocentric-to-egocentric perspective-taking
Cognitive Mapping	Judge whether a top-down map matches the room layout	Allocentric map construction

The task design deliberately escalates the demands placed on thought space. Anchor Recognition tests whether a shared referential substrate can be established. Counting and Relative Distance require the integration of exclusive and shared observations under deduplication and metric comparison. Relative Direction requires the Helper’s allocentric description to be transformed into the Answerer’s egocentric frame, using eight discretized egocentric orientations at $45^\circ$ increments with bearings assigned within $R_i$0. Cognitive Mapping is the strongest test: distractors swap positions of Helper-exclusive objects so that success depends on global layout consistency rather than object identity (Sikarwar et al., 28 Mar 2026).

3. Performance hierarchy in spatial communication

COSMIC reveals a capability hierarchy in which performance declines from anchors to relations to maps (Sikarwar et al., 28 Mar 2026). Anchor Recognition is the easiest task. Relational reasoning, including counting, distance, and direction, degrades substantially. Cognitive Mapping is near chance even for frontier models. The benchmark therefore shows that current MLLMs can partially establish a shared thought space at the anchor level, but struggle to consolidate it into a robust allocentric model.

On the Cosmic-Human subset of 250 questions, humans achieve 95.22% average accuracy, while the best model, Gemini-3-Pro-Thinking, reaches 71.82%, leaving a gap greater than 23% (Sikarwar et al., 28 Mar 2026). Humans are near ceiling on Relative Direction and Cognitive Mapping, with 94% on mapping, whereas frontier models are near the 50% chance baseline on mapping. This is the benchmark’s central empirical result: language-mediated perspective alignment is not enough, by itself, to yield stable map construction.

Per-task results make the hierarchy more concrete. On Anchor Recognition, Gemini-3-Pro-Thinking reaches 91.99%, GPT-5.2-Thinking 76.83%, and Qwen3-VL-32B 66.59%. On Relative Direction, Gemini-3-Pro-Thinking reaches 46.21% and GPT-5.2-Thinking 51.61%, showing that allocentric-to-egocentric perspective-taking is a bottleneck. On Cognitive Mapping, frontier models remain near chance (Sikarwar et al., 28 Mar 2026).

The paper also compares closed-source and open-source models. Gemini-3-Pro-Thinking and Gemini-3-Flash-Thinking lead overall at approximately 71.64% and 67.88%. Qwen3-VL-32B is the strongest open-source model at 52.47%, outperforming InternVL3.5-38B at 39.45% and Gemma-3-27B at 36.22%. Scale helps within Qwen3-VL, from 40.08% at 8B to 52.47% at 32B, but not consistently for Gemma-3 and InternVL3.5 (Sikarwar et al., 28 Mar 2026).

“Thinking” capability improves only part of the stack. The paper reports gains on Anchor Recognition and Relative Distance but not on Relative Direction or Cognitive Mapping. For Anchor Recognition, Gemini-3-Flash improves from 77.78% to 87.19%, and GPT-5.2 from 64.34% to 76.83%. For Relative Distance, Gemini-3-Flash improves from 76.81% to 88.00%, and GPT-5.2 from 58.01% to 74.01%. The absence of gains on direction and mapping suggests that the limiting factor is geometric reconciliation rather than deliberation depth alone (Sikarwar et al., 28 Mar 2026).

A further comparison isolates the cost of communication itself. When both views are given to a single model without dialogue, performance is higher than in the two-agent setting. Qwen3-VL-32B rises from 52.62% in the two-agent condition to 64.62% in the single-agent condition, and Gemini-3-Flash-Thinking rises from 68.07% to 78.38% (Sikarwar et al., 28 Mar 2026). This indicates that natural-language coordination and the maintenance of a shared thought space impose measurable difficulty beyond raw perception and reasoning.

4. Dialogue dynamics, repair, and error propagation

The benchmark does not only measure final accuracy; it also analyzes how thought space evolves during interaction. Human conversations become more precise as partners align on a shared spatial understanding, whereas MLLMs keep exploring without converging (Sikarwar et al., 28 Mar 2026). Humans rapidly converge on anchors and then taper object mentions per turn, which reflects focused refinement of the shared mental model. MLLMs continue to enumerate new objects across turns, failing to stabilize a consistent shared spatial frame.

This divergence appears in efficiency statistics. Humans achieve 95.22% accuracy with 199.65 average words per conversation, while MLLMs average 438.48 words with 50.48% accuracy. Verbosity does not correlate with accuracy, with Pearson $R_i$1 and $R_i$2 (Sikarwar et al., 28 Mar 2026). The issue is therefore not simply insufficient dialogue length; it is the inability to turn dialogue into a stable allocentric representation.

Repair behavior provides a sharper diagnostic. Humans show a repair rate of 79.31%, meaning they frequently detect and correct flawed reasoning trajectories mid-conversation. Gemini-3-Pro-Thinking reaches 28.04%, and Qwen3-VL-32B only 7.8% (Sikarwar et al., 28 Mar 2026). Limited metacognitive repair contributes directly to non-convergent thought spaces, because early misalignments are not actively corrected.

The error taxonomy clarifies what goes wrong. Across 150 failed instances for the best model, with multiple labels per conversation, perceptual failures account for 19.70%, cross-view grounding failures for 46.09%, and geometric and relational failures for 34.21% (Sikarwar et al., 28 Mar 2026). Cross-view grounding dominates in Anchor Recognition and Global Counting, accounting for 67.85% and 64.00% of errors. Geometric and relational failures dominate in Relative Direction and Cognitive Mapping, accounting for 57.33% and 69.44% of errors. The paper explicitly notes cascading errors: early grounding mistakes propagate through dialogue, corrupt the shared thought space, and lead to downstream geometric inconsistencies.

5. Limitations, formal implications, and design directions

COSMIC evaluates zero-shot models with accuracy as the main metric, using 90% bootstrap confidence intervals; open-source models are averaged over 4 runs and closed-source models over 2 runs (Sikarwar et al., 28 Mar 2026). Its design also imposes clear boundaries. Scenes are static, viewpoints are fixed, object vocabularies are limited, and there is no active perception or movement. The multiple-choice framing, especially the binary mapping task, simplifies evaluation and avoids free-form map generation. Accuracy-only metrics do not directly reward dialogue quality. The human study uses university students in controlled settings (Sikarwar et al., 28 Mar 2026).

These limitations matter because they define what the benchmark does and does not isolate. It measures language-mediated spatial integration under partial views, but not active exploration, continuous pose estimation, or open-ended diagram production. A plausible implication is that current failures in allocentric map building may be even more consequential in dynamic settings, because the present benchmark already shows near-chance performance on mapping without motion (Sikarwar et al., 28 Mar 2026).

The paper therefore recommends architectural changes aimed directly at thought-space maintenance. These include explicit spatial memory, such as persistent scene graphs keyed by unique object descriptors and anchors; structured map building through internal top-down maps or metric cognitive maps updated each turn; graph-based representations with multi-view scene graphs and constraint satisfaction; alignment protocols based on anchor-first grounding and explicit agreement on reference frames; grounding with object detection, segmentation, and multi-view geometry to estimate $R_i$3 and relative pose; longer-term memory and consistency checks that detect contradictions and trigger repair moves; and visual sketch-based communication when intermediate artifacts are permitted (Sikarwar et al., 28 Mar 2026).

Future work is correspondingly framed around richer thought-space supervision. Proposed next steps include dynamic scenes and agent motion, free-form map generation and evaluation, richer object taxonomies, protocols that explicitly measure dialogue quality and repair, visual or metric cognitive maps, mental rotation modules, robust anchor-first conventions, reference-frame-agreement conventions, and proactive repair strategies (Sikarwar et al., 28 Mar 2026). The benchmark’s broader implication is that robust spatial thought space requires more than verbal deliberation: it requires persistent structure, explicit geometry, and active consistency maintenance.

6. Broader research landscape

Outside collaborative spatial communication, “thought space” has become a broader research term for structured intermediate reasoning spaces. In ThoughtProbe, it is the tree-structured response space induced by sampling multiple continuations, together with internal activation-space signals that score how “thoughtful” each branch is; a simple linear classifier on hidden representations guides search and branch aggregation across arithmetic benchmarks (Wang et al., 9 Apr 2025). In SoftCoT, thought space is a continuous latent representation space populated by assistant-generated “soft thought tokens” that are projected into the backbone LLM without modifying the backbone itself (Xu et al., 17 Feb 2025). CODI similarly compresses chain-of-thought into a fixed number of continuous tokens and aligns explicit and implicit reasoning through self-distillation on hidden states, reaching a reported 3.1× compression rate on GSM8k-Aug at GPT-2 scale (Shen et al., 28 Feb 2025). SoftCoT++ extends this line to test-time scaling by perturbing latent thoughts with multiple specialized initial tokens and contrastive learning to diversify exploration in continuous space (Xu et al., 16 May 2025).

In multimodal research, thought space is often a latent visual-language substrate rather than a dialogue-induced map. CoCoVa represents intermediate reasoning as a sequence of latent thought vectors refined by a Latent Q-Former, with symmetric InfoNCE and diffusion-based reconstruction enforcing multimodal grounding (Ma et al., 4 Nov 2025). MCOUT uses the LLM’s last hidden state, or a multimodally attended variant of it, as a continuous thought that is iteratively appended back into the input sequence for latent-space reasoning in vision-LLMs (Pham et al., 18 Aug 2025). Render-of-Thought renders textual chain-of-thought as images, uses a frozen vision encoder as a semantic anchor, and reports 3–4× token compression with substantial inference acceleration relative to explicit CoT (Wang et al., 21 Jan 2026). SpecFlow defines a bounded visual thought space in a discrete cosine domain and updates it with classifier-free guided flow dynamics, reducing KV-cache costs by up to 2.1 times in multimodal spatial reasoning (Shen et al., 1 Jun 2026). “3D Space as a Scratchpad for Editable Text-to-Image Generation” turns thought space into an explicit 3D reasoning substrate for placement, orientation, and viewpoint selection, reporting a 32% improvement in text alignment on GenAI-Bench relative to Flux.1-dev (Saha et al., 21 Jan 2026).

Multiagent work generalizes the concept again. Thought Communication in Multiagent Collaboration models thoughts as latent variables $R_i$4 and proves identifiability of shared thoughts, private thoughts, and global thought-sharing structure under sparsity-regularized decoding (Zheng et al., 23 Oct 2025). Think-Before-Speak operationalizes a structured internal state space for social simulation, separating private evaluation from public utterance generation across intervals and showing that dissonance-related appraisal increases willingness to speak while silence-pressure appraisal decreases it (Yang et al., 2 Jun 2026).

Taken together, these papers suggest that thought space has become an organizing concept for intermediate reasoning substrates that are neither identical to final outputs nor reducible to raw hidden states. In the COSMIC formulation, the crucial point is that a thought space must be jointly maintained, geometrically consistent, and repairable under communication. That requirement makes collaborative spatial thought space a particularly stringent test of whether multimodal systems can move from local recognition to stable shared world models (Sikarwar et al., 28 Mar 2026).