Metric Cognitive Map

• Metric Cognitive Map is a dual-format spatial representation that combines a discrete grid for symbolic reasoning with continuous metric embeddings for precise 3D geometry.
• Its design integrates symbolic relational reasoning and geometric computation to enable deterministic spatial inference and explainable chain-of-thought processes.
• Empirical evaluations demonstrate that Metric-CogMap outperforms grid-only methods, achieving state-of-the-art accuracy in multimodal spatial benchmarks.

A Metric Cognitive Map (Metric-CogMap) is a dual-format spatial representation central to recent advances in explicit 3D spatial reasoning in multimodal LLMs (VLMs, MLLMs). Its design unifies symbolic relational reasoning with precise geometric computation by tightly integrating a discrete grid map and a continuous metric-scale embedding. This structure enables explicit, interpretable chain-of-thought (COT) spatial inference and supports a range of deterministic geometric operations. Metric-CogMap forms the core of the Map2Thought framework, delivering state-of-the-art explainable 3D understanding and outperforming grid-only baselines in a variety of benchmarks (Gao et al., 16 Jan 2026). Related work on metric-grounded cognitive maps, such as Video2Layout, reinforces the value of metric embeddings for quantitative spatial reasoning, demonstrating significant performance improvements over grid-based methods (Huang et al., 20 Nov 2025).

1. Formal Definition and Structure

A Metric Cognitive Map for a single scene is formally defined as the tuple

$\mathcal{C} = (G, M)$

where:

• $G$ is a discrete $N \times N$ grid representation for symbolic reasoning and relational spatial structure.
• $M$ is a set of continuous, metric-scale embeddings for each object, encoding precise 3D geometry.

Discrete Grid ($G$):

Each object $i$ is assigned:

• A grid coordinate $$g^{(i)} = [g_x^{(i)}, g_y^{(i)}] \in \{0, \ldots, N-1\}^2$$
• An axis-aligned bounding box in grid coordinates:

$$B^{(i)}_{\mathrm{grid}} = [\, [g_x^{(i),\min}, g_x^{(i),\max}],\, [g_y^{(i),\min}, g_y^{(i),\max}] ]$$

$N=20$ in Map2Thought.

Continuous Metric Embedding ($M$):

Each object $i$ stores:

• 3D centroid $$c^{(i)} = [x^{(i)}, y^{(i)}, z^{(i)}] \in \mathbb{R}^3$$
• Half-sizes of the axis-aligned bounding box $$s^{(i)} = \frac{1}{2}[w^{(i)}, h^{(i)}, d^{(i)}] \in \mathbb{R}^3$$
• Full embedding $m^{(i)} = (c^{(i)}, s^{(i)}) \in \mathbb{R}^6$
• $M = \{ m^{(i)} \}_{i=1}^{N_{\text{obj}}}$

This duality affords both qualitative (topological, relational) and quantitative (distance, size, orientation) queries.

2. Construction, Mapping Functions, and Alignment

Construction of the Metric-CogMap is achieved through deterministic mapping functions:

• 3D Reconstruction to Continuous Embedding:

$$\phi_{\mathrm{geom}}: \mathcal{P} \to \{(c^{(i)}, s^{(i)})\}$$

where $\mathcal{P}$ is the set of multi-view 3D point clouds and poses. Off-the-shelf 3D detectors combined with covisibility-guided clustering group points into object instances, with bounding boxes extracted for centroids and sizes.

• Continuous to Discrete Grid Projection:

Given $c^{(i)} = [x^{(i)}, y^{(i)}, z^{(i)}]$, discretize:

$$g_x^{(i)} = \left\lfloor \frac{x^{(i)} - x_\mathrm{min}}{x_\mathrm{max} - x_\mathrm{min}} N \right\rfloor,\quad g_y^{(i)} = \left\lfloor \frac{y^{(i)} - y_\mathrm{min}}{y_\mathrm{max} - y_\mathrm{min}} N \right\rfloor$$

Grid boxes are computed in the same manner from continuous coordinates.

Integration and Indexing:

A strict one-to-one index alignment between every discrete grid entry and the corresponding metric embedding ensures direct correspondence. No learned transform is introduced; normalization steps such as clipping and scene-level rescaling ensure compatibility.

3. Deterministic Operations and Spatial Reasoning

Metric-CogMap enables explicit spatial reasoning via deterministic, closed-form operations, executed by the Cognitive Chain-of-Thought (Cog-CoT) module:

• Relative Direction:

Given objects (face, target, origin) with centroids, compute:

$$\mathbf{f} = c^{(\mathrm{face})} - c^{(\mathrm{orig})},\quad \mathbf{t} = c^{(\mathrm{target})} - c^{(\mathrm{orig})}$$

Dot and cross-product ($f_x t_y - f_y t_x$) reveal front/back and left/right relations.

• Absolute Distance:

$$\Delta = |c^{(1)} - c^{(2)}|,\; s = s^{(1)} + s^{(2)},\; d = \max(\Delta - s, 0)$$

Euclidean norm of $d$ yields approximate object center distances.

• Axis-Aligned Bounding Box (AABB) Distance:

For boxes $[x_\mathrm{min}^i, x_\mathrm{max}^i]$ and $[x_\mathrm{min}^j, x_\mathrm{max}^j]$:

$$d_x = \begin{cases} x_\mathrm{min}^j - x_\mathrm{max}^i & x_\mathrm{min}^j > x_\mathrm{max}^i \ x_\mathrm{min}^i - x_\mathrm{max}^j & x_\mathrm{min}^i > x_\mathrm{max}^j \ 0 & \text{otherwise} \end{cases} \quad d_y = \ldots$$

AABB distance: $\sqrt{d_x^2 + d_y^2}$

• Occlusion-Aware Appearance Ordering:

Re-project each object's 3D points to video frames and determine the first un-occluded frame $F^{(i)}$:

$$F^{(i)} = \min \{ t \mid \text{visibility}(i,t) = \mathrm{True} \}$$

These explicit operations underpin interpretable COT inference traces.

4. Training Paradigms and Objectives

Map2Thought employs sequence-to-sequence cross-entropy loss over the joint input of the question, Cog-CoT trace, and Metric-CogMap without auxiliary losses placed on $G$ or $M$: $$\mathcal{L}_{\mathrm{QA}} = -\sum_{t=1}^T \log p_\theta(y_t \mid y_{<t},\;[\text{question}+ \text{CogCoT} + \text{MetricCogMap}])$$ Partial supervision simply subsamples QA pairs; no other loss terms directly affect the map components (Gao et al., 16 Jan 2026).

In Video2Layout, a two-stage paradigm uses:

1. Supervised fine-tuning with $L_2$ box regression loss:

$$L_\mathrm{boundary} = \frac{1}{N} \sum_{i=1}^N \|b_i^{\text{gt}} - \hat b_i\|_2^2$$

2. Reinforcement learning with a composite reward for chain-of-thought format and task correctness, optimizing a clipped PPO objective (Huang et al., 20 Nov 2025).

5. Empirical Evaluations, Ablations, and Interpretability

Empirical results confirm the essential role of metric-scale embeddings and explicit reasoning:

Map/Reasoning Configuration	VSI-Bench Accuracy (%) (Gao et al., 16 Jan 2026)
Pred. Metric-CogMap + Cog-CoT	58.8
Pred. Metric-CogMap only (no Cog-CoT)	54.0
No CogMap (VLM-3R baseline)	54.0
Grid-only (no metric)	49.7
Upper-bound (GT Metric-CogMap + Cog-CoT)	73.7

Qualitative visualizations reveal interpretable multi-format scene maps. Ablations consistently demonstrate:

• Continuous metric embeddings are critical for precise distance/size/area queries.
• Symbolic grid alone is insufficient for fine-grained geometric reasoning.
• Cog-CoT reasoning over the unified map is necessary to achieve full performance.

Video2Layout's metric-grounded cognitive map yields a $+4.92\%$ improvement on core spatial reasoning tasks compared to grid-only variants, with structured numeric COT further raising accuracy to $51.52\%$ (Huang et al., 20 Nov 2025).

6. Related Approaches and Limitations

Video2Layout frames the metric-grounded cognitive map as a set of continuous 2D BEV bounding boxes $(x_i^{\min}, y_i^{\min}, x_i^{\max}, y_i^{\max})$ for every object $o_i$, supporting direct Euclidean reasoning and perspective transforms, in contrast to coarse $M\times M$ grids. The mapping function $f_\theta$ links frame sequences to bounding boxes and derived metrics, validated by grid-to-metric ablations:

• Grid 10×10: $46.07\%$
• Grid 20×20: $46.60\%$
• Metric-grounded: $51.52\%$

Limitations include:

• Frame overload: For $T>8$, additional viewpoints degrade map quality (suggesting a sweet spot at $T=4$ for QVS-Bench).
• 2D BEV assumption: Fails to capture full 3D variability or occlusions.
• Synthetic-to-real gap: While RFT mitigates, exact metric grounding remains challenging in cluttered or poorly lit real scenes.
• Output budget: Scalability to dense scenes remains an open challenge (Huang et al., 20 Nov 2025).

This suggests future research may extend the metric representation to full 3D bounding volumes, integrate temporal consistency constraints, and develop self-supervised map refinement pipelines.

7. Significance, Impact, and Future Directions

Metric Cognitive Maps and their deterministic reasoning schemes represent a significant advance in the pursuit of explainable, precise, and verifiable spatial reasoning in multimodal LLMs. They enable step-by-step, math-grounded chain-of-thought suitable for both quantitative and qualitative tasks, offering interpretable traces for spatial understanding. Experimental evidence across Map2Thought and Video2Layout demonstrates consistent superiority over grid-only baselines, with improvements on key benchmarks and compelling ablations validating every component.

Open challenges include robust scalability, mitigating perception errors, generalization beyond 2D BEV, and learning without privileged metric supervision. Progress in these areas promises to further solidify the role of Metric-CogMap as a foundational representation for spatial intelligence in AI systems (Gao et al., 16 Jan 2026, Huang et al., 20 Nov 2025).