CogniMap3D: Explicit 3D Cognitive Mapping

• CogniMap3D is a recurring design motif that integrates explicit 3D mapping with cognitive representations, replacing latent inference with structured maps.
• It spans diverse applications including neuroimaging, EEG-driven object reconstruction, dynamic scene mapping, and multi-view vision-language reasoning.
• The framework employs detailed pipelines—voxel re-projection, graph attention, cognitive memory banks, and deterministic geometric operators—to enhance interpretability and performance.

Searching arXiv for the cited works and related uses of “CogniMap3D” to ground the article. CogniMap3D is a label used for several research programs that combine cognitive-map abstractions with explicit spatial or semantic structure. Across the cited literature, the term denotes at least four distinct technical lineages: a neuroimaging pipeline for mapping cognitive ontologies to 3D brain activation maps (Schwartz et al., 2013); an EEG-driven system for generating 3D objects through an intermediate language representation (Deng et al., 16 Apr 2025); a bio-inspired framework for dynamic 3D scene mapping with persistent memory and rapid retrieval (Wang et al., 13 Jan 2026); and a blueprint for explicit 3D spatial reasoning based on a Metric Cognitive Map and Cognitive Chain-of-Thought (Gao et al., 16 Jan 2026). A closely related line of work, though named differently, is Cog3DMap, which recurrently constructs an explicit 3D memory from multi-view images for MLLM reasoning (Gwak et al., 24 Mar 2026). The shared conceptual thread is the replacement of purely implicit representations with explicit, structured maps that support retrieval, interpretation, or reasoning.

1. Terminological scope and conceptual background

The term “cognitive map” is explicitly tied to the tradition inaugurated by O’Keefe and Nadel’s The Hippocampus as a Cognitive Map (1978), which proposed that the hippocampus constructs and maintains an internal, allocentric representation of space underlying spatial memory and navigation. The précis provided for that work emphasizes allocentric coding, flexible navigation, and cue integration, as well as the broader legacy for episodic memory and spatial cognition. This background is directly invoked in the broader family of map-centric systems summarized here, particularly those that seek persistent spatial memory, metric scene structure, or explicit relational reasoning (Gwak et al., 24 Mar 2026).

Within the present corpus, however, CogniMap3D does not denote a single canonical architecture. Instead, it names multiple systems in different domains that share a commitment to explicit representation. In neuroimaging, the emphasis is a many-to-many mapping between activation images and ontology terms, coupled with reverse-inference atlases (Schwartz et al., 2013). In EEG-based generation, the emphasis is a staged translation from neural signals to language and then to 3D Gaussian scenes (Deng et al., 16 Apr 2025). In embodied 3D perception, the emphasis is persistent memory, static-scene retrieval, and factor-graph refinement across revisits (Wang et al., 13 Jan 2026). In 3D VLM reasoning, the emphasis is a dual-format map—discrete and continuous—paired with deterministic geometric operators (Gao et al., 16 Jan 2026).

This multiplicity matters because references to CogniMap3D can otherwise be misconstrued as referring to a single benchmarked method. A more accurate characterization is that CogniMap3D functions as a recurring design motif: explicit map construction in place of latent-only inference.

2. Neuroimaging instantiation: cognitive ontology to 3D brain maps

In the neuroimaging setting, CogniMap3D is described as a pipeline introduced in “Mapping cognitive ontologies to and from the brain” (Schwartz et al., 2013). Its purpose is to build a bidirectional link between observed brain activity and cognitive function by annotating fMRI contrast maps with a shared ontology and learning predictive models over reduced voxel-space features.

The ontology layer relies on the Cognitive Paradigm Ontology (CogPO). Four CogPO categories are used: stimulus modality, explicit stimulus, instructions, and overt response. Each contrast map in the 19-study corpus—3 826 subject-level maps and 131 map types—is manually annotated with the subset of CogPO terms that describes the experimental condition. This yields a many-to-many mapping between images and cognitive terms, allowing co-analysis across heterogeneous tasks (Schwartz et al., 2013).

The imaging pipeline begins with spatial normalization into MNI space using SPM, followed by Gaussian smoothing. Each map $\mathcal{x}\in\mathbb{R}^p$ is represented on a common grid of approximately $p\approx48\,000$ voxels. Dimensionality reduction proceeds in two stages: spatially constrained Ward clustering reduces the representation to approximately 15 000 parcels, after which a univariate one-way ANOVA retains the top 30% most discriminative parcels. The resulting feature vector is $X\in\mathbb{R}^d$ with $d\approx4\,500$ (Schwartz et al., 2013).

The predictive layer is formulated in terms of forward and inverse modeling. For a binary label $Y$ indicating the presence or absence of a CogPO term $T$, the forward model is $P(X\mid Y)$ and the reverse-inference model is $P(Y\mid X)$. Bayes’ rule is given as

$P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$

Because direct estimation of the full high-dimensional $P(X\mid Y)$ is ill-posed, the implemented model is an $p\approx48\,000$0-penalized logistic regression:

$p\approx48\,000$1

with parameters tuned by minimizing the penalized negative log-likelihood

$p\approx48\,000$2

A central technical issue is the “long tail” of rare paradigms. Two mitigation strategies are specified. First, terms are modeled category-wise with one-vs-all classifiers, sharing feature-selection and reduction steps within each CogPO parent category. Second, inverse-frequency weighting is applied during training so that under-represented classes contribute equally to the loss; at test time, raw probabilities are re-scaled by the prior prevalence $p\approx48\,000$3 (Schwartz et al., 2013). This directly addresses statistical power for rarely observed ontology terms.

Evaluation is based on leave-one-study-out cross-validation, supplemented by a leave-one-laboratory-out experiment. Reported metrics per term are precision and recall, with chance levels computed analytically from prevalence. The stated result is that logistic regression regularly achieves precision and recall well above chance even for moderately rare terms such as “face,” “move,” and “saccades,” whereas k-nearest neighbors and naïve Bayes suffer from very low precision or recall on the long tail (Schwartz et al., 2013).

Once a weight vector $p\approx48\,000$4 has been learned for a term, the weights are re-projected onto the original voxel grid by un-clustering parcels back into voxels, producing a 3D map of $p\approx48\,000$5 in MNI space. Optional smoothing and thresholding of the reverse-inference $p\approx48\,000$6-map yield “reverse-inference atlases,” with examples including the fusiform face area for “face,” frontal eye fields and IPS for “saccades,” and the dorsal attention network for “count” (Schwartz et al., 2013). This use of “3D map” is anatomical rather than scene-geometric, but it remains a paradigmatic instance of explicit cognitive mapping.

3. EEG-driven 3D reconstruction: neural signals to language to Gaussians

A second usage appears in the description of CogniMap3D within “Mind2Matter: Creating 3D Models from EEG Signals” (Deng et al., 16 Apr 2025). Here the system is organized as an EEG-to-embedding-to-text-to-layout-to-3D pipeline. The objective is not ontology decoding from fMRI, but direct 3D object reconstruction from EEG through an intermediate natural-language representation.

The EEG encoder takes multi-channel EEG time-series $p\approx48\,000$7 with $p\approx48\,000$8 electrodes and $p\approx48\,000$9 ms at 1 kHz, and outputs an embedding $X\in\mathbb{R}^d$0 with $X\in\mathbb{R}^d$1. The architecture includes a Graph Attention module with 4-head attention on a graph $X\in\mathbb{R}^d$2, multi-scale temporal convolution with five 2D convolution layers whose kernel dilations double at each layer, four spatial convolution layers, transformer-style residual blocks, and a final $X\in\mathbb{R}^d$3 convolution plus ReLU leading to $X\in\mathbb{R}^d$4 (Deng et al., 16 Apr 2025).

Training minimizes a composite EEG objective:

$X\in\mathbb{R}^d$5

The cross-entropy term supervises ImageNet-style object labels; the CAML component combines InfoNCE with an adaptive-margin loss, using similarity terms $X\in\mathbb{R}^d$6 and margins $X\in\mathbb{R}^d$7 (Deng et al., 16 Apr 2025). The formulation is explicitly multimodal: the EEG signal is trained to align with image semantics while remaining class-discriminative.

The language stage maps $X\in\mathbb{R}^d$8 into a sequence of prefix embeddings via a 2-layer MLP:

$X\in\mathbb{R}^d$9

These prefix embeddings are prepended in-network to a fixed LLM, specified as Mistral-7B-Instruct-v0.3, with the prompt template:

• system: “You are an EEG interpreter.”
• user: “<EEG><Label> Describe it in one sentence.”

Only the MLP prefix is trained; the LLM weights remain frozen. The LLM loss is token-level cross-entropy against the ground-truth description $d\approx4\,500$0 (Deng et al., 16 Apr 2025).

The 3D generative stage uses 3D Gaussians. A scene is $d\approx4\,500$1, where each Gaussian is parameterized by $d\approx4\,500$2, covariance $d\approx4\,500$3, color $d\approx4\,500$4, and opacity $d\approx4\,500$5, and evaluates as

$d\approx4\,500$6

From the text description $d\approx4\,500$7, an LLM predicts object-level bounding boxes $d\approx4\,500$8. Optimization then proceeds in two stages—object-level and scene-level—using Score Distillation Sampling losses, combined as

$d\approx4\,500$9

The full strategy is two-stage rather than end-to-end: Stage A trains EEG-to-text with $Y$0; Stage B freezes Stage A, generates descriptions and layouts, and optimizes Gaussians with $Y$1 (Deng et al., 16 Apr 2025).

The reported evaluation covers both EEG-to-text and text-to-3D. On the test split averaged over 6 subjects, ROUGE-1 F1 is 34.21% versus 31.84% for the best baseline, BLEU-4 is 7.62% versus 6.14%, and BERTScore F1 is 37.19% versus 35.41%. Over 40 categories for text-to-3D, CLIP Similarity is 0.701 versus 0.602 for GraphDreamer, LPIPS is 0.664 versus 0.689, Chamfer Dist is 4.66 versus 10.68, and EMD is 10.93 versus 14.55 (Deng et al., 16 Apr 2025). Qualitatively, the system is said to capture object color, shape, and inter-object layout, and to produce view-consistent reconstructions relative to an EEG→Image→3D pipeline.

This version of CogniMap3D is therefore a neural decoding and generative modeling framework rather than a scene-memory system. The commonality with the other variants lies in explicit intermediate structure: EEG is not mapped directly to 3D geometry, but mediated by embeddings, language, layout, and Gaussian scene parameters.

4. Dynamic 3D mapping, memory, and rapid retrieval

The most literal usage of the name appears in “CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval” (Wang et al., 13 Jan 2026). Here CogniMap3D is a bio-inspired framework for dynamic 3D mapping that combines a multi-stage motion cue pipeline, a cognitive memory bank for static scenes across revisits, and factor-graph optimization for pose refinement.

The overall architecture contains four interacting modules: monocular video input $Y$2; a visual foundation model, specified as VGGT, producing depth maps $Y$3 and camera poses $Y$4; a Multi-Stage Motion Cue Framework that produces a dynamic mask $Y$5 and static mask $Y$6; and a Cognitive Mapping Memory Bank storing static scene representations $Y$7, which are later consumed by a factor graph optimizer to yield refined poses $Y$8 and an updated global map (Wang et al., 13 Jan 2026).

Dynamic segmentation is achieved by three cascaded cues. The first is an optical-flow motion cue: dense flow is computed via

$Y$9

then clustered into $T$0 Gaussian components by GMM. A coarse dynamic mask is defined by excluding the cluster with the smallest average flow magnitude. The second is a geometry-based cue: each pixel is back-projected to 3D using depth and camera intrinsics,

$T$1

and a predicted static-background flow $T$2 is derived from camera motion. Residual flow is thresholded by Otsu to obtain $T$3. The third is a robust motion cue based on 3D keypoints: matched keypoints are unprojected, displacements are measured after camera-motion compensation, and outliers are marked dynamic via a Mahalanobis-style criterion before dilation and refinement. The final dynamic mask is the union

$T$4

Prompt points inside $T$5 are fed to SAM2 for temporal mask refinement (Wang et al., 13 Jan 2026).

The memory bank maintains persistent maps for discovered static scenes. For each map $T$6, it stores a downsampled static point cloud $T$7, a global 2D embedding $T$8 computed from DINOv2 over keyframes, and a 3D feature vector $T$9 from PointNet++. For a new keyframe and static mask, descriptor extraction produces

$P(X\mid Y)$0

Matching is based on a score

$P(X\mid Y)$1

Top-$P(X\mid Y)$2 candidates are retrieved via approximate nearest neighbors, the best map is selected by sequence voting, and ICP verification is applied. A candidate is accepted if the inlier count exceeds $P(X\mid Y)$3 and RMSE is below $P(X\mid Y)$4 (Wang et al., 13 Jan 2026).

Upon relocalization, the memory is updated. The 2D memory is updated by

$P(X\mid Y)$5

and the 3D memory is updated by voxel-downsampling the union of stored static points with transformed current observations. If no match passes geometric verification, a new map is created from $P(X\mid Y)$6 (Wang et al., 13 Jan 2026). This explicitly instantiates “storage,” “recall,” and “updating,” which are the core bio-inspired metaphors of the system.

Pose and landmark refinement is formulated as factor-graph optimization over

$P(X\mid Y)$7

with prior, projection, and inter-frame motion factors. The cost is

$P(X\mid Y)$8

where $P(X\mid Y)$9 is the Huber loss. Landmarks from memory are assigned smaller covariance by the update $P(Y\mid X)$0 with $P(Y\mid X)$1. The solver is Levenberg–Marquardt on the $P(Y\mid X)$2 manifold with iterative re-orthogonalization by SVD and per-iteration outlier rejection (Wang et al., 13 Jan 2026).

The evaluation spans depth estimation, camera pose, and 3D reconstruction. Reported highlights include BONN depth performance of AbsRel = 0.058 and $P(Y\mid X)$3; TUM-dynamic pose performance in the FF setting with ATE = 0.012 m, RPE$P(Y\mid X)$4 = 0.010 m, and RPE$P(Y\mid X)$5 = $P(Y\mid X)$6; KITTI 3D reconstruction performance of Accuracy = 0.052 m, Completion = 0.073 m, and Normal Consistency = 0.942; and long-sequence recall reducing ATE from 0.092 to 0.060 and further to 0.052 m with memory (Wang et al., 13 Jan 2026). These results frame CogniMap3D as a persistent mapping system for dynamic environments rather than merely a single-pass 3D reconstruction pipeline.

5. Explicit spatial reasoning via Metric-CogMap and Cog-CoT

A related but distinct formulation appears in “Map2Thought: Explicit 3D Spatial Reasoning via Metric Cognitive Maps” (Gao et al., 16 Jan 2026). The paper itself introduces Map2Thought, but the detailed blueprint explicitly describes a system that “realizes the Map2Thought framework” as “CogniMap3D.” In this formulation, the emphasis is not map retrieval or SLAM-style pose refinement, but explicit and interpretable 3D reasoning.

The core representational structure is the Metric Cognitive Map (Metric-CogMap), in which every detected object $P(Y\mid X)$7 is encoded in two parallel formats. The first is a discrete $P(Y\mid X)$8 grid suitable for symbolic relational queries. The second is a continuous, metric-scale representation in a globally aligned real-world frame. For the discrete component, horizontal extents are defined by $P(Y\mid X)$9, and cell indices are computed by functions $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$0 and $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$1, producing a discrete centroid

$P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$2

A discrete AABB or “grid-box” is ასევე stored. For the continuous component, each object stores a centroid

$P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$3

and AABB half-extents

$P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$4

The unified object representation is

$P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$5

(Gao et al., 16 Jan 2026).

Reasoning is handled by Cognitive Chain-of-Thought (Cog-CoT), described as a small library of deterministic geometric operators plus a trace format for step-by-step inference. Primitive operations include vector subtraction, dot product, a 2D cross product whose sign indicates left or right, Euclidean centroid distance, AABB distance in the $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$6 plane, and occlusion-aware first-appearance ordering via depth-tested projection into frames (Gao et al., 16 Jan 2026). The explicitness of these operators contrasts with methods that ask a VLM to infer spatial relations purely from latent embeddings or unstructured captions.

Two example algorithms are given. One computes relative direction such as “front-left” or “front-right” from an origin, a facing point, and a target by combining the sign of the dot product with the sign of the 2D cross product. Another selects the nearest object by minimizing AABB distance over a candidate set. A toy three-object example demonstrates that if $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$7 is at $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$8, $P(Y\mid X)=\frac{P(X\mid Y)\,P(Y)}{P(X)}.$9 at $P(X\mid Y)$0, and $P(X\mid Y)$1 at $P(X\mid Y)$2, then under an orientation where $P(X\mid Y)$3 faces along $P(X\mid Y)$4, $P(X\mid Y)$5 is “front-right” and $P(X\mid Y)$6 is “front-left” (Gao et al., 16 Jan 2026).

The upstream pipeline from raw data to map construction includes 2D detectors such as Detic and Grounding DINO, video-level tracking, Covisibility Maps via point-cloud reprojection, multi-view merging with SAM2, spatial deduplication through covisibility filtering, and global alignment and metric rescaling with MoGe-2. A JSON-like map structure is then assembled containing scene bounds, grid size, and per-object entries with category, grid position, grid box, centroid, and extent (Gao et al., 16 Jan 2026). Inference proceeds by parsing the question, extracting the relevant submap, appending a task instruction template such as “Compute dot, cross,” and feeding map, task, and question into a VLM, which executes Cog-CoT operations step by step before emitting the answer.

The paper’s headline performance claim is that Map2Thought achieves 59.9% accuracy using only half the supervision, closely matching the 60.9% baseline trained with the full dataset, and outperforms state-of-the-art methods by 5.3%, 4.8%, and 4.0% under 10%, 25%, and 50% training subsets on VSI-Bench (Gao et al., 16 Jan 2026). A plausible implication is that explicit metric representations can substitute for part of the supervision otherwise needed by end-to-end latent reasoning systems.

6. Relation to multi-view VLM reasoning and the broader cognitive-map motif

The family resemblance between these systems becomes clearer when set beside “Cog3DMap: Multi-View Vision-Language Reasoning with 3D Cognitive Maps” (Gwak et al., 24 Mar 2026). Although this work is not titled CogniMap3D, it articulates the same central diagnosis: MLLMs often lack explicit geometric grounding, and merely augmenting visual tokens with geometric cues still leaves the model to infer 3D structure implicitly. The proposed remedy is to recurrently construct an explicit 3D memory from multi-view images, with each token grounded in 3D space and carrying both semantic and geometric information. Feeding these tokens into the MLLM enables direct reasoning over a spatially structured 3D map, with state-of-the-art performance on various spatial reasoning benchmarks (Gwak et al., 24 Mar 2026).

This places the different CogniMap3D usages on a common spectrum. In the neuroimaging case, the explicit map is a reprojected voxelwise $P(X\mid Y)$7-field linked to ontology terms (Schwartz et al., 2013). In EEG-based generation, it is a structured sequence of descriptions, layouts, and 3D Gaussians that mediates from neural activity to geometry (Deng et al., 16 Apr 2025). In dynamic mapping, it is a persistent bank of static scene representations coupled to retrieval and optimization (Wang et al., 13 Jan 2026). In Map2Thought-style reasoning, it is a hybrid discrete-continuous object map equipped with deterministic operators (Gao et al., 16 Jan 2026). Cog3DMap, finally, frames explicit 3D memory as an interface between visual perception and language reasoning (Gwak et al., 24 Mar 2026).

The underlying design principle is therefore stable even where the implementations are not. These works all move away from purely semantic or implicit latent states toward explicit, inspectable structures that support one or more of the following: reverse inference, long-tail generalization, viewpoint consistency, revisitation-aware mapping, or interpretable geometric reasoning. This suggests that “CogniMap3D” is best understood less as a single method than as a recurring research program: explicit 3D or volumetric cognitive maps as an organizing substrate for prediction, retrieval, and reasoning.