3DThinker: Implicit 3D Spatial Reasoning

• 3DThinker is a framework for grounded spatial reasoning that internally generates latent 3D representations without using explicit 3D inputs at inference.
• The method aligns vision-language model latent tokens with a 3D foundation model, enhancing spatial understanding through projected geometric features.
• A two-stage training process—supervised alignment followed by reinforcement learning—drives improved geometry preservation and answer accuracy across benchmarks.

Searching arXiv for 3DThinker and closely related spatial-reasoning frameworks to ground the article. 3DThinker is a framework for grounded spatial reasoning from limited views that trains a vision-LLM to form an internal latent representation of an imagined 3D scene during reasoning, rather than relying only on text or 2D visual cues. It is introduced in "Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views" (Chen et al., 21 Oct 2025), which describes the method as enabling "3D mentaling" without any 3D prior input at inference and without explicitly labeled 3D data for training. In this formulation, the model interleaves ordinary language tokens with a block of 3D special tokens whose hidden states are treated as a compact scene-geometry latent, then aligns that latent with a 3D foundation model during supervision and refines the full reasoning trajectory using outcome signals (Chen et al., 21 Oct 2025). Within the recent literature on spatial intelligence, 3DThinker belongs to a broader shift from passive visual recognition toward geometry-grounded reasoning, but it differs from tool-using and explicit-geometry systems by locating the 3D representation inside the model’s own reasoning process rather than in an external point cloud, camera controller, or API program (Chen et al., 21 Oct 2025, Zhang et al., 19 Jan 2026, Li et al., 5 Feb 2026).

1. Definition and problem setting

3DThinker addresses spatial reasoning from limited views, especially ego-centric or multi-view observations where only partial slices of the environment are visible. The motivating claim is that a human can infer the rest of a scene by mentally constructing a 3D layout from a few observations, whereas current vision-LLMs usually lack a mechanism for such geometry-grounded imagination (Chen et al., 21 Oct 2025).

The paper distinguishes two prevailing families of methods. One family reasons with pure text or 2D cues, whose representational capacity is limited for tasks that require 3D spatial imagination. A second family injects extra inputs or external tools such as depth estimators, point clouds, camera parameters, or 3D token encoders, but these methods may require additional supervision, reduce applicability to monocular settings, or add inference overhead (Chen et al., 21 Oct 2025). 3DThinker is proposed as an intrinsic alternative: the model forms 3D representations during reasoning from limited views, with no 3D prior input at inference and no densely labeled 3D data in training (Chen et al., 21 Oct 2025).

This places 3DThinker in contrast with frameworks such as Think3D, which reconstruct an explicit 3D scene and let an agent iteratively explore it through camera-based operations and ego/global-view switching (Zhang et al., 19 Jan 2026), and GeoThinker, which selectively retrieves geometry from a 3D encoder through Spatial-Grounded Fusion and frame-strict cross-attention (Li et al., 5 Feb 2026). A plausible implication is that 3DThinker seeks to internalize geometry as latent cognition, whereas these related systems either externalize geometry as an environment to manipulate or expose it as an auxiliary feature stream.

2. Core notion of “3D mentaling”

The central concept in 3DThinker is “3D mentaling.” The model generates special latent 3D tokens inside its reasoning chain; these are not ordinary text tokens, but compact carriers of an imagined scene geometry (Chen et al., 21 Oct 2025). The reasoning output is structured as

$o = o_{\text{pre} \oplus t_{\text{3D} \oplus o_{\text{post},$

where $o_{\text{pre}}$ is the text before the 3D latent block, $t_{\text{3D}}=\{t_1,\dots,t_k\}$ is the sequence of 3D special tokens, and $o_{\text{post}}$ is the text after the latent block (Chen et al., 21 Oct 2025). The last-layer hidden states corresponding to those tokens form the latent vectors

$F_{\text{latent}=\{h_1,\dots,h_k\}.$

These hidden states are generated recursively from the base VLM conditioned on the prior textual and latent context (Chen et al., 21 Oct 2025).

The intended role of the latent is to bridge multimodal perception and reasoning. The model first interprets the image set, then internally imagines a likely 3D scene structure, and then continues reasoning conditioned on that internal scene state (Chen et al., 21 Oct 2025). The method therefore does not perform explicit image generation, explicit textual map construction, or inference-time tool use. Instead, it treats spatial cognition as a latent-space process.

This conceptual move is important in relation to nearby work. Think3D frames spatial reasoning as an interactive 3D chain-of-thought over an explicit reconstructed point cloud and camera poses (Zhang et al., 19 Jan 2026). DeepThink3D frames complex 3D situated reasoning as programmatic tool use over APIs such as scene(), filter(x, c), relate(...), and query_relation(...) (Song et al., 21 Aug 2025). 3DThinker, by contrast, keeps the 3D structure implicit at inference time, although its internal latent is trained to correspond to actual geometry (Chen et al., 21 Oct 2025).

3. Architecture and latent-to-geometry alignment

Architecturally, 3DThinker is built on top of a base VLM $f_\theta$ and takes as input a question $Q$, a set of images $\mathcal{I}=\{I_1,\dots,I_n\}$, and a response trajectory (Chen et al., 21 Oct 2025). To connect the latent reasoning state to actual 3D structure, the method introduces a projector that maps the VLM latent space into the feature space of a 3D foundation model. In the reported implementation, the 3D foundation model is VGGT (Chen et al., 21 Oct 2025).

The image encoder produces visual features $F_{\text{images}}$, VGGT yields geometry features $F_{\text{3D}}=f_{\text{vggt}}(\mathcal{I})$, and the projected latent is

$o_{\text{pre}}$0

The paper explicitly prefers projecting from VLM latent space into VGGT space rather than compressing VGGT into VLM space, because the former keeps the 3D latent recoverable and visually interpretable (Chen et al., 21 Oct 2025). After projection, the latent can be decoded back into a 3D representation such as a point cloud via VGGT’s downstream dense prediction module (Chen et al., 21 Oct 2025).

This design choice differentiates 3DThinker from GeoThinker. GeoThinker also uses VGGT, but there geometry is extracted as per-frame 3D-aware features and injected into the VLM backbone through SGF modules at carefully selected layers (Li et al., 5 Feb 2026). In 3DThinker, geometry is not injected as an external stream during inference; instead, the model is trained so that its internally generated latent becomes geometrically meaningful (Chen et al., 21 Oct 2025).

4. Two-stage training procedure

Training proceeds in two stages: supervised alignment followed by reinforcement learning with outcome-based signals only (Chen et al., 21 Oct 2025).

In stage 1, the model is supervised to align its generated 3D latent with the feature space of the 3D foundation model while preserving textual coherence. The authors synthesize chain-of-thought data using 10K training examples from MindCube and a strong teacher, GPT-4o (Chen et al., 21 Oct 2025). For each tuple $o_{\text{pre}}$1, the teacher produces a reasoning chain $o_{\text{pre}}$2 containing the 3D special tokens, yielding a dataset

$o_{\text{pre}}$3

The supervised objective combines a 3D alignment term with standard text cross-entropy. The 3D alignment loss is the Frobenius distance between projected latent features and VGGT geometry features,

$o_{\text{pre}}$4

while textual coherence is maintained by separate cross-entropy terms for the text before and after the 3D block (Chen et al., 21 Oct 2025). The total objective is

$o_{\text{pre}}$5

with $o_{\text{pre}}$6 and $o_{\text{pre}}$7 in the reported experiments (Chen et al., 21 Oct 2025).

In stage 2, the projector is frozen and the policy is updated with GRPO using only outcome-based rewards (Chen et al., 21 Oct 2025). For each $o_{\text{pre}}$8, the model samples $o_{\text{pre}}$9 candidate completions, and the GRPO objective optimizes the entire reasoning trajectory, including the latent block. The reward comprises three components: a 3D alignment reward based on cosine similarity between the rollout’s projected latent and VGGT features, a binary formatting reward that enforces the latent and answer structure, and a $t_{\text{3D}}=\{t_1,\dots,t_k\}$0 answer reward relative to the ground-truth answer (Chen et al., 21 Oct 2025). The paper emphasizes that these rewards are distributed across tokens in the trajectory, including the 3D latent tokens, so the whole chain is shaped by outcome supervision rather than explicit supervision of intermediate 3D steps (Chen et al., 21 Oct 2025).

A plausible implication is that the second stage is not merely answer optimization; it is an attempt to preserve and sharpen latent geometry by making successful reasoning trajectories depend on geometrically useful internal states.

5. Empirical performance and ablation findings

The framework is evaluated on MindCube-Tiny and Ego3D-Bench, and more broadly on VSI-Bench, SPBench, CV-Bench, SPAR-Bench, ViewSpatial-Bench, and MMSI-Bench (Chen et al., 21 Oct 2025). On MindCube-Tiny and Ego3D-Bench, the reported pattern is that stage 1 improves the base VLM substantially, and stage 1 plus stage 2 improves it further (Chen et al., 21 Oct 2025).

For Qwen2.5-VL-3B, the base model scores 33.2 overall on MindCube-Tiny, stage 1 raises this to 62.7, and stage 1 plus stage 2 reaches 75.2. On Ego3D-Bench, the same base rises from 39.1 to 46.7 after stage 1 and to 50.8 after stage 2 (Chen et al., 21 Oct 2025). The best model reported, $t_{\text{3D}}=\{t_1,\dots,t_k\}$1, achieves 77.1 on MindCube-Tiny overall and 70.0 on Ego3D-Bench average (Chen et al., 21 Oct 2025). On broader spatial benchmarks, the paper reports that for Qwen2.5-VL-3B, average performance rises from 37.5 for the plain base model to 55.3 after stage 1 and 60.4 after stage 2; for Qwen2.5-VL-7B, it rises from 41.1 to 59.4 and then 64.7 (Chen et al., 21 Oct 2025).

The ablations are central to the method’s interpretation.

Ablation topic	Reported finding	Significance
Latent size	Best performance is around latent size 12	Too small limits capacity; too large degrades answers (Chen et al., 21 Oct 2025)
Token placement	Proper placement yields 75.2; middle placement drops to 42.0	Middle insertion disrupts language coherence (Chen et al., 21 Oct 2025)
Projector direction	VLM $t_{\text{3D}}=\{t_1,\dots,t_k\}$2 VGGT gives 75.2 vs. 74.1 for the reverse direction	Supports interpretability and slightly better accuracy (Chen et al., 21 Oct 2025)
Reward removal	Removing $t_{\text{3D}}=\{t_1,\dots,t_k\}$3 drops to 68.3; removing $t_{\text{3D}}=\{t_1,\dots,t_k\}$4 drops to 64.2	Both geometry preservation and final-answer supervision matter (Chen et al., 21 Oct 2025)

These experiments support the claim that 3DThinker is not merely benefiting from longer chain-of-thought or generic RL. The paper reports that raw-QA SFT gives 52.3 overall on MindCube-Tiny with Qwen2.5-VL-3B, CoT SFT gives 53.4, plain cognitive-map SFT reaches 60.8, while 3DThinker stage 1 achieves 62.7 and stage 1 plus stage 2 reaches 75.2 (Chen et al., 21 Oct 2025). This suggests that the gain is associated with the particular latent-3D formulation rather than with supervision alone.

6. Interpretation, related frameworks, and technical significance

A distinctive aspect of 3DThinker is that the latent 3D tokens can be projected into VGGT space and decoded through VGGT’s DPT module to point clouds. The reconstructed point clouds roughly resemble the scene and tend to be sharpest around prompt-relevant objects (Chen et al., 21 Oct 2025). The paper interprets this as evidence that the latent is semantically and geometrically meaningful rather than arbitrary hidden-state noise (Chen et al., 21 Oct 2025).

This makes 3DThinker part of a broader research trajectory in which spatial intelligence is increasingly understood as requiring explicit or semi-explicit geometric structure. However, the mechanisms differ substantially across frameworks.

Think3D uses a 3D reconstruction backend, explicit camera poses $t_{\text{3D}}=\{t_1,\dots,t_k\}$5, a cleaned colored point cloud $t_{\text{3D}}=\{t_1,\dots,t_k\}$6, and novel-view rendering $t_{\text{3D}}=\{t_1,\dots,t_k\}$7 to turn spatial reasoning into an external interactive 3D exploration problem (Zhang et al., 19 Jan 2026). GeoThinker treats geometry as evidence that should be selectively retrieved via frame-strict cross-attention and Importance Gating, achieving a peak score of 72.6 on VSI-Bench (Li et al., 5 Feb 2026). DeepThink3D separates perception, reasoning, and execution, training an LLM to build executable toolchains over APIs for Scene Description, Object Filtering, Object Querying by Relation, and Object Information Querying, and reports 62.11% accuracy on SQA3D (Song et al., 21 Aug 2025).

Against this background, 3DThinker can be understood as a latent-space counterpart to explicit-geometry and tool-augmented systems. It neither reconstructs a point cloud for inference-time manipulation nor queries an external geometry encoder at each reasoning step. Instead, it teaches the VLM to carry a compact internal 3D scene state. This suggests a distinct research program: intrinsic 3D reasoning as hidden-state organization rather than as tool use or multimodal fusion.

A potential misconception is that 3DThinker is simply another external-geometry method because it uses VGGT. The paper’s formulation is narrower: VGGT is used during supervision and for interpretability, but the method’s principal claim is precisely that it requires no 3D prior input at inference (Chen et al., 21 Oct 2025).

7. Limitations and open directions

The limitations are explicitly stated. First, the recovered 3D mental representations are not autoregressively fed back into the model; they are extracted from hidden states rather than becoming part of a unified token stream (Chen et al., 21 Oct 2025). Second, the current method does not explicitly model iterative 3D mentaling across multiple reasoning steps (Chen et al., 21 Oct 2025). The authors propose that a unified tokenizer or a more integrated latent-text architecture could improve this, and they also identify iterative latent imagination inside the reasoning trajectory as future work (Chen et al., 21 Oct 2025).

These limitations are significant because they mark the boundary between 3DThinker and frameworks such as Think3D. Think3D already performs explicit multi-step spatial exploration by repeatedly manipulating a reconstructed scene and appending newly rendered observations to memory (Zhang et al., 19 Jan 2026). 3DThinker, by contrast, has an internal latent that is not yet a fully interactive spatial workspace. A plausible implication is that future systems may combine intrinsic latent 3D scene formation with explicit iterative manipulation, or may unify the latent and action spaces into a single multimodal reasoning loop.

Within the literature cited here, 3DThinker’s main historical importance is that it reframes spatial reasoning as something a VLM can internalize without external 3D inputs at inference, while still maintaining a concrete geometric grounding during training (Chen et al., 21 Oct 2025). Its contribution is therefore not only benchmark improvement, but a representational proposal: that multimodal reasoning can be made more spatial by inserting a geometrically aligned latent scene state directly into the model’s chain of thought.