Global-to-Local Spatial Grounding

• Global-to-local spatial grounding is a framework that integrates broad scene context with precise local spatial resolution to enable targeted localization.
• Key methodologies include scene-wide relational reasoning, prune-then-ground strategies, and explicit fusion of global and local outputs.
• Empirical results across domains such as 3D object, medical, and remote sensing tasks show that transitioning from global context to local commitment significantly boosts performance.

Global-to-local spatial grounding denotes a family of grounding procedures in which a model first uses scene-wide context—such as all objects and their pairwise relations, a language-relevant subregion, a global mask, a block-level route position, or a globally encoded 3D scene—and then resolves a local target region, object, box, or feasible space mask. Recent work instantiates this pattern in 3D object grounding, monocular 3D reasoning, medical visual grounding, remote sensing, embodied pointing, navigation, video-language modeling, and even text-only verbalized spatial reasoning, but the common structure remains a coarse-to-fine transition from global spatial interpretation to local spatial commitment (Chen et al., 2022, Dinh et al., 30 Jun 2026, Gao et al., 2 Apr 2026, Cheng et al., 28 May 2026).

1. Conceptual scope and historical development

A minimal definition emerges from several task formulations. In 3D object grounding, the input is a 3D scene as a point cloud, a natural-language expression, and a set of object proposals, and the output is the index of the referred object together with its 3D box. In one-stage 3D visual grounding, the output becomes a continuous 3D box $\langle c,\; p_x, p_y, p_z,\; s_x, s_y, s_z\rangle$. In remote sensing, the model predicts a set of oriented boxes $B \in \mathbb{R}^{L \times 5}$. In medical visual grounding, the target is a set of phrase labels and corresponding boxes $y_b$. In tabletop space grounding, the output is a binary space mask $\mathcal{M}_\Lambda$. In outdoor navigation, the grounded output is an action sequence over $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$ (Chen et al., 2022, Dinh et al., 30 Jun 2026, Toker et al., 9 Dec 2025, Gao et al., 2 Apr 2026, Oh et al., 19 Nov 2025, Tian et al., 2024).

Setting	Grounded output	Representative paper
3D object grounding	Object index and 3D box $B_T \in \mathbb{R}^6$	(Chen et al., 2022)
One-stage 3DVG	$\langle c,\; p_x,p_y,p_z,\; s_x,s_y,s_z\rangle$	(Dinh et al., 30 Jun 2026)
Remote sensing grounding	$B \in \mathbb{R}^{L \times 5}$ oriented boxes	(Toker et al., 9 Dec 2025)
Medical visual grounding	Phrase regions $y_b$	(Gao et al., 2 Apr 2026)
Space grounding	Binary mask $\mathcal{M}_\Lambda$	(Oh et al., 19 Nov 2025)
Outdoor VLN	Action sequence	(Tian et al., 2024)

The explicit separation between global and local reasoning has a clear antecedent in grid-world spatial reasoning. A language-conditioned representation was factored into an instruction-conditioned local convolutional map $B \in \mathbb{R}^{L \times 5}$0 and an instruction-conditioned global gradient map $B \in \mathbb{R}^{L \times 5}$1, and the combined value map $B \in \mathbb{R}^{L \times 5}$2 supported both local references such as “two cells left of the triangle” and global references such as “westernmost rock” (Janner et al., 2017). This early factorization already exhibited the canonical intuition of the later literature: local predicates are often easy once the globally relevant frame, anchor, or region has been identified.

A recurring conceptual distinction is that global context is not merely “more input.” In these systems it is usually a structured spatial prior: pairwise relations over all objects, a scene-wide mask, a pruned 3D sub-cloud, a block-progress coordinate, or a globally pooled scene representation. Local grounding is the subsequent commitment to a single box, mask, patch set, or action.

2. Recurrent architectural patterns

One major pattern is scene-wide relational reasoning followed by object-level selection. ViL3DRel embodies this directly: all objects are tokens, pairwise distances and orientations are encoded as $B \in \mathbb{R}^{L \times 5}$3, a language-conditioned relevance score $B \in \mathbb{R}^{L \times 5}$4 modulates standard self-attention, and the model ends with a local grounding head that scores each proposal. The resulting “global relational view” is refined across layers until a single target box is selected (Chen et al., 2022). DASANet preserves the same overall transition but decouples attribute and spatial streams: it separately aligns $B \in \mathbb{R}^{L \times 5}$5 with $B \in \mathbb{R}^{L \times 5}$6 and $B \in \mathbb{R}^{L \times 5}$7 with $B \in \mathbb{R}^{L \times 5}$8, then combines them as $B \in \mathbb{R}^{L \times 5}$9, so the final local score is explicitly the sum of an attribute term and a spatial-relation term (Xu et al., 2024).

A second pattern is prune-then-ground. PruneGround starts from the full point cloud $y_b$0, renders a top view and four oblique RGB+depth views, asks a frozen VLM for a language-relevant top-view box $y_b$1, projects that box back to 3D to obtain a pruned point cloud $y_b$2, reformulates the description into target–anchor statements, and only then performs local grounding inside $y_b$3 (Dinh et al., 30 Jun 2026). C2F-Space follows the same logic in 2D tabletop manipulation: a VLM first predicts a coarse canonical region $y_b$4 as one or more ellipses on a grid-inpainted image, and a superpixel graph then refines that region to the final mask $y_b$5 (Oh et al., 19 Nov 2025).

A third pattern is explicit fusion of global and local branches. KnowMVG constructs a global branch by using phrase-conditioned knowledge prompts to drive a frozen SAM decoder and obtain a global mask $y_b$6, while a local branch converts a coarse box $y_b$7 into sparse SAM-style prompts $y_b$8 and refines local evidence through a box decoder. The two streams are fused as $y_b$9, with the final box predicted from $\mathcal{M}_\Lambda$0 (Gao et al., 2 Apr 2026). VistaRef uses the same broad principle for pointing-based grounding: a global visual-language context predicts hand keypoints, those keypoints define an explicit ray, and a local hand feature plus ray embedding forms a ray-aware query that scans the global patch set through cross-attention before regressing the target box (Li et al., 23 Jun 2026).

A fourth pattern is grounding-through-generation. SATGround augments the vocabulary with $\mathcal{M}_\Lambda$1 and $\mathcal{M}_\Lambda$2, letting a VLM decide when to emit a box and using the hidden state at $\mathcal{M}_\Lambda$3 as input to a separate grounding head $\mathcal{M}_\Lambda$4 (Toker et al., 9 Dec 2025). GR3D instead inserts grounded region tokens into the language stream during generation, so that later reasoning tokens can attend to visual evidence tied to specific entity mentions (Cheng et al., 28 May 2026). GS-Reasoner makes the same autoregressive idea fully 3D: it emits <bbox>(x_1, y_1, z_1, x_2, y_2, z_2)</bbox> subsequences inside its chain of thought and then reasons over those generated coordinates without external grounding modules (Chen et al., 15 Oct 2025).

3. Spatial representations and coordinate systems

The most common 3D representation is an object-centered one based on centers, sizes, and relative geometry. ViL3DRel computes object centers $\mathcal{M}_\Lambda$5, sizes $\mathcal{M}_\Lambda$6, and pairwise relations through Euclidean distance and horizontal and vertical angles, encoding both absolute location and pairwise orientation in the transformer stack (Chen et al., 2022). N3D-VLM instead begins from dense geometry: a monocular depth map $\mathcal{M}_\Lambda$7 and camera intrinsics produce a dense point cloud by back-projection,

$\mathcal{M}_\Lambda$8

and 3D sinusoidal encodings are fused with image features before the model autoregressively generates 3D boxes of the form $\mathcal{M}_\Lambda$9 (Wang et al., 18 Dec 2025).

A closely related but more token-efficient formulation appears in GS-Reasoner. There, RGB frames provide image-patch semantics, depth and camera parameters produce point maps and a global point cloud, a PTv3-based geometric encoder extracts scene-level geometric features, and dual-path pooling aligns those geometric features with both patch semantics and patch position. The final hybrid patch token combines semantic features, geometry, and 3D positional encoding without increasing the number of input tokens (Chen et al., 15 Oct 2025).

In video-language modeling, the local units are not boxes but group tokens. S-ViLM introduces $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$0 learnable group tokens, repeatedly updates them through grouping blocks, and uses them as semantic regions over the whole clip. Spatial grounding is then a noun-to-region alignment problem in which noun embeddings attend to these region tokens, so local region-object alignment is learned against a globally encoded video (Xiong et al., 2023).

Navigation work uses a different spatial coordinate system but preserves the same global-to-local logic. Loc4Plan defines a block-aware progress variable

$$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$1

where $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$2 represents relative position within the current street block. This scalar is not a box, but it serves the same role as a coarse spatial prior: it drives sentence-level instruction grounding before local action selection (Tian et al., 2024).

Even text-only models can instantiate grounded spatial reasoning when coordinates are made explicit. In the verbalized VSR setting, each object is encoded as a sequence $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$3, where the discretized coordinates belong to $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$4. The LLM then converts a global list of location tokens into local relational judgments such as “left of,” “inside,” or “overlapping” (Azkune et al., 2024).

4. Supervision, distillation, and grounded reasoning traces

Teacher–student transfer is a recurrent mechanism when raw visual inputs are noisy. ViL3DRel first trains a teacher on clean symbolic inputs—ground-truth class labels embedded by GloVe and color embeddings derived from a 3-component Gaussian Mixture Model—and then distills its relational structure into a point-cloud student through attention matching

$$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$5

and hidden-state matching

$$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$6

combined with grounding and auxiliary classification losses (Chen et al., 2022). DASANet adopts a comparable logic: the teacher uses auxiliary losses $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$7, $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$8, and $$\{\text{FORWARD}, \text{LEFT}, \text{RIGHT}, \text{STOP}\}$$9, while the student receives $B_T \in \mathbb{R}^6$0 on top of the main grounding, foreground, and text losses (Xu et al., 2024).

Loss design often enforces consistency between global priors and local spatial commitments. In KnowMVG, the objective is

$B_T \in \mathbb{R}^6$1

where $B_T \in \mathbb{R}^6$2 is phrase classification cross-entropy and $B_T \in \mathbb{R}^6$3 combines Smooth $B_T \in \mathbb{R}^6$4 and GIoU losses on boxes predicted from the global–local fused representation $B_T \in \mathbb{R}^6$5 (Gao et al., 2 Apr 2026). VistaRef uses

$B_T \in \mathbb{R}^6$6

with $B_T \in \mathbb{R}^6$7, so that hand presence, finger keypoints, and ray consistency all support the same local target box (Li et al., 23 Jun 2026). C2F-Space supervises the local stage differently: the superpixel graph predicts a residual over coarse canonical-region logits and is trained with focal loss on superpixel labels, so the refinement learns deviations from a global ellipse prior rather than unconstrained segmentation (Oh et al., 19 Nov 2025).

Another important line of work moves supervision from final answers to grounded reasoning traces. GR3D trains explicit and implicit 2D grounding together with region-prompted monocular 3D grounding, then teaches the model to generate spatial chain-of-thought in which grounded region tokens are inserted into the text stream and 3D boxes are generated as part of the answer sequence (Cheng et al., 28 May 2026). GS-Reasoner’s GCoT dataset pushes this further: chain-of-thought examples explicitly list relevant objects and 3D boxes before reasoning about distances, directions, room size, route planning, or appearance order, and the model is trained end-to-end with next-token cross-entropy over the entire sequence (Chen et al., 15 Oct 2025).

5. Empirical findings across domains

The reported gains are task-specific and not directly comparable across domains, because the outputs range from proposal selection to continuous 3D boxes, masks, and action sequences. Even so, a consistent empirical pattern appears: once global context is converted into an explicit local prior, task performance usually improves.

Setting	Representative result	Paper
3D object grounding	ViL3DRel reaches 64.4% on Nr3D and 72.8% on Sr3D, versus 55.1% and 64.5% for the best previous Multi-view Transformer	(Chen et al., 2022)
Dense 3D object grounding	3DOGSFormer reaches 49.29 overall @0.5 on ScanRefer, 73.5 on Nr3D, and 80.8 on Sr3D	(Huang et al., 2023)
Prune-then-ground 3DVG	PruneGround reports 63.8% / 56.1% on ScanRefer Overall [email protected] / [email protected], 75.1% on Nr3D, and 81.5% on Sr3D	(Dinh et al., 30 Jun 2026)
Medical visual grounding	KnowMVG reports gains of 3.0% in AP50 and 2.6% in mIoU over prior state-of-the-art methods	(Gao et al., 2 Apr 2026)
Remote sensing grounding	SATGround reports a 24.8% relative improvement over previous methods on visual grounding	(Toker et al., 9 Dec 2025)
Pointing-to-object detection	VistaRef reports [email protected] 0.9393, [email protected] 0.8261, and mIoU 0.8201 on EgoPoint-Ground	(Li et al., 23 Jun 2026)

Ablation evidence is especially informative. In ViL3DRel’s teacher model on Nr3D, a strong baseline without spatial self-attention reaches 62.4%, while full spatial self-attention with distance, orientation, multi-head attention, and sigsoftmax reaches 74.4%; on the student side, moving from no teacher and no distillation to full attention-plus-hidden-state distillation raises accuracy from 58.1% to 64.4% (Chen et al., 2022). DASANet reports 65.1% on Nr3D, exceeding the 63.8% of ViL3DRel and doing particularly well on hard and view-dependent subsets, which is consistent with its explicit attribute–spatial decomposition (Xu et al., 2024).

Region restriction and ambiguity reduction also show measurable effects. PruneGround reduces the average number of candidates on ScanRefer Multiple from 5.72 to 1.62 and on Sr3D Hard from 2.43 to 1.28; with four oblique RGB+depth views, its pruning stage reaches 94.7% target recall, and its ablations attribute +3.2 [email protected] on ScanRefer to LGSP alone and another +2.1 to MCDR simplification plus augmentation (Dinh et al., 30 Jun 2026). In C2F-Space, removing the superpixel refinement barely changes success rate but more than halves IoU, while removing the grid from the global stage causes an approximately 11.5% drop in success rate, showing that coarse semantic correctness and local geometric precision are carried by different stages (Oh et al., 19 Nov 2025).

Several papers also show that explicit grounding improves reasoning beyond grounding benchmarks. N3D-VLM reaches 89.7 on open-ended and 92.1 on numerical questions on N3D-Bench, while a QA-only variant of the same architecture reaches 80.6 and 62.4, respectively; moreover, simply giving Qwen3-VL-8B N3D-VLM’s 3D grounding as input raises N3D-Bench numerical performance from 36.3 to 54.6 (Wang et al., 18 Dec 2025). GS-Reasoner reports 64.7 average on VSI-Bench with predicted depth and 70.1 with ground-truth depth, while also remaining competitive on 3D grounding benchmarks without external modules (Chen et al., 15 Oct 2025). In the text-only setting, spatially trained T5-3B with location tokens reaches 74.52 on VSR and surpasses several vision-language baselines, indicating that explicit coordinates alone can ground local relational judgments when the model is trained to use them (Azkune et al., 2024).

6. Limitations, misconceptions, and research trajectories

A recurrent misconception is that strong global scene modeling is sufficient by itself. Two lines of evidence directly oppose this view. In grounded visual spatial reasoning, correct image–text matching can occur even when the subject and object are not correctly localized, so global decisions need not be grounded in local evidence (Rajabi et al., 2023). In hallucination mitigation, Active-Look shows that a single visual operator is inadequate: highlight preserves global context but can miss small-object detail, zoom-in improves local detail but can break global relations, and noisy local proposals can reduce performance below standard prompting, with reported drops to 48.2% on the Simple subset and 47.1% on the Complex subset in the “TwI (Noisy)” setting (Jiang et al., 27 Apr 2026).

Several technical limitations recur across domains. Two-stage 3D grounding still depends on object proposals, so missed targets or missed anchors can make grounding impossible; ViL3DRel explicitly notes failures when detectors miss outlets or thin TVs, and performance drops substantially from ground-truth to detected proposals (Chen et al., 2022). PruneGround inherits stage-wise error propagation: bad pruning can remove the target or anchors, while reformulation errors can misguide the grounder (Dinh et al., 30 Jun 2026). Monocular 3D models remain sensitive to depth estimation, reflections, and clutter; N3D-VLM reports failures on reflective water and dense jellyfish scenes (Wang et al., 18 Dec 2025). Loc4Plan is tied to block structure in outdoor street graphs and to pseudo labels for sentence relevance, which limits the portability of its locating-before-planning strategy beyond similar environments (Tian et al., 2024). KnowMVG emphasizes that latent-token prompting alone lacks explicit localization priors, but its own gains still depend on the robustness of pretrained VLM and SAM components (Gao et al., 2 Apr 2026).

The forward trajectory of the literature is therefore not a simple move toward larger end-to-end models. The explicit directions proposed in these works include 3D scene graphs and richer geometric features for 3D grounding, multi-level scene $B_T \in \mathbb{R}^6$8 room $B_T \in \mathbb{R}^6$9 local-region hierarchies for prune-then-ground systems, richer spatial outputs such as masks or polygonal regions in satellite and tabletop settings, dynamic or temporal extensions, and more systematic grounding-first chain-of-thought supervision (Chen et al., 2022, Dinh et al., 30 Jun 2026, Oh et al., 19 Nov 2025, Chen et al., 15 Oct 2025). A plausible implication is that future systems will continue to hybridize global priors with local commitments rather than collapsing the two into a single undifferentiated attention pass.