View-Centric Scene Space: Geometry & Fusion

• View-centric scene space is a coordinate- and representation-centered paradigm that parameterizes spatial queries by an agent’s viewpoint, preserving 3D structure.
• It employs methodologies like camera projection, ray-based attention, and homography transforms to fuse heterogeneous, multi-view sensory data for tasks such as SLAM and tracking.
• Empirical results show improved loop closure, occlusion handling, and multi-view consistency, yielding gains in video prediction, object tracking, and visual grounding.

A view-centric scene space is a coordinate- and representation-centric paradigm in which sensory observations, predictions, and spatial reasoning are parameterized and aligned by the agent’s viewpoint(s), rather than by a fixed global reference (“world-centric”) system. This approach underlies a wide array of recent models for visual reasoning, scene reconstruction, multi-object tracking, video modeling, navigation, visual grounding, and interactive planning. Across literature, instantiations of “view-centric” spaces consistently aim to preserve 3D spatial structure, enable viewpoint-consistent retrieval or reasoning, maintain robustness under camera motion (including loop closure), and integrate heterogeneous, partial, or occluded observations into a unified latent space tied to epistemic accesses (views, rays, or camera frames) (Xiang et al., 8 Feb 2026, Zhan et al., 8 Mar 2026, Liu et al., 2023, Ji et al., 2024, Matsuki et al., 2023, Ke et al., 17 Mar 2026, Wei et al., 2024, Wang et al., 28 May 2026, Li et al., 2024, Huang et al., 15 Jul 2025).

1. Mathematical and Geometric Foundations

The formalism of view-centric scene spaces is built on expressing all spatial queries, features, and attention mechanisms with explicit knowledge of camera parameters.

• Camera projection and back-projection:
  • Camera $i$ is parameterized by intrinsics $K_i \in \mathbb{R}^{3 \times 3}$ and extrinsics $(R_i, t_i)$.
  • Pixel to world: $$X = R_i \left( d_i(u,v) \cdot K_i^{-1}[u,v,1]^T \right) + t_i$$ (Ke et al., 17 Mar 2026).
  • For ray-based parameterization (e.g., ViewRope), each token at $(u, v)$ in image $i$ is mapped to a unit 3D ray $r_{i, u, v}$ via $K_i^{-1}$ and $R_i$ (Xiang et al., 8 Feb 2026).
• View-anchored local coordinate systems:
  • In SLAM and mapping, view-centric representations allocate local frames attached to keyframes, enabling drift-robust mapping under loop closure (Matsuki et al., 2023).
  • For reasoning, models often alternate between egocentric (camera-centered) and allocentric (world-aligned/projected) views, or use orthographic projections to bridge the gap (Zhan et al., 8 Mar 2026).
• Homography and multi-view transforms:
  • In multi-object tracking, inter-frame homographies $H \in \mathbb{R}^{3 \times 3}$ allow mapping object locations and features into a shared plane for robust matching and ID assignment (Ji et al., 2024).

View-centricity is thus instantiated by a systematic alignment of all geometric, spatial, and functional queries to the agent’s current or recent viewpoints, permitting flexible fusion or decoupling from global frames.

2. Core Methodologies and Representations

The operationalization of view-centric scene spaces varies by downstream task but follows certain thematic patterns:

• Ray- and View-Parameterized Attention:
  • In ViewRope, transformer queries/keys encode per-token camera-ray orientations through rotary embedding, such that self-attention computes geometric dot-products in rotation space: $K_i \in \mathbb{R}^{3 \times 3}$0 (Xiang et al., 8 Feb 2026).
• Multi-View Fusion via Local/Global Grids:
  • Navigation systems convert per-view 2D features into local and global bird’s-eye-view (BEV) grids, maintaining both per-view and globally fused graphs for robust action selection (Liu et al., 2023, Li et al., 2024).
• Homography-Based Feature and Box Alignment:
  • Motion tracking models employ fast homography estimation (DLT/interpolation) to register features and bounding boxes for data association, identity learning, and IoU computation, crucial for UAV tracking (Ji et al., 2024).
• Multi-Branch Gaussian Representations:
  • Omni-Scene represents ego-centric scenes using both volume-based and pixel-anchored Gaussians, enabling coverage of both occluded/visible regions while maintaining spatial detail and overcoming minimal overlap scenarios (Wei et al., 2024).
• Structured Multi-View Embedding:
  • Visual grounding frameworks (e.g., ViewSRD) construct joint embeddings for each simulated view, integrating text and scene features through attention to assemble a latent “view-centric” space, which supports disentangled grounding of complex multi-anchor queries (Huang et al., 15 Jul 2025).

3. Training Objectives and View-Consistency Losses

A view-centric formulation mandates explicit or implicit losses that enforce cross-view consistency:

• Loop-Closure and Geometric Consistency:
  • Video world models evaluate and encourage revisit consistency with loop-closure losses that warp predictions between frames using depth and known pose (Xiang et al., 8 Feb 2026).
  • Point cloud and SLAM systems maintain inter-frame consistency by decoupling geometry updates (pose graphs) from local field features, so local correction occurs via transform update, not destructive field rewriting (Matsuki et al., 2023).
• Projection/Multi-View Anchoring Supervision:
  • Multi-object trackers incorporate both standard detection loss and slot reconstruction losses, with ID-association constraints in the presence of cross-view correspondences (via homographies) (Ji et al., 2024).
  • Multi-view grounding models employ object- and anchor-supervised losses, as well as sentence-level consistency, to tie language and observation spaces (Huang et al., 15 Jul 2025).
• View-Consistent Orthographic and Occupancy Modeling:
  • Orthographic projections are supervised for cross-view 3D agreement; in occupancy perception, learnable view-guided transformers aggregate voxel, temporal, and camera-aligned regions for both static and dynamic scene structure supervision (Zhan et al., 8 Mar 2026, Li et al., 2024).

4. Applications: Reasoning, Planning, Mapping, and Tracking

View-centric scene spaces have been deployed across diverse settings:

Area	Key Models/Techniques	Primary Contribution
Video world modeling	ViewRope (Xiang et al., 8 Feb 2026)	3D-consistent, long-term video prediction
Spatial reasoning (VL)	3ViewSense (Zhan et al., 8 Mar 2026), VIEW2SPACE (Ke et al., 17 Mar 2026)	Robust multi-view spatial reasoning/aggregation
Navigation	BSG (Liu et al., 2023), ViewFormer (Li et al., 2024)	BEV-based decision, multi-view occupancy/flow modeling
Multi-object tracking	HomView-MOT (Ji et al., 2024)	Homography-registered cross-view association
Scene mapping/SLAM	NEWTON (Matsuki et al., 2023)	Loop-closure robust, scalable map construction
Visual grounding	ViewSRD (Huang et al., 15 Jul 2025)	Multi-view decomposed scene-language alignment
View planning/embodied AI	Self-exploration + view graphs (Wang et al., 28 May 2026)	Compositionally robust view-action planning
Sparse-view reconstruction	Omni-Scene (Wei et al., 2024)	Hybrid pixel/volume ego-centric representation

These applications leverage the flexibility of view-centric spaces to address spatial drift, identity preservation, geometric consistency, and multi-source fusion.

5. Empirical Evaluation and Performance Benchmarks

View-centric approaches display consistent empirical advantages:

• World Modeling: ViewRope reduces loop closure error (LCE) by 4.5%–16% and maintains or improves PSNR/SSIM across rotation and long-horizon context (Xiang et al., 8 Feb 2026).
• Spatial Reasoning: 3ViewSense surpasses baselines by $K_i \in \mathbb{R}^{3 \times 3}$1 on occlusion-heavy, view-consistent reasoning (Zhan et al., 8 Mar 2026); VIEW2SPACE’s grounded chain-of-thought yields 319% relative improvement on multi-view detection (Ke et al., 17 Mar 2026).
• Object Tracking: HomView-MOT achieves +6.8% IDF1 on VisDrone and UAVDT, with view-centric ID learning and homographic filters critical for identity preservation (Ji et al., 2024).
• SLAM: NEWTON’s view-centric map maintains stable depth and appearance metrics under global loop closure, unlike world-centric fields (Matsuki et al., 2023).
• Occupancy/Flow Perception: ViewFormer’s view-guided attention improves mIoU by 2.7–5.2 points and reduces training/inference cost in BEV-based occupancy and flow prediction (Li et al., 2024).
• Visual Grounding: ViewSRD leads prior 3D grounding models by 5.2–14.9% on view-dependent or multi-anchor queries (Huang et al., 15 Jul 2025).
• Sparse-View Reconstruction: Omni-Scene outperforms prior methods in ego-centric settings and matches scene-centric performance under minimal overlap (Wei et al., 2024).

Ablations across works universally demonstrate that view-centric alignment, attention, or fusion is causally necessary—removal always degrades 3D consistency or task success.

6. Theoretical and Practical Implications

The persistence of ambiguity, drift, and erroneous fusion in world-centric or screen-space approaches is directly linked to insufficient alignment with the geometry of viewpoint access. View-centric scene spaces mitigate these issues by:

• Providing inductive bias for geometric persistence and revisit fidelity (critical for loop closure, tracking, and video prediction) (Xiang et al., 8 Feb 2026, Matsuki et al., 2023).
• Enabling sparse-view integration in scenarios with occlusion, truncation, or minimal overlap (e.g., ego-centric vehicles, multi-UAV) (Wei et al., 2024, Ji et al., 2024).
• Supporting compositional reasoning across multi-anchor, multi-hop, or planning tasks (decomposition into single-view or orthographic queries, explicit multi-view graphs) (Zhan et al., 8 Mar 2026, Huang et al., 15 Jul 2025, Wang et al., 28 May 2026).

A plausible implication is that future multi-modality, planning, and bio-realistic modeling pipelines will increasingly adopt explicit view-centric scene spaces both for efficiency and for spatial consistency, especially under challenging real-world deployment.

7. Limitations, Open Problems, and Frontiers

Current limitations and challenges in view-centric scene spaces include:

• Map size scaling: View-centric (multi-field) SLAM/map representations grow linearly with trajectory length; solutions such as field merging/pruning or hierarchical pose graphs remain open (Matsuki et al., 2023).
• Dynamic scene handling: Most frameworks assume static geometry; per-object or dynamic field extensions are an active area for future work (Matsuki et al., 2023).
• Compositional reasoning depth: Difficulty scaling analyses in VIEW2SPACE indicate that deep multi-hop reasoning beyond 3–4 steps quickly saturates accuracy, suggesting an intrinsic limit on fusion depth under occlusion (Ke et al., 17 Mar 2026).
• Integration with language and high-level cognition: Despite clear gains in view-centric visual spatial modeling, connection with compositional semantics, pragmatic reasoning, or long-term memory in language agents remains nascent (Zhan et al., 8 Mar 2026, Huang et al., 15 Jul 2025, Wang et al., 28 May 2026).
• Scene understanding in complex agent-environment loops: Active planning and continual scene access (as in view planning tasks) are only partially solved; distilled view graphs markedly improve multi-turn performance, but long-horizon synthesis remains challenging (Wang et al., 28 May 2026).

This suggests that while view-centric scene space realization constitutes a fundamental pivot in spatial AI, principled scalability, full dynamic handling, and integration with generalized reasoning are outstanding research directions.