ViewBench: Spatial AI Benchmarks

• ViewBench is a comprehensive family of benchmarks that evaluates view-conditioned reasoning, novel view generation, spatial localization, and 3D consistency in AI systems.
• It encompasses four key suites targeting single-image near-view reasoning, multi-perspective spatial localization, multi-view generation, and loop-closure in video world models.
• Empirical findings emphasize the advantages of token-level warping, dedicated allocentric training, and balanced 2D versus 3D evaluations for robust spatial cognition.

ViewBench is a family of benchmarks and evaluation protocols designed to rigorously quantify and advance view-conditioned reasoning, novel view generation, spatial localization, and 3D consistency in machine learning systems. The term “ViewBench” spans several independently developed suites, each targeting a distinct facet of spatial cognition and geometric consistency in vision-LLMs, multimodal LLMs, multi-view generative models, and video world models. Collectively, ViewBench benchmarks enable systematic ablation and comparison of methods under hard, controlled viewpoint shifts, view-conditioned reasoning tasks, and long-horizon trajectory traversals.

1. Taxonomy of ViewBench Variants

The “ViewBench” label encompasses at least four major publicly documented suites, each crystallizing a different analytical goal:

1. Single-Image, Near-View Reasoning for MLLMs: A benchmark probing whether multimodal LLMs can maintain spatial and semantic consistency under controlled small camera shifts using real indoor ScanNet data, with token- and pixel-level warping baselines (Lee et al., 3 Apr 2026).
2. Multi-Perspective Spatial Localization for VLMs: “ViewSpatial-Bench” (sometimes named “ViewBench” in the literature) targets egocentric-vs-allocentric spatial frame generalization, quantifying how well VLMs re-anchor spatial relationships from the camera’s or a human’s viewpoint (Li et al., 27 May 2025).
3. Multi-View Generation Consistency for MVG Models: A protocol suite built atop MVGBench, for evaluating geometry and texture consistency, semantic fidelity, and robustness of multi-view generative models using synthetic and real object datasets (Xie et al., 11 Jun 2025).
4. Loop-Closure Consistency in Video World Models: Diagnostic tasks emphasizing the preservation of 3D scene structure over long, closed-loop camera trajectories, with metrics for spatial drift and recurrence failure (Xiang et al., 8 Feb 2026).

Each suite is engineered to expose brittle failure modes of state-of-the-art models and to provide granular, model- and data-agnostic metrics beyond standard frame- or image-level accuracy.

2. Motivations and Core Evaluation Challenges

ViewBench was formulated in response to systemic deficiencies in existing spatial reasoning and generative evaluation methodologies:

• Fragility to Viewpoint Change: MLLMs and VLMs are brittle under even small camera movements, failing view-invariant reasoning and hallucinating or omitting details when required to “imagine” a new view from a single photo (Lee et al., 3 Apr 2026).
• Egocentric/Allocentric Biases: Pretraining on large-scale image–text pairs imparts strong egocentric priors, leading to poor transfer when spatial queries are re-anchored to another agent’s perspective (Li et al., 27 May 2025).
• Limitations of 2D Metrics: Existing MVG evaluations rely on direct image-wise PSNR or SSIM against ground truth, which penalizes plausible generative outputs and overlooks whether novel views are mutually consistent in 3D (Xie et al., 11 Jun 2025).
• Spatial Persistence in World Models: Standard video prediction benchmarks do not capture the geometric drift and loss of consistency as a camera follows long or loop-closure trajectories (Xiang et al., 8 Feb 2026).

By introducing explicit view manipulation, controlled spatial transformations, 3D self-consistency metrics, and loop-closure diagnostics, ViewBench variants provide rigorous testing beds for spatially-aware AI.

3. Task Definitions and Benchmark Structure

The suite-specific task definitions are as follows:

• Single-Image Near-View Reasoning: Each example consists of a source image with depth, a relative pose, and a VQA-style prompt. Tasks include binary spatial relations (“Is A left or right of B from the new viewpoint?”) and open-ended object description at a marked location after a camera move. Paired source/target views are constructed with 5–35% co-visible overlap using ScanNet scenes. Models are evaluated zero-shot, using warping at inference time, not trained on ViewBench itself (Lee et al., 3 Apr 2026).
• Multi-Perspective Spatial Localization: Contains 5,712 multiple-choice questions over >1,000 scenes (ScanNet and MS-COCO), balanced between camera- and person-perspective spatial reasoning. Five main task types include camera-relative and person-relative directional reasoning, object orientation, and triadic scene simulation, with automated directional annotation via 3D transforms or body-keypoint pipelines (Li et al., 27 May 2025).
• Multi-View Generation Consistency: Evaluates MVG models on rendering N novel views of an object from controlled poses. The cornerstone metric is 3D self-consistency: two disjoint view sets each fit with 3D Gaussian Splatting (3DGS), and geometric/texture discrepancies are measured via Chamfer distance, absolute depth difference, and cross-rendered metrics. Tasks span synthetic (GSO, Omni3D) and real (CO3D, MVImgNet) datasets, with robustness tested against systematic input perturbations (Xie et al., 11 Jun 2025).
• Loop-Closure Video World Models: Comprises two suites—pure-rotation and rotation+translation trajectories—executed in photorealistic UE5 environments. Each trajectory is annotated with SE(3) camera matrices and per-frame depth. The primary metric is loop closure error (LCE), defined as the LPIPS distance between the initial and revisit frames under perfect round-trip pose matching (Xiang et al., 8 Feb 2026).

4. Metrics and Evaluation Protocols

Each ViewBench instantiation employs bespoke metrics and evaluation schemes:

• Spatial Reasoning Accuracy: For binary spatial tasks, accuracy is measured as the fraction of correct left/right classifications. For open-ended descriptions, LLM-evaluated scores (on a 1–10 scale) are used, operationalized with Qwen2.5-VL-14B as an automatic rater (Lee et al., 3 Apr 2026).
• Multiple-Choice Directional Accuracy: Evaluation is standardized as accuracy over 8 compass bins, with a random-choice baseline of 26.3%, and mean angular error as a potential continuous alternative. No training is performed on the evaluation set—split to ensure cross-scene generalization (Li et al., 27 May 2025).
• 3D Self-Consistency (MVG): Decomposes as:
  • Chamfer distance between disjoint 3DGS reconstructions
  • Mean depth difference across K test poses
  • Texture consistency: cPSNR, cSSIM, cLPIPS averaged over views.
  • Image quality: per-object CLIP FID (oFID), and VLM-based binary scoring (IQ-vlm).
  • Semantic alignment: VLM-prompted attribute and class consistency (Xie et al., 11 Jun 2025).
• Loop-Closure Error (World Models): LCE is the reference LPIPS between start and return frame; low values indicate high spatial persistence. Standard short-term metrics (PSNR, SSIM, frame-wise LPIPS) are also reported. The dataset structure ensures exact pose-return for binary revisit indicators (Xiang et al., 8 Feb 2026).
• Protocol Details: Most protocols demand zero-shot inference or test-time warping. For world models, sequences are autoregressively generated under prescribed action/camera paths with ground-truth first-frame context; code evaluation harnesses standardize interface conversion and latent caching.

5. Key Empirical Findings and Comparative Analyses

Results across ViewBench variants consistently reveal:

• Superiority of Token-Layer Warping for MLLMs: Backward token warping—dense grid in the target view, token retrieval via depth proxy—achieves highest stability for viewpoint-conditioned reasoning, outperforming pixel warping, generative synthesizers, and spatial MLLM specialists. Backward nearest-neighbor token fetching matches adaptive variants, confirming robustness (Lee et al., 3 Apr 2026).
• Allocentric Perspective Remains Unlearned Without Explicit Training: Off-the-shelf large VLMs perform barely above random on allocentric relocalization tasks. Fine-tuning with multi-perspective data (MVSM) yields absolute gains of 40–55% on all subtasks, demonstrating that data/task structure, not scale, governs spatial cognition generalization (Li et al., 27 May 2025).
• 3D Consistency-Quality Tradeoff in MVG: Methods optimized for 2D perceptual fidelity (e.g., Zero123) exhibit poor inter-view consistency; strongly consistent models often degrade per-view appearance. SV3D and ViFiGen architectures approach an optimal tradeoff. Generalization to real images remains poor across all methods, and robustness to input viewpoint/illumination shifts is limited (Xie et al., 11 Jun 2025).
• Geometric Awareness Needed for Persistent World Models: ViewRope positional encoding and geometry-aware frame-sparse attention reduce LCE by up to 11% and recover dense-attention performance at 25% computation savings. Proper patch-level ray encoding and historically-aware frame selection are critical for loop closure (Xiang et al., 8 Feb 2026).

6. Dataset Composition and Construction

A core property of all ViewBench suites is dataset precision and control:

• Synthetic/Real 3D Assets: Use of ScanNet (∼1,500 rooms), MS-COCO with body keypoints, Google Scanned Objects, Omni3D, CO3D, and MVImgNet (Lee et al., 3 Apr 2026, Li et al., 27 May 2025, Xie et al., 11 Jun 2025).
• Careful Overlap and Camera Path Management: For near-view reasoning, source/target pairs exhibit controlled co-visible 3D point overlap (down to 5%). For world models, full SE(3) rotational and translational coverage is obtained in rendered UE5 environments (Xiang et al., 8 Feb 2026).
• Automated 3D Annotation Pipelines: Directional and object orientation labels generated using coordinate transforms, keypoint-driven orientation estimation (“Orient-Anything”), and set-cover-based sampling for efficient dataset coverage (Li et al., 27 May 2025).
• Train/Test Splits: Designed for cross-scene generalization, no scene overlap between training and evaluation, and fine-grained distributional balancing (e.g., long-tailed planning difficulty distributions) (Li et al., 27 May 2025, Lee et al., 3 Apr 2026).

7. Implications, Best Practices, and Future Directions

ViewBench benchmarks have catalyzed several recommendations for both evaluation and model design:

• Prioritize Backward Token Warping with Dense Meshes: Efficient, semantically robust for view-invariant MLLMs; avoids pixel-space artifacts and is amenable to standard ViT representations (Lee et al., 3 Apr 2026).
• Report Robustness Under Controlled Overlap and Input Perturbations: Ensures that method gains are not confined to narrow operating points (Xie et al., 11 Jun 2025).
• Incorporate Allocentric Training Objectives: Perspective-aware fine-tuning is necessary for embodied AI applications involving spatial negotiation, object navigation, or multi-agent understanding (Li et al., 27 May 2025).
• Unified GT-Free 3D Self-Consistency Metrics: MVG method comparison should not depend on precise ground-truth 3D models, enabling fair cross-dataset and synthetic-to-real evaluation (Xie et al., 11 Jun 2025).
• Loop-Closure Diagnostics for World Models: Persistent scene structure is essential for interactive AI; benchmarks and metrics must go beyond short-term frame fidelity (Xiang et al., 8 Feb 2026).

A plausible implication is that future spatial AI systems will be benchmarked with ViewBench-style protocols, with strong geometric and mental-imagery grounding and resilience to unconstrained viewpoint changes constituting the new standard for multimodal, generative, and world-model architectures.