---
title: 'SpaceVista-1M: All-Scale Spatial Reasoning'
url: https://www.emergentmind.com/topics/spacevista-1m
type: topic
---

# SpaceVista-1M: All-Scale Spatial Reasoning

Searching arXiv for the primary paper and closely related spatial reasoning datasets mentioned in the source material.
SpaceVista-1M is a large-scale, video-based spatial reasoning dataset introduced in the SpaceVista framework to endow multimodal LLMs with “all-scale” visual spatial reasoning, from millimeter-level tabletop manipulation to drone-view and city-scale reasoning [2510.09606]. It comprises roughly **1 million QA pairs** over about **38,000 video scenes**, spans **19 task types**, and was created to address two limitations of earlier spatial reasoning resources: heavy reliance on **indoor 3D scans** and **labor-intensive manual annotations**, and the absence of effective **all-scale scene modeling**, which tends to produce overfitting to individual scenes. The dataset is designed primarily for **training** and **reinforcement learning** of spatial MLLMs rather than for evaluation; evaluation is delegated to a separate manually grounded benchmark, **SpaceVista-Bench** [2510.09606].

## 1. Scope, motivation, and design objective

SpaceVista-1M was created in response to persistent scale limitation, indoor bias, annotation cost, and narrow task coverage in prior spatial reasoning datasets. Earlier resources were mostly built from **indoor 3D scan data** such as ScanNet or ScanNet++ and concentrated on room-scale scenes, roughly **1–30 m**, with weak coverage of **tiny / manipulation-scale scenes**, **large outdoor / street / city / drone scales**, and **drone or aerial viewpoints**. They also relied on **labor-intensive 3D annotations** and were often restricted to a small number of task families such as counting, simple distance estimation, and basic relative-position judgments [2510.09606].

The stated goal of SpaceVista-1M is therefore not merely dataset scaling in the conventional sense, but the construction of a **holistic, all-scale, spatial reasoning resource**. In the reported formulation, the dataset covers real-world videos from **tiny object scale (mm)** up to **drone-view and urban scale (≈km)**, provides **structured 2D and 3D spatial supervision** such as depth, camera poses, and object masks or boxes, includes **19 spatial tasks**, and is built through a **task-specific, specialist-driven automated pipeline**. A plausible implication is that the dataset is intended as a substrate for learning spatial abstractions that are stable across radically different metric regimes rather than only across semantic categories.

## 2. Scale composition and scene domains

The dataset contains about **38K** scenes from diverse sources and a reported total of approximately **1,014K QA pairs**. The paper frames the corpus as spanning **5 spatial scales**, while the detailed scale statistics report QA counts for **Tiny Tabletop**, **Tabletop**, **Indoor**, **Wild Indoor**, **Outdoor**, and **Drone-View**. The corresponding physical ranges extend from **\(2 \text{ mm} \sim 5\text{ cm}\)** to **\(10\text{ m} \sim 0.7\text{ km}\)**, effectively covering **six orders of magnitude** [2510.09606].

| Scale | QA pairs | Physical range |
|---|---:|---|
| Tiny Tabletop | 79K | \(2 \text{ mm} \sim 5\text{ cm}\) |
| Tabletop | 242K | \(5\text{ cm} \sim 2\text{ m}\) |
| Indoor | 162.5K | \(0.5\text{ m} \sim 20\text{ m}\) |
| Wild Indoor | 213.3K | \(0.3\text{ m} \sim 50\text{ m}\) |
| Outdoor | 284.3K | \(0.5\text{ m} \sim 500\text{ m}\) |
| Drone-View | 33.1K | \(10\text{ m} \sim 0.7\text{ km}\) |

These scales are associated with distinct scene types and application domains. **Tiny tabletop** scenes, sourced from **uCO3D** and self-collected videos, include screws, tiny mechanical parts, and other very small objects, often in **360° object-centric videos**, with applications in **precision manufacturing**, fine manipulation, and potentially surgery or medical scenarios in the future. **Tabletop** scenes, from **WildRGB-D**, **SMOT**, and self-collected videos, depict kitchens, desks, beds, or floors with cluttered everyday objects and are directly linked to **robotic manipulation**, grasp planning, and rearrangement. **Indoor** scenes are structured and scan-based, using sources such as **ScanNet**, **ScanNet++**, **SpaceR**, and **SPAR**, and are tied to **indoor navigation** and embodied agents. **Wild Indoor** scenes derive from the indoor subset of **DL3DV** and emphasize more complex geometry, reflective or translucent surfaces, and public indoor environments. **Outdoor** scenes cover streets, parks, landmarks, plazas, and courtyards and are motivated by **autonomous driving**, pedestrian navigation, and urban understanding. **Drone-View** scenes, drawn from the drone subset of **DL3DV**, provide aerial perspectives for **drone navigation**, aerial mapping, and remote-sensing-style tasks [2510.09606].

On the video side, raw source data covers over **100 million frames**, while rich annotations are computed on about **10M selected frames**. The reported video resolution ranges from **480p to 2.7K**, usually at **24–30 fps**. Training uses up to **32 frames per video** at **128×28×28 tokenized resolution**, while evaluation often uses higher resolution. These details matter because the dataset is explicitly video-centric: temporal continuity is part of the supervision regime rather than an auxiliary convenience.

## 3. Task taxonomy and supervision schema

SpaceVista-1M is organized around **19 spatial reasoning tasks**, grouped into cross-scale and scale-specific categories [2510.09606].

**General cross-scale tasks** include **Position Comparison**, **Size Comparison**, **Existence Estimation**, **Object Counting**, **Rotation Estimation**, **Relative Distance**, **Absolute Distance**, **Object Size**, **Route Plan**, **Appearance Order**, **Depth Estimation**, **View Change Inference (Camera Moving)**, **Object Matching**, and **Spatial Relation**. Collectively, these tasks probe symbolic spatial relations, metric estimation, temporal geometry, identity persistence, ego-motion reasoning, and 3D structural relations such as **support**, **stacking**, **hanging**, **adhesion**, **encircling**, and **plug-in**.

**Scale-specific tasks** add global or application-specific reasoning. **Room Size** is specific to indoor scenes and requires estimation of room **area or volume**. **Navigation** is specific to outdoor scenes and extends route planning across streets and parks under multiple viewpoints. **Area Estimation** is specific to drone-view scenes and targets world-area estimation of roofs or fields from aerial perspective. **Obstacles Location** and **Manipulation Planning** are tabletop-specific and are formulated around collision-aware motion planning; for the latter, planning is computed via **RRT (Rapidly-exploring Random Tree) in 3D** and then discretized into linguistic actions [2510.09606].

The annotation schema is correspondingly heterogeneous. Each QA entry includes a **video clip**, optional visual grounding in the form of **keyframe masks**, **bounding boxes**, or **points**, and a **question** that may be **template-generated** or **GPT-generated**. The paper reports about **3,000 templates** to diversify structured prompts and reduce language shortcuts. Answers are provided in three formats: **free-form natural language**, **multiple choice**, and **regression**. Free-form answers are used for **SFT with chain-of-thought rationales**; multiple-choice and regression targets are used for **RL** and accuracy evaluation. CoT rationales are generated with **Qwen2.5-VL-72B** and related LLMs and then filtered for consistency. For RL, the rationales are removed and replaced at training time by structured anchors such as **`<semantic>`**, **`<scale>`**, and **`<answer>`**. A notable property of the corpus is that each “semantic unit” is counted only once in the **1M QA** total, producing a **QA/Scene ratio ~25**; this suggests a design that emphasizes scene diversity rather than exhaustive question proliferation on a small set of reconstructed environments.

## 4. Specialist-driven automated construction pipeline

SpaceVista-1M is built with an **automated, task-specific pipeline** that integrates specialist models for depth estimation, geometry, detection, segmentation, tracking, and language generation [2510.09606]. Source selection prioritizes video datasets with **known camera intrinsics/extrinsics** and or precomputed **3D models**, including **uCO3D**, **WildRGB-D**, **SMOT**, **ScanNet**, **ScanNet++**, **DL3DV**, and self-collected videos. A key design choice is to avoid estimating camera poses from scratch where possible, thereby reducing geometric drift.

Metric and temporally consistent depth are central. The reported depth stack uses **Metric3Dv2**, **UniDepthV2**, and **Video-Depth-Anything**. Video-consistent depth is obtained by minimizing

$$
D^{*} = \arg\min_D \left\{ \| D - M \|_F^2 + \lambda \| \nabla_t(D) - \nabla_t(N) \|_F^2 \right\},
$$

where \(M\) is per-frame metric depth, \(N\) is Video-Depth-Anything depth, and \(\nabla_t\) denotes the temporal gradient. This objective enforces temporal smoothness while retaining metric grounding.

Semantic detection and tracking are handled by **DINO-X** and **Grounding DINO** for open-vocabulary detection, and **SAM 2** in combination with Grounding DINO through **Grounded-SAM2** for instance masks and tracking. The resulting per-frame annotations include object categories, bounding boxes, masks, temporal IDs, depth, and camera parameters. With depth maps and camera calibration, pixels are projected into a **canonical camera space**, defined as a 3D Cartesian system centered at the camera optical center. For each segmented instance, the pipeline computes a **3D point cloud**, applies **PCA** to estimate principal axes, size, and centroid, and derives distances between centroids; room and area estimates are obtained by analyzing planar structures. For distance and size tasks, only objects with masks of at least **20×20 pixels** are retained.

Representative task workflows are explicitly specified. In outdoor **Counting**, open-vocabulary detection is run with a threshold such as **conf ≥ 0.3**, detections are projected into 3D and tracked across frames, and instances are filtered by **box size ≥ 32 px**, object count such as **2–10**, and track consistency over **≥ 10 frames**. Tabletop counting uses Grounding DINO boxes **>0.4** and SAM2 propagation with **IoU** and **center-distance thresholds** of **0.4** and **32 px**. **Distance** and **size** tasks use back-projected depth inside object masks and PCA-derived object extent. **Planning**, **Navigation**, and **Manipulation** use 3D representations and bounding boxes as obstacles and compute collision-free paths through **RRT** in world coordinates before converting them to discrete language actions through an LLM. **Spatial Relation** is derived from candidate object pairs selected by proximity and vertical arrangement, followed by an LLM decision over predefined relation types using 3D features and semantics under few-shot CoT prompting. **Depth** targets are taken directly from metric depth in canonical space, and **camera motion** is computed from extrinsics as the translation direction between frames.

## 5. Reliability, noise, and benchmark separation

The dataset is explicitly not treated as a reliable evaluation source because its annotations are largely generated by specialist models and automated procedures [2510.09606]. Two issues are emphasized. The first is **specialist model noise**: metric depth, 3D reconstruction, detection, and tracking remain imperfect, especially at extreme scales such as tiny objects and drone view, and in scenes with reflective or transparent surfaces. The second is **knowledge conflict in all-scale training**: similar visual patterns can correspond to very different physical scales, such as a toy car and a real car, or a toy room and a full-sized room. In the paper’s description, naïvely mixing all scales leads to **scale-confused representations**, visible as a wider spread of the **answer/gt** ratio away from the ideal value of **1**.

Several mitigation strategies are reported. The pipeline uses **cross-checking** with multiple depth models and the video-consistent depth objective, **filtering** by bounding-box size, track length, and CoT consistency, and **human verification** for a subset of the training QA through **MTurk** workers. Per-task human validation accuracy against pipeline answers is reported as mostly high, around **80–95%**, while tasks such as **route plan** and **navigation** are harder, around **50–60%**, indicating ambiguity or disagreement even for human annotators. The intended use is therefore learning **human-like perception**, not high-precision metrology.

Evaluation is separated into **SpaceVista-Bench**, a distinct benchmark of about **3,000 QA pairs** across about **500 video scenes** with approximately **99% accuracy** of benchmark labels. It covers **tiny tabletop**, **tabletop**, **indoor**, and **outdoor** settings and is built through self-recorded videos of around **50 objects of precisely measured sizes**, retrieval of authoritative measurements such as **Wikipedia** entries and **architectural plans**, and manual annotation by human experts for non-metric tasks. The authors state that there is **no scene overlap** between training and benchmark data. A common misconception is therefore that SpaceVista-1M itself functions as the definitive test set for all-scale spatial reasoning; the paper explicitly rejects that usage.

## 6. Use in training SpaceVista-7B and empirical outcomes

SpaceVista-1M is the core training resource for **SpaceVista-7B**, a specialized spatial reasoning MLLM built on **Qwen2.5-VL-7B** [2510.09606]. The model combines the Qwen2.5-VL video encoder, which produces semantic tokens \(F_V\), with a **DINOv3** auxiliary encoder that provides dense, semantics-agnostic features \(F_D\) intended to capture depth, normals, and other geometric patterns. After alignment to the same spatial resolution and feature dimension, the features are fused by cross-attention:

$$
F'_V = \mathrm{CA}(F_V, F'_D, F'_D) + F_V.
$$

This setup is explicitly tied to SpaceVista-1M’s rich spatial supervision and is meant to enhance geometry-aware perception beyond semantics.

To mitigate cross-scale conflict, the model introduces **LoRA-like scale experts** on top of the LLM. Each projection layer contains a base weight \(W_0\) and a weighted sum of rank-\(r\) LoRA modules:

$$
h = W_0 x + \sum_{i=1}^M \alpha_i^* B_i A_i x, \quad \alpha_i^* = \alpha_i \cdot \lambda_i,
$$

where the routing weights \(\lambda_i\) are produced by a **scale router**, described as an **MLP + softmax** conditioned on the input and potentially on scale metadata. Experts are trained per scale, and only about **0.5% of total parameters per expert** are trainable. The dataset’s explicit scale labels are therefore not incidental metadata; they are used directly to maintain independence of scale-specific knowledge.

Training proceeds in stages: **SFT** with CoT on SpaceVista-1M, activation of the scale router for further expert specialization, and **RL training (GRPO-style)** on the multiple-choice and regression subset. Reward design uses three anchors—**`<semantics>`**, **`<scale>`**, and **`<answer>`**—with semantic reward from cosine similarity, scale reward from log-scale discrepancy, and task-specific answer reward. The updated correctness reward is defined as

$$
\bar{R}_{\text{answer}} = \sum_{k=1}^3 \prod_{n=1}^k R_{j_n}, \quad (j_1, j_2, j_3) = (\text{answer}, \text{scale}, \text{semantic}),
$$

so that later anchors contribute less when earlier anchors fail. SpaceVista-1M is essential here because it supplies the **per-sample scale label**, semantic metadata, and a large number of MC and regression QAs.

Reported evaluation covers **MMSI-Bench**, **SPAR-Bench**, **VSI-Bench**, **STI-Bench**, and **SpaceVista-Bench**. The paper gives the following scores: **Qwen2.5-VL-7B** achieves **31.7 / 33.1 / 32.7 / 32.1 / 28.9** on those five benchmarks; **Qwen2.5-VL-7B w/ SpaceVista-1M** achieves **27.3 / 36.9 / 42.0 / 35.0 / 29.5**; **SpaceVista-7B (SFT)** reaches **29.1 / 38.1 / 46.3 / 35.9 / 34.5**; and **SpaceVista-7B w/ RL** reaches **32.3 / 41.6 / 48.6 / 38.2 / 36.7**. The degradation on **MMSI** after naïve finetuning and the subsequent gains from scale-aware experts and RL are presented as evidence that raw all-scale data mixing is suboptimal, whereas scale-aware use of SpaceVista-1M improves generalization.

## 7. Position within the dataset landscape, release, and limitations

Relative to prior spatial QA datasets, SpaceVista-1M is distinguished by its combination of scale breadth, scene diversity, task diversity, and video-based grounding [2510.09606]. The comparison table in the paper reports **SpaceR** at **191K QA pairs / 1.2K video scenes / QA per scene 159**, **SPAR-7M** at **7M / 4.5K / 1,556**, **Spatial-MLLM** at **120K / 1.5K / 83**, **InternSpatial** at **2.5M / 5.5K / 455**, and **SpaceVista-1M** at **1M / 38K / 25**. The lower **QA/Scene ratio (25)** is interpreted in the source material as evidence of many distinct scenes with relatively few questions per scene, mitigating overfitting to a small number of scanned apartments or rooms.

The dataset, model, and benchmark are stated to be released through the project page **https://peiwensun2000.github.io/mm2km/**. Licensing is reported as **Apache License 2.0 and CC BY 4.0**, aligned with source datasets. The data format includes videos or paths to videos, camera intrinsics and extrinsics, depth and normals, object detections with boxes and categories, instance masks and IDs, and QA JSON containing question, answer, rationale, task type, and scale type. Intended usage includes training spatial MLLMs for **all-scale spatial reasoning**, **robotics**, **autonomous-driving-like perception**, **drone aerial understanding**, and **reinforcement learning** with structured rewards.

The stated limitations are substantial. Specialist-generated labels are **noisy** and may align better with **human perceptual correctness** than with precise physical measurement. Certain scales remain underrepresented, including true **micrometer-level scenes** and extremely large **km\(^2\)** satellite imagery. Some application domains, including **medical**, **industrial inspection**, and long-range remote sensing, are missing or only lightly represented. Depth and reconstruction remain imperfect under occlusion and on reflective surfaces, and some QA instances may encode these biases. The authors accordingly recommend using SpaceVista-1M **primarily for training and model development**, while reserving **SpaceVista-Bench** and other benchmarks for rigorous evaluation.

Source: https://www.emergentmind.com/topics/spacevista-1m