---
title: 'Spatial Region 3D: Benchmarks & Models'
url: https://www.emergentmind.com/topics/spatial-region-3d-sr-3d
type: topic
---

# Spatial Region 3D: Benchmarks & Models

Searching arXiv for the supplied SR-3D-related papers and closely related benchmark work to ground the article.
arXiv search: "Spatial Region 3D SR-3D 3D spatial reasoning benchmark vision language model MM-Spatial"
Spatial Region 3D (SR-3D) denotes a cluster of closely related ideas in recent multimodal and 3D scene-understanding research rather than a single universally fixed construct. In benchmark work, SR-3D refers to 3D spatial reasoning: inferring positions, orientations, and inter-object relations in world space from visual input and a question. In MM-Spatial, SR-3D is the fundamental volumetric element for 3D perception tasks, instantiated as a yaw-oriented, gravity-aligned 3D bounding box in camera coordinates. In the vision-language modeling line, SR-3D is also the name of a 3D-aware region-prompted model that connects single-view 2D images and multi-view 3D data through a shared visual token space [2412.07825][2503.13111][2509.13317].

## 1. Conceptual scope and nomenclature

The most formal benchmark definition appears in 3DSRBench, which defines 3D spatial reasoning as the capacity to infer the true positions, orientations, and inter-object relations of entities in world space from a single image and an associated question. Under that definition, a model must calibrate the camera’s extrinsic parameters, infer each object’s 3D location \(X_w \in \mathbb{R}^3\), estimate an orientation vector \(\mathbf v\), and combine these ingredients to answer queries about height, depth, orientation, or multi-object layout [2412.07825].

MM-Spatial uses the same label differently. There, Spatial Region 3D is introduced as the fundamental volumetric element underlying 3D perception tasks such as distance-and-size estimation, 3D grounding, and spatial relationship prediction. The region is realized concretely as a 7-DOF oriented bounding box defined in the camera coordinate frame \(\{X_{\text{cam}}, Y_{\text{cam}} \text{ (up)}, Z_{\text{cam}} \text{ (forward)}\}\), parameterized by center, dimensions, and yaw angle [2503.13111].

The paper titled "3D Aware Region Prompted Vision Language Model" uses SR-3D as the name of a model family. That SR-3D augments a frozen 2D vision-language model with 3D awareness and flexible region prompting, and is intended to unify single-view 2D and multi-view 3D representation space for scene understanding [2509.13317].

A separate acronym collision exists in the literature: "Real-Time 3D Object Detection with Inference-Aligned Learning" names its detector SR3D, short for Spatial-prioritized and Rank-aware 3D object detection, for indoor point clouds. That usage concerns inference-aligned 3D detection rather than Spatial Region 3D as a region representation or 3D spatial reasoning benchmark [2511.16140].

## 2. Geometric formalizations

In 3DSRBench, SR-3D reasoning is grounded in explicit geometry. The world-to-camera transformation is written as
$$
X_c = R\,(X_w - t),
\quad R \in SO(3),\; t \in \mathbb{R}^3,
$$
with \(R\) parameterized by Euler angles roll \(\phi\), pitch \(\theta\), and yaw \(\psi\). Height above ground is defined either as the world \(z\)-coordinate,
$$
h = X_{w,z},
$$
or, for a general plane with normal \(\mathbf n\) and offset \(d_0\),
$$
h = \frac{\mathbf n^\top X_w + d_0}{\|\mathbf n\|}.
$$
Depth relative to the camera is the projection onto the camera-frame forward axis, and angular relations are computed through the standard vector-angle formula. Left/right or front/back judgments are reduced to the sign of a cross product in the horizontal plane [2412.07825].

MM-Spatial supplies a more object-centric volumetric definition. Each SR-3D region is a 7-DOF OBB with center \(C = (x,y,z)^\top\), dimensions \((l,w,h)\), and yaw \(\theta\) about the vertical \(Y\)-axis. The region is
$$
R = \{ C + R_Y(\theta)\cdot \Delta : \Delta \in [-l/2,+l/2]\times[-w/2,+w/2]\times[-h/2,+h/2] \},
$$
where
$$
R_Y(\theta) =
\begin{bmatrix}
\cos\theta & 0 & \sin\theta\\
0 & 1 & 0\\
-\sin\theta & 0 & \cos\theta
\end{bmatrix}.
$$
Volume is \(V(R)=l\cdot w \cdot h\), and 3D IoU is defined as \(\text{Volume}(R_1 \cap R_2) / \text{Volume}(R_1 \cup R_2)\) [2503.13111].

The SR-3D model uses 3D geometry as a positional-enrichment mechanism. A depth estimator produces a relative depth map \(D \in \mathbb{R}^{H\times W}\); each pixel is back-projected to camera coordinates,
$$
P_{\text{cam}}(u,v) = D(u,v)\cdot K^{-1}[u,v,1]^\top,
$$
then canonicalized to world coordinates via
$$
P_{\text{world}}(u,v) = R\cdot P_{\text{cam}}(u,v) + t.
$$
A sinusoidal encoding followed by a point-wise MLP produces a learned 3D positional embedding that is added to the frozen visual token \(f(u,v)\), yielding the enriched token \(f'(u,v)=f(u,v)+p(u,v)\) [2509.13317].

## 3. Benchmark structure and task taxonomy

3DSRBench is described as the first comprehensive 3D spatial reasoning benchmark, with 2,772 manually annotated visual question-answer pairs across 12 question types. It is balanced in data distribution, uses a novel FlipEval strategy, and includes paired-image subsets with common and uncommon viewpoints in order to probe robustness to camera extrinsics [2412.07825].

The benchmark’s 12 question types are grouped into four major categories:

| Category | Subtypes | Representative question forms |
|---|---:|---|
| Height | 1 | Which of two objects stands higher above the world ground plane? |
| Location | 3 | Near/far, closer to the camera, directly above/underneath |
| Orientation | 3 | Which side faces the camera, in front of/behind, left/right from mutual perspective |
| Multi-Object Reasoning | 5 | Which side of \(A\) faces \(B\), which faces toward \(C\), nearest neighbors, front-to-back ordering |

FlipEval is a paired evaluation protocol. For each sample \((I,Q,A)\), the image is horizontally flipped. If the question involves “left” or “right,” the answer is swapped accordingly; otherwise it is kept unchanged. A prediction is counted as correct only if the model answers both the original and flipped instances correctly. In practice, 3DSRBench combines FlipEval with CircularEval, which shuffles answer-choice order [2412.07825].

Viewpoint robustness is studied through synthetic common and uncommon camera splits. Camera extrinsics are parameterized via \(E=[R|t]\) with \(R = R_z(\psi)R_x(\phi)R_y(\theta)\) and \(t=(x,y,z)\). “Common” viewpoints use roll \(\phi \approx 0\), small pitch \(\theta\) within \(\pm 10^\circ\), and camera height \(z \approx 1.6\,\text{m}\). “Uncommon” viewpoints use large roll within \(\pm 30^\circ\), large pitch within \(\pm 60^\circ\), or floor-level and overhead camera heights [2412.07825].

MM-Spatial contributes a separate data and evaluation substrate centered on SR-3D regions. CA-VQA extends CA-1M by sampling RGB frames at 1 FPS for training and 0.1 FPS for evaluation, transforming all 7-DOF boxes into camera space, retaining full invisible-plus-visible boxes without clipping to the image plane, rendering a high-density point cloud by back-projecting a ground-truth FARO depth map, and generating QA pairs for counting, binary spatial relations, metric regression, and 2D/3D grounding. Its “blind filtering” stage removes any evaluation sample that at least 3 out of 7 high-capability vision-and-language models can answer correctly using text alone [2503.13111].

## 4. Model architectures built around SR-3D

MM-Spatial is built on MM1.5. Its vision encoder is DFN-CLIP, which processes an input image of \(672\times 672\) and up to four support views in multi-view mode; C-Abstractor fuses visual tokens into the LLM token space; and a 3B-parameter causal LLM produces answer text, tool calls, or chain-of-thought. The paper studies two mechanisms for metric depth incorporation. In Full Encoded Depth, a normalized depth map is replicated as three-channel pseudo-RGB and fed through a separate depth-connector, with the stated limitation that only relative normalized depth is learned. In Tool-Use / CoT mode, the text stream includes intermediate steps such as box coordinates and median depth values, and at test time the model either emits a depth-query tool call or predicts the depth itself within the chain of thought. Training uses standard causal-LM cross-entropy, with an auxiliary cross-entropy for tool-use coordinates and function-call tokens [2503.13111].

The SR-3D model uses a different architectural strategy. It augments a frozen 2D VLM with a 3D positional embedding module and a dynamic tiling-based region extractor. The 2D visual encoder is a tiled transformer backbone such as NVILA-Lite processing \(448\times 448\) tiles; the backbone remains frozen. For each pixel, depth is back-projected to a 3D point in a canonical world coordinate system, sinusoidally encoded, mapped by a point-wise MLP, and added to the corresponding visual token. This establishes a shared visual token space for both single-view and multi-view inputs. A user-supplied bounding-box mask, segmentation mask, or projected 3D box is retiled at full resolution, encoded tile by tile, stitched back to full spatial size, and mask-pooled into a fixed-length region token. A language model such as Qwen-2-7B attends over global vision tokens and optional region tokens to generate free-form or multiple-choice answers [2509.13317].

Region prompting in SR-3D is explicitly flexible. The model supports 2D bounding boxes or segmentation masks on a single frame, sparse multi-frame masks, and direct 3D inputs through 3D axis-aligned boxes that are projected into each view. Training is instruction-finetuned end-to-end on mixed data; all vision encoder weights remain frozen, while the 3D positional embedding module, the region extractor heads, and the LLM are trained. The reported losses are task-driven cross-entropy terms only: answer generation, region classification or multiple choice, and dense captioning. No additional contrastive or reconstruction losses are used in the publicly reported experiments [2509.13317].

A plausible implication is that the two model families instantiate complementary views of SR-3D. MM-Spatial treats SR-3D primarily as an annotated volumetric primitive for supervision and reasoning, whereas the SR-3D model treats it as a tokenizable interface through which 2D and 3D evidence can be jointly queried.

## 5. Evaluation results and empirical behavior

3DSRBench reports human supervision at approximately \(95.7\%\) overall accuracy across all 12 subtypes. Among model categories on the real natural-image split, Location questions are the easiest at approximately \(60\%\)–\(63\%\), Height questions are around \(49\%\)–\(51\%\), Orientation questions are the most difficult at approximately \(23\%\)–\(37\%\), and Multi-Object Reasoning is approximately \(40\%\)–\(44\%\). The best open-source model, LLaVA-NeXT-8B, achieves \(49.6\%\) overall, while the best proprietary models are reported at approximately \(49.1\%\)–\(49.3\%\). On the common-to-uncommon synthetic viewpoint split, GPT-4o drops \(6.9\%\) from \(51.2\%\) to \(44.3\%\), Gemini-Pro drops \(10.4\%\) from \(59.9\%\) to \(49.5\%\), and LLaVA-NeXT-8B drops \(8.7\%\) from \(45.5\%\) to \(36.8\%\) [2412.07825].

MM-Spatial reports strong gains on CA-VQA. MM-Spatial-3B achieves Binary \(72\%\), Counting \(76\%\), 2D AP@50 \(53\%\), 3D AP@15 \(21\%\), Regression for ego-distance \(40\%\) at \(10\%\) relative error, and an overall CA-VQA average of \(47.0\%\), compared with \(23\%\) for GPT-4o and \(24\%\) for SpatialRGPT. Adding reasoning and auxiliary modalities raises performance further: \(+\,\)CoT reaches \(49.1\%\), \(+\,\)Tool(ground-truth depth) reaches \(48.5\%\), \(+\,\)Multi-View+CoT reaches \(49.7\%\), and \(+\,\)Multi-View+Tool(GT) reaches \(52.4\%\). On CV-Bench, MM-Spatial-3B reaches \(91.2\%\) overall and nearly solves indoor 3D tasks, with \(93.5\%\) depth-order and \(89\%\) distance; Multi-View+Depth pushes performance to \(96+\). The model’s chain-of-thought depth predictions are also reported as rivaling DepthPro, with \(\delta_1 = 84.6\%\) versus \(82.2\%\) and \(\text{AbsRel}=0.128\) versus \(0.134\) [2503.13111].

The SR-3D model reports state-of-the-art results across both 2D and 3D benchmarks. On COCO-2017 Region Classification with ground-truth boxes, SR-3D 8B reaches mAP \(78.0\) and accuracy \(88.6\%\), compared with SpatialRGPT 8B at mAP \(72.9\) and accuracy \(82.9\%\). On BLINK_Depth it reaches \(90.3\%\). On general VQA benchmarks, SR-3D 8B reports BLINK_S \(83.9\), SAT \(64.0\), EmbSpat \(72.5\), and RWQA \(68.1\). On 3D multi-view tasks, SR-3D 8B reaches Scan2Cap CIDEr \(97.9\), B-4 \(44.7\), Meteor \(31.5\), Rouge \(67.3\); ScanQA EM \(30.4\%\), Cider \(109.3\); and SQA3D EM \(62.2\%\). On region-level 3D spatial QA in SR-3D-Bench, it reports approximately \(79.5\%\) average qualitative accuracy and \(83.3\%\) on quantitative metrics for width, height, and distance. On VSI-Bench, it reports \(82.3\%\) on Relative Direction, \(75.5\%\) on Height MRA, and \(41.9\%\) on Distance MRA. When ground-truth point clouds are replaced by CUT3R reconstructions on ScanQA, Cider remains \(109.3\) and EM changes from \(30.4\) to \(30.2\) [2509.13317].

Taken together, these results indicate that SR-3D remains challenging at the level of orientation and multi-object relational inference, while region-aware and multi-view-aware architectures can substantially improve performance once explicit geometric structure or 3D supervision is introduced.

## 6. Limitations, misconceptions, and research directions

A recurring misconception is that strong 2D visual competence automatically implies strong SR-3D competence. The benchmark evidence argues against that view. 3DSRBench explicitly concludes that implicit 3D knowledge from large-scale training does not suffice, and that models need explicit 3D supervision or synthetic multi-view data. It also argues for architectural inductive biases for geometry, including depth priors, plane detectors, and object pose estimation modules; data augmentation with varied Euler angles and multi-view consistency objectives; 3D-aware vision encoders fused with semantic encoders via a spatial vision aggregator; and chain-of-thought prompts that decompose camera roll, object coordinates, and relational comparison [2412.07825].

A second misconception is that SR-3D denotes a single canonical representation. The literature instead uses the term at three levels: as a reasoning competency, as an oriented volumetric region, and as a 3D-aware region-prompted model. This suggests that SR-3D is best understood as a research program organized around geometry-aware region reasoning rather than as one standardized object.

Current limitations are also explicit. The SR-3D model reports weakness on orientation reasoning, is designed for static multi-view settings rather than articulated or rapidly moving objects, and incurs a slight drop on pure OCR tasks. Its proposed extensions include explicit orientation embeddings or angular bins, temporal positional embeddings on top of static 3D positional embeddings, and a single checkpoint jointly trained for both 2D single-view and multi-view 3D tasks [2509.13317].

MM-Spatial highlights a different set of methodological implications. It argues that high-quality oriented 3D boxes preserve true object dimensions better than axis-aligned approximations, that tool-use of depth maps outperforms naïve full encoding of relative depth, and that carefully curated supervised fine-tuning data alone can endow a general-purpose MLLM with metric depth capabilities comparable to dedicated monocular depth estimation models. Its stated future directions are extension to outdoor scenes with larger metric scales and richer volumetric representations beyond cuboids, such as convex polyhedra or learned occupancy fields [2503.13111].

Across these works, the broad direction is consistent: robust SR-3D requires explicit geometric parameterization, region-level supervision, and evaluation protocols that control for viewpoint bias and shortcut learning. The principal open problem is not merely adding depth channels or extra views, but aligning region representations, camera geometry, and reasoning objectives so that multimodal models can sustain accurate spatial inference under viewpoint shift, sparse prompting, and metric queries.

Source: https://www.emergentmind.com/topics/spatial-region-3d-sr-3d