SpatialScore Architecture Overview

• SpatialScore is a dual architecture that combines a reward model for optimizing text-to-image generation with a comprehensive benchmark for spatial reasoning in multimodal systems.
• It employs a joint vision-language backbone with LoRA adapters and reinforcement learning to achieve high spatial accuracy and robust performance in complex spatial tasks.
• SpatialScore sets new standards by rigorously evaluating spatial fidelity and multi-agent reasoning across diverse challenges, driving advances in AI’s spatial understanding.

SpatialScore refers to two pivotal, independently developed architectures addressing distinct but complementary aspects of spatial understanding in artificial intelligence: (1) a reward model architecture for evaluating and optimizing spatial faithfulness in text-to-image generation (Tang et al., 27 Feb 2026), and (2) a unified multimodal spatial understanding benchmark coupled with a modular multi-agent system for the evaluation and advancement of spatial reasoning in multimodal LLMs (MLLMs) (Wu et al., 22 May 2025). Both approaches have set new standards for measuring and enhancing AI’s capabilities in representing, perceiving, and reasoning about complex spatial relationships.

1. SpatialScore as a Reward Model for Text-to-Image Generation

SpatialScore, introduced by (Tang et al., 27 Feb 2026), is a reward model designed to quantitatively assess how well a generated image conforms to the spatial relationships described in a text prompt. It functions as an intermediate module within reinforcement learning (RL) pipelines targeting the improvement of spatial faithfulness in text-to-image (T2I) models, especially in cases involving multiple objects and intricate spatial relations.

Architecture and Training

SpatialScore consists of two main modules:

• Hφ (Joint Vision-Language Backbone): Based on Qwen2.5-VL-7B (a Transformer-based visual LLM), it integrates a text encoder (token embeddings, positional embeddings, self-attention), a visual encoder (patch embeddings, 2D positional embeddings, self-attention), and cross-modal layers facilitating bidirectional attention between image and text tokens. LoRA adapters (rank 8) are injected at every Transformer weight matrix for parameter-efficient fine-tuning.
• Rφ (Reward Head): During inference, a special <|Reward|> token appends to the text input; after cross-modal processing, the token's final hidden state $h \in \mathbb{R}^d$ (with $d \approx 4096$) encapsulates the joint image-text context. Rφ is a two-layer MLP: $$h \to [\text{Linear}(d \to d), \mathrm{GELU}] \to d\text{-dim} \to [\text{Linear}(d \to 2)] \to (\mu, \log \sigma)$$, interpreted as mean $\mu$ and log-std $\sigma$ of a Gaussian reward distribution. During training, the model draws $s \sim \mathcal{N}(\mu, \sigma^2)$ and averages over 1000 samples for stability.
• Objective: Pairwise preference scoring and a Bradley–Terry probabilistic formulation are used:

$$s_\theta(c, y) = R_\phi(H_\phi(c, y)), \quad P(y_w \succ y_l | c) = \sigma(s_\theta(c, y_w) - s_\theta(c, y_l))$$

with reward-model loss defined as:

$$\mathcal{L}_{\rm reward}(\theta) = \mathbb{E}_{(c, y_w, y_l) \sim \mathcal{D}}\left[ -\log\,P(y_w \succ y_l \mid c)\right].$$

Role in Reinforcement Learning and Policy Optimization

SpatialScore sits between the T2I generator (policy) and an RL optimizer (e.g., GRPO/PPO). For each batch of generated images, it produces scalar rewards that are converted to normalized advantages:

$$A^i = \frac{R^i - \text{mean}(R^j)}{\text{std}(R^j)}$$

Advantage calculation can utilize top-$k$ filtering (selecting the highest and lowest $k$ samples for robustness). The GRPO (Generalized Reward-Policy Optimization) objective for policy tuning is:

$$\mathcal{L}_{\rm GRPO}(\theta) = \frac{1}{|S|}\sum_{i\in S} \frac{1}{T} \sum_{t=0}^{T-1} \min\left(r^i_t(\theta)A^i_t,\, \text{clip}(r^i_t(\theta), 1-\epsilon, 1+\epsilon)A^i_t\right) + \lambda_{\rm KL} D_{\rm KL}(\pi_\theta \| \pi_{\rm ref})$$

with $$r^i_t(\theta) = \frac{p_\theta(x^i_{t-1} \mid x^i_t, c)}{p_{\theta_{\rm old}}(x^i_{t-1} \mid x^i_t, c)}$$ and $\lambda_{\rm KL} = 0.01$.

Dataset Construction

The SpatialReward-Dataset underpins the learning of the reward function:

• Size: $80\,000$ adversarially curated preference pairs.
• Pipeline: Prompts with complex spatial compositions ($\geq 3$ objects) are generated and perturbed by GPT-5; images are synthesized via state-of-the-art T2I models (Qwen-Image, HunyuanImage-2.1, Seedream-4.0). Human experts verify instance-level correctness, ensuring only quality-controlled examples are included.

Empirical Performance

On a held-out spatial benchmark:

• SpatialScore (7B): $95.8\%$ accuracy (surpassing Gemini-2.5 Pro's $95.1\%$)
• Open-source VLMs: $76.4\%$
• Proprietary GPT-5: $89.0\%$

SpatialScore-RL fine-tuning confers substantial improvements in spatial faithfulness, e.g., on DPG-Bench spatial subtask ($+6.1\%$ absolute), TIIF-Bench long prompts (Relation) ($+8.7\%$), and UniGenBench++ layout-3D ($+10.6\%$) (Tang et al., 27 Feb 2026).

2. SpatialScore as a Unified Benchmark for Multimodal Spatial Understanding

SpatialScore also denotes a large-scale, unified evaluation suite tailored for the rigorous analysis of 3D spatial reasoning abilities in MLLMs (Wu et al., 22 May 2025). It integrates twelve prior spatial QA datasets and introduces VGBench as its centerpiece.

Components and Task Taxonomy

• Datasets: $28\,000$ QA pairs across modalities (single-image, multi-image, video) and response formats (judgment, multiple choice, open-ended).
• Eight spatial task categories:

1. Counting
2. Object Localization (2D/3D boxes)
3. 3D Positional Relations
4. Depth/Distance Estimation
5. Object Properties (size, orientation)
6. Camera/Image Transformation
7. Point/Object Tracking
8. Scene Layout and Order

• SpatialScore-Hard: A curated subset ($1,400$ samples) selected for high difficulty, where representative MLLMs perform below $20–30\%$ but tool-augmented systems (SpatialAgent) score $35–45\%$.

Data Pipeline

VGBench is synthesized from approximately $300$ reconstructed scenes (from ScanNet, ScanNet++, WildRGB-D, CA-1M), providing depth maps, bounding boxes, and full camera parameters. Items are generated through templated prompts and LLM rewriting for diversification, with distractors engineered both randomly and via plausible perturbation for multiple-choice construction.

All spatial quantities—3D coordinates, depths, camera intrinsics, and relative poses—adhere to metric conventions and rigorous geometric standards. Evaluation tolerances and metrics are explicitly specified, e.g., depth/distance within $[0.5, 2]\times$ ground truth.

3. SpatialAgent: Multi-Agent Architecture for Spatial Reasoning

SpatialAgent is an algorithmic system devised to tackle the unified SpatialScore benchmark (Wu et al., 22 May 2025). It orchestrates nine specialized, open-source vision-geometry tools across four functional families:

• 2D Perception: object detection and segmentation (RAM++, Owlv2, SAM2)
• Motion & Transformation: optical flow, keypoint matching, homography estimation (RAFT, OpenCV SIFT, SIFT+RANSAC)
• Camera Geometry & Depth: depth estimation, camera pose, object orientation (SAM2, DepthAnythingV2, VGGT, OrientAnything)
• Auxiliary Utilities: arithmetic, cropping, self-reflective reasoning

SpatialAgent implements two reasoning paradigms:

• Plan-Execute (PE): hierarchically plans a sequence of tool invocations, executes, and summarizes.
• ReAct: alternates planning and execution with memory updates, allowing adaptive reasoning.

These modular strategies decompose complex spatial queries into executable sequences, each step invoking the most relevant tool and integrating outputs for precise, metric-level spatial understanding.

4. Evaluation Protocols and Performance Metrics

For both the reward-model and benchmark use cases, evaluation is strictly quantitative:

• Reward Model Setting: Accuracy in judging spatial relationships is calculated on reference preference pairs.
• Benchmark Setting: Strict accuracy is used for all formats; numeric responses are correct if within a predefined tolerance. Additional metrics (e.g., Mean Relative Accuracy on VSI-Bench) are adopted as necessitated by subtask conventions.

SpatialScore’s benchmarks highlight pronounced limitations in contemporary MLLMs on 3D spatial perception and reasoning, especially for tasks involving transformation, depth, and layout, where tool-augmented architectures like SpatialAgent provide clear improvements.

5. Significance and Architectural Impact

SpatialScore encompasses two complementary advances:

• As a Reward Model: It enables direct optimization of spatial faithfulness in T2I generation, facilitating consistent gains in rendering multi-object spatial scenes and surpassing prior open-source and proprietary systems (Tang et al., 27 Feb 2026).
• As a Benchmark/Framework: It provides the first large-scale, cross-modal, multi-format yardstick for 3D spatial reasoning, exposing both strengths and critical gaps in modern MLLMs, and justifying the value of modular, tool-based architectures for spatial intelligence (Wu et al., 22 May 2025).

A plausible implication is that a combination of fine-grained reward modeling for generative tasks and modular multi-agent tool invocation for understanding tasks delineates a pathway toward human-level spatial competence in AI systems. This systematic focus on spatial grounding and explicit metric reasoning sets a precedent for future research at the intersection of vision-language modeling, RL, and embodied reasoning.