Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model (2510.12276v1)

Abstract: Vision-language-action (VLA) models have recently shown strong potential in enabling robots to follow language instructions and execute precise actions. However, most VLAs are built upon vision-LLMs pretrained solely on 2D data, which lack accurate spatial awareness and hinder their ability to operate in the 3D physical world. Existing solutions attempt to incorporate explicit 3D sensor inputs such as depth maps or point clouds, but these approaches face challenges due to sensor noise, hardware heterogeneity, and incomplete depth coverage in existing datasets. Alternative methods that estimate 3D cues from 2D images also suffer from the limited performance of depth estimators.We propose Spatial Forcing (SF), a simple yet effective alignment strategy that implicitly forces VLA models to develop spatial comprehension capabilities without relying on explicit 3D inputs or depth estimators. SF aligns intermediate visual embeddings of VLAs with geometric representations produced by pretrained 3D foundation models. By enforcing alignment at intermediate layers, SF guides VLAs to encode richer spatial representations that enhance action precision.Extensive experiments in simulation and real-world environments demonstrate that SF achieves state-of-the-art results, surpassing both 2D- and 3D-based VLAs. SF further accelerates training by up to 3.8x and improves data efficiency across diverse robotic tasks. Project page is at https://spatial-forcing.github.io/

• The paper introduces Spatial Forcing, an auxiliary alignment method that implicitly matches VLA visual tokens with 3D geometric features to improve spatial reasoning.
• It achieves state-of-the-art performance by substantially increasing success rates and data efficiency on both simulation (LIBERO, RoboTwin) and real-world benchmarks.
• The method accelerates training by up to 3.8× and outperforms explicit 3D input approaches without additional sensor overhead.

Spatial Forcing: Implicit Spatial Representation Alignment for Vision-Language-Action Models

Introduction

The paper introduces Spatial Forcing (SF), a representation alignment strategy for Vision-Language-Action (VLA) models that enables implicit acquisition of spatially-aware knowledge without explicit 3D sensor inputs or depth estimation. The motivation stems from the observation that most VLA models are built on vision-LLMs (VLMs) pretrained on 2D data, which lack the spatial awareness required for precise 3D robotic manipulation. Existing approaches that incorporate explicit 3D modalities (e.g., depth maps, point clouds) face challenges related to sensor noise, hardware heterogeneity, and limited dataset coverage. SF addresses these limitations by aligning intermediate visual embeddings of VLAs with geometric representations from pretrained 3D foundation models, specifically the Visual Geometry Grounded Transformer (VGGT), thereby guiding VLAs to encode richer spatial representations.

Figure 1: Overview of Spatial Forcing (SF): (a) SF aligns VLA visual embeddings with geometric representations from 3D foundation models; (b) SF improves training efficiency and test accuracy; (c) Depth probing demonstrates that SF injects spatial information into VLA representations.

Methodology

Problem Formulation and Motivation

VLA models process multi-view images, language instructions, and generate action tokens in an auto-regressive manner. The core challenge is that visual embeddings derived from 2D-pretrained VLMs lack 3D spatial structure, limiting the model's ability to reason about the physical world. The authors conduct a depth probing experiment, freezing a VLA model and training a DPT head to predict depth from visual embeddings. The results show that embeddings from 2D-only training do not encode meaningful spatial structure, confirming the spatial deficiency.

Figure 2: Depth probing reveals that 2D-trained embeddings lack spatial structure, while SF-aligned embeddings encode rich spatial information.

Spatial Forcing (SF) Alignment

SF introduces an auxiliary alignment loss that supervises intermediate visual embeddings of the VLA model to match spatial representations from a pretrained 3D foundation model (VGGT). The process is as follows:

1. Spatial Supervision Signal: Multi-view images are input to VGGT, which outputs per-pixel spatial representations (e.g., depth, 3D point tracks).
2. Token Alignment: VLA visual tokens are batch-normalized and projected via a two-layer MLP to match the dimension of VGGT features.
3. Cosine Similarity Loss: The alignment loss is computed as the negative cosine similarity between the processed VLA visual tokens and the corresponding VGGT spatial representations, optionally augmented with positional embeddings.
4. Layer Selection: Alignment is most effective when applied to relatively deep, but not the deepest, transformer layers—empirically, the 24th of 32 layers yields optimal results.
5. Training Objective: The final loss is a weighted sum of the standard action loss and the alignment loss:

$$\mathcal{L}_\mathrm{SF} = \mathcal{L}_\mathrm{action} + \alpha \mathcal{L}_\mathrm{align}$$

where $\alpha$ is a hyperparameter controlling the alignment strength.

During inference, the VLA model operates identically to a standard VLA, incurring no additional computational overhead.

Experimental Results

Simulation Benchmarks

Experiments are conducted on LIBERO and RoboTwin benchmarks, covering a wide range of spatial, object, goal, and long-horizon tasks. SF is evaluated against both 2D- and 3D-based VLA baselines.

Figure 3: Comparison of 3D VLA paradigms: explicit 3D input, 2D-to-3D estimation, and implicit alignment (SF).

Figure 4: SF achieves the highest success rates on the RoboTwin 2.0 benchmark, outperforming both 2D and explicit 3D VLA baselines.

Key findings:

• SF achieves an average success rate of 98.5% on LIBERO, surpassing all 2D and implicit 3D baselines, and matching or exceeding explicit 3D methods that require additional sensor inputs.
• On RoboTwin, SF yields substantial improvements over the base model ($\pi_0$), especially in hard, domain-randomized settings, indicating enhanced spatial awareness and robustness.

Training and Data Efficiency

SF significantly accelerates training convergence and improves data efficiency:

Figure 5: (a) Success rate vs. training iterations; (b) Success rate vs. training data; (c) t-SNE visualization shows aligned representations closely match the target spatial distribution.

• SF achieves target success rates up to 3.8× faster than the base model.
• With only 5% of the training data, SF attains 75.8% success rate, a 25.8% absolute improvement over the base model at the same data fraction.
• t-SNE analysis demonstrates that SF-aligned embeddings adopt the relational geometry of the 3D target representations without representational collapse.

Real-World Experiments

SF is validated on a bimanual AgileX platform across diverse single-arm and dual-arm tasks, with significant variations in lighting, object type, and spatial configuration.

Figure 6: (a) Single-arm tasks under varying conditions; (b) Dual-arm tasks for spatial balance; (c) Top-view robot setup.

• SF consistently outperforms the base model, with up to 47.5% higher success rates in challenging tasks (e.g., stacking transparent cups under variable lighting).
• The method demonstrates strong generalization to new objects, spatial layouts, and embodiment configurations, with high data efficiency (40 demonstrations for single-arm, 20 for dual-arm tasks).

Ablation and Analysis

• Target Representation: Using VGGT as the alignment target yields the highest gains, confirming the importance of 3D-aware supervision. Adding positional embeddings further improves long-horizon task performance.
• Layer Selection: Supervising deep, but not final, transformer layers is optimal; shallow supervision is less effective due to loss of spatial information in subsequent layers.
• Alignment Weight: The alignment loss weight $\alpha$ must be tuned; excessive values destabilize training, while moderate values (e.g., $\alpha=0.5$) yield the best results.
• Generalization: SF does not cause representational collapse; aligned VLA features preserve modality-specific information while adopting the spatial structure of the 3D target.

Implications and Future Directions

SF demonstrates that implicit spatial representation alignment is a scalable, sensor-agnostic approach to endowing VLA models with 3D spatial awareness. The method is compatible with any VLA architecture that exposes intermediate visual embeddings and can leverage advances in 3D foundation models for supervision. The approach is particularly valuable for real-world robotics, where 3D sensor data is often noisy, incomplete, or unavailable.

Potential future directions include:

• Extending SF to multi-modal or multi-task settings, leveraging richer 3D representations (e.g., scene graphs, affordances).
• Investigating alignment with other forms of geometric supervision, such as self-supervised 3D structure prediction.
• Exploring curriculum learning strategies for progressive spatial alignment.
• Integrating SF with reinforcement learning or closed-loop control for further robustness.

Conclusion

Spatial Forcing provides a simple, effective, and generalizable method for implicitly aligning VLA visual embeddings with 3D spatial representations, resulting in substantial improvements in spatial reasoning, training efficiency, and data efficiency. The approach achieves state-of-the-art results on both simulation and real-world robotic manipulation benchmarks, without requiring explicit 3D sensor inputs or depth estimation. SF represents a significant step toward scalable, spatially-aware VLA models suitable for deployment in diverse and unstructured environments.