RetoVLA: Enhanced Spatial Reasoning in Robotics

• RetoVLA is a vision-language-action model that repurposes register tokens from vision transformers to capture compressed global spatial context.
• The architecture integrates dual streams—semantic patch tokens and spatial register tokens—with a gating mechanism to enhance robotic manipulation.
• Empirical studies show significant gains, including up to +36 percentage points in real-world tasks, confirming its efficiency in complex spatial reasoning.

RetoVLA is a Vision-Language-Action (VLA) model architecture that leverages historically discarded "Register Tokens" from Vision Transformers (ViTs) to enhance spatial reasoning in robotic manipulation, achieving high efficiency and improved task performance without significantly increasing parameter count or model latency. The design principle is to repurpose these tokens—originally intended to absorb global scene artifacts during ViT optimization—as an explicit, compressed spatial context injected into the action policy module, thereby augmenting the VLA's ability to generalize across complex real-world robotic tasks while maintaining a lightweight computational profile (Koo et al., 25 Sep 2025).

1. Background: Register Tokens in Vision Transformers

Within modern ViTs, Register Tokens were introduced to mitigate the adverse effects of certain patch tokens developing large self-attention norms—an effect linked to visually uninformative regions (e.g., blank backgrounds), which the ViT exploited to store global scene statistics. Darcet et al. (2023) proposed introducing $K$ learnable Register Tokens ($$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$), absorbing this global context and "cleaning" the patch tokens for dense prediction tasks. Traditionally, these tokens were discarded post-training, under the assumption they were merely optimization artifacts.

2. Architectural Innovation and Hypothesis

RetoVLA posits that these Register Tokens encapsulate a compressed, scene-wide spatial summary (3D layout, object relations, workspace structure). Unlike standard pipelines, RetoVLA reuses these Register Tokens by injecting them directly into the Action Expert module (policy head) as an auxiliary context stream, adapting their dimensionality as needed via learned linear projections. The architecture comprises two principal information streams:

• Semantic stream: Patch tokens processed through a partial VLM backbone (first $L/2$ layers) serve as the primary semantic context.
• Spatial stream: Register Tokens, extracted from the same encoder output, are aggregated and projected as task-level spatial context.

Visualization of data flow:

• Image $\rightarrow$ Patch Embedding $\rightarrow$ ViT (partial) $\rightarrow$ $$\{\text{Patch tokens} \rightarrow \textrm{Action Expert};\ \text{Register Tokens} \rightarrow \textrm{Aggregator} \rightarrow \textrm{Projection} \rightarrow \textrm{Action Expert}\}$$.
• Textual/language tokens proceed along a canonical path into the Action Expert's query inputs.

3. Token Aggregation and Injection Mechanism

Let $P \in \mathbb{R}^{B \times N \times D_\text{vlm}}$ denote a batch of patch embeddings, and $$R_\text{init} \in \mathbb{R}^{K \times D_\text{vlm}}$$ the Register Tokens. A standard multi-head attention block aggregates spatial context: $$R_\text{scene} = \mathrm{Attention}(Q=R_\text{init}, K=P, V=P)$$ The scene-aware Register Token embedding ($$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$0) is then projected into the Action Expert space via learned weights $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$1: $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$2 A learnable scalar gate $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$3 dynamically controls the degree of register token influence, with final cross-attention keys/values: $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$4

$$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$5

This gating allows the downstream Action Expert to modulate reliance on global spatial context per task instance.

4. Training Objective and Implementation

Action policy learning uses conditional flow matching objectives in denoising-diffusion coordinates. Let $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$6 denote the ground-truth action, $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$7, and $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$8. The vector field $$\mathcal{R}_\text{init} \in \mathbb{R}^{K \times D}$$9 corresponds to the denoising direction. The Action Expert, conditioned on semantic and spatial features as well as language and proprioception, is optimized to minimize: $L/2$0 where $L/2$1 is the concatenated context of visual, spatial (register token), textual, and proprioceptive features.

5. Empirical Performance and Ablation Results

Simulation Benchmarks

LIBERO Benchmark Success Rates

Category	SmolVLA SR	RetoVLA SR	Δ
Spatial	75.8%	76.2%	+0.4
Object	70.8%	71.8%	+1.0
Goal	80.4%	80.4%	0.0
Long-horizon	50.4%	50.4%	0.0

RetoVLA provides the strongest gains on tasks requiring long-term working memory (+11.5 pp) and global 3D spatial reasoning (+9.0 pp) over baselines lacking register token pathways. There is a slight performance trade-off on tasks demanding precise local control.

Real-World and Sim-to-Real Robotic Manipulation

Custom Simulation Tasks

Task	SmolVLA SR	RetoVLA SR	Δ
Pick and Place	88%	96%	+6.0
Stack by Size	86%	88%	+2.0
Pull and Place	66%	82%	+16.0
Build Domino Line	28%	52%	+24.0
Clean Marker Mirror	46%	56%	+10.0
Mean SR	62.8%±11.6	74.8%±8.8	+12.0

Real-Robot Task Success

Task	SmolVLA SR	RetoVLA SR	Δ
Pick and Place	86%	92%	+6.0
Stack by Size	80%	76%	-4.0
Pull and Place	60%	78%	+18.0
Build Domino Line	12%	40%	+28.0
Clean Marker Mirror	38%	52%	+14.0
Close Drawer	60%	96%	+36.0
Move Bowl	16%	38%	+14.0
Mean SR	50.3%±11.1	67.4%±9.1	+17.1

Ablations indicate 2 register tokens is optimal; using more introduces redundancy or impairs the spatial summarization.

6. Analysis of Limitations and Future Directions

In some scenarios demanding highly localized dexterous behavior, additional global context may distract the policy, supported only partially by scalar gating. Scaling the approach to larger VLM backbones (e.g., OpenVLA-scale) or more dynamic tasks (e.g., mobile navigation, multitask 3D assembly) is an open challenge. The hardware and code releases, once available, are expected to facilitate further benchmarking across broader domains and architectures.

This suggests register tokens are a previously untapped resource that can drive substantial gains in spatially-complex tasks, contradicting the prior belief that their utility ceases after artifact removal in ViT pipelines.

7. Significance and Implications

RetoVLA provides a lightweight yet substantially more spatially aware VLA alternative, achieving up to +36 pp absolute gain in real-robot success on geometry-centric tasks without increasing the model size or inference cost measurably. Its method—injecting scene-level register token context via gated cross-attention into action generation—proves that information historically regarded as optimization "waste" can be productively leveraged for enhanced robotics intelligence (Koo et al., 25 Sep 2025).