VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling (2512.02902v1)

Abstract: Vision-language-action (VLA) models achieve strong in-distribution performance but degrade sharply under novel camera viewpoints and visual perturbations. We show that this brittleness primarily arises from misalignment in Spatial Modeling, rather than Physical Modeling. To address this, we propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates. Our first method, Feature Token Modulation (FTM), applies a global affine transformation to visual tokens and improves Libero viewpoint accuracy from 48.5% to 87.1% with only 4K parameters. Building on this, Feature Linear Adaptation (FLA) introduces low-rank updates to the ViT encoder, achieving 90.8% success with 4.7M parameters -- matching LoRA-scale finetuning at far lower cost. Together, these results reveal substantial untapped robustness in pretrained VLA models and demonstrate that targeted, minimal visual adaptation is sufficient to restore viewpoint generalization.

• The paper demonstrates that lightweight one-shot adaptations (FTM and FLA) significantly restore spatial alignment and boost performance from 48.5% to over 90% using minimal parameters.
• The paper dissects VLA models into spatial and physical components, identifying misaligned spatial representations as the primary driver of policy brittleness under viewpoint shifts.
• Empirical results on the LIBERO benchmark and real-world validations confirm that the adaptation techniques match or exceed full finetuning while avoiding global retraining.

Vision-Language-Action Models and Generalization: Revisiting Physical and Spatial Modeling

Introduction

Vision-Language-Action (VLA) models have become foundational in embodied AI, integrating visual perception, linguistic grounding, and policy learning to enable general-purpose robotic manipulation. Despite robust in-distribution performance, these models demonstrate pronounced brittleness under domain shifts, especially with out-of-distribution (OOD) camera viewpoints and ambient visual perturbations. This paper ("VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling" (2512.02902)) provides a rigorous analysis of this brittleness and presents parameter-efficient, one-shot adaptation strategies that exploit latent robustness in existing VLA models.

Problem Decomposition: Spatial vs. Physical Modeling

The paper dissects VLA models into two functional subsystems:

• Spatial Modeling (visual encoder): Responsible for extracting spatial relations from raw images, such as object positions and orientations.
• Physical Modeling (VLM + action expert): Integrates spatial representations, language instructions, and action history to synthesize executable manipulation trajectories.

Performance degradation under viewpoint changes is traced to misalignment in spatial representations—not deficiencies in the physical reasoning pipeline. The high-level policy remains functionally intact, but receives visually encoded embeddings that are spatially distorted, resulting in failures in grounded reasoning and manipulation.

Existing Approaches and Their Limitations

Two dominant robustness paradigms exist:

• Data-centric approaches: Domain randomization and multi-view datasets (e.g., Libero-Plus) increase visual diversity but scale poorly to real-world settings due to resource constraints.
• Representation-centric approaches: Techniques such as multi-view supervision and 3D-consistent architectures (e.g., GeoAware-VLA, Adapt3R) improve viewpoint invariance but are sensitive to non-geometric perturbations (e.g., background clutter, lighting).

Central to these approaches is a persistent assumption: robustness deficiencies necessitate additional data or model complexity. Few works empirically isolate whether failures originate from spatial representation misalignment.

One-Shot Robustness Adaptation: Methodology

The paper introduces a unified, lightweight, one-shot adaptation framework for pretrained VLA models, consisting of:

Feature Token Modulation (FTM)

A global affine transformation $(\gamma, \beta)$ is applied to visual token embeddings. This re-scales and re-centers feature distributions with only $2D$ parameters (4K in total), yielding significant gains in viewpoint robustness.

Figure 1: Feature Token Modulation applies a learned affine shift to all visual token features, enabling rapid correction of viewpoint-induced drift.

Feature Linear Adaptation (FLA)

Low-rank adaptation is performed on the ViT encoder via LoRA, introducing trainable linear shifts inside backbone layers. With only 4.7M parameters, FLA achieves success rates matching full LoRA finetuning ($\sim$ 467M parameters).

Figure 2: Comparison of adaptation strategies; FLA provides visual backbone updates using low-rank structure, avoiding global retraining.

Empirical Evaluation

Libero-V Benchmark: Design and Protocol

Libero-V extends the LIBERO benchmark to include systematically controlled visual perturbations: camera viewpoint, illumination, background texture, and image noise. These axes simulate realistic domain shifts and form a unified robustness evaluation protocol.

Figure 3: Success rates under Small, Medium, and Large camera viewpoint shifts, evidencing stable performance of the adapted policy.

Quantitative Results

• Viewpoint Adaptation: One-shot FTM increases LIBERO viewpoint accuracy from 48.5% to 87.1%. One-shot FLA further boosts performance to 90.8%, exceeding LoRA finetuning (90.3%) by using orders of magnitude fewer parameters.
• Cross-perturbation Generalization: FLA achieves 94.8% average success across camera, lighting, texture, and noise variants—consistently matching or exceeding the accuracy of parameter-intensive adaptation and outperforming baselines with geometry-aware vision backbones.

  Figure 4: Success rates before and after adaptation on LIBERO, illustrating the efficacy of FTM/FLA against standard baselines.

Real-World Validation

Robustness is validated on a Franka Emika Panda robot using the GELLO teleoperation framework. Even with a single demonstration from a novel viewpoint, the adapted policy reliably executes five diverse manipulation tasks under significant viewpoint and scene shift.

Figure 5: Real-world experimental setup comprising dual camera views and the novel viewpoint used for adaptation.

Figure 6: Post-adaptation manipulation rollouts demonstrating spatial realignment and successful execution in all evaluated tasks.

Embedding Alignment Analysis

t-SNE analyses reveal that without adaptation, embedding distributions for novel viewpoints are distant from the source manifold, resulting in policy failure. FLA projects target embeddings to tightly adjoin the source manifold, restoring feature space connectivity and enabling policy functionality.

Figure 7: t-SNE visualization showing domain gap in visual token embeddings before adaptation and structured alignment after FLA.

Theoretical Justification

The paper frames the adaptation process via locally affine or low-rank corrections that bridge the representation gap induced by visual perturbation. Three theorems formalize:

• Policy degradation is linearly upper-bounded by visual feature drift.
• Affine transformations (FTM) provably restore policy under mild domain shift conditions.
• Low-rank corrections (FLA) approximate the optimal embedding shift, with analytic residual error bounds.

These results justify why highly parameter-efficient adaptation modules—FTM and FLA—restore OOD robustness without global retraining.

Implications and Future Directions

Practical Impact

• Parameter efficiency: Achieving SOTA robustness with $<1\%$ of the parameters required for LoRA-style adaptation enables scalable deployment of spatially robust policies on resource-constrained hardware.
• Adaptation scalability: One-shot, demonstration-driven adaptation supports rapid field deployment in novel environments, with minimal operational overhead.
• Model modularity: Isolating adaptation to the vision module avoids catastrophic forgetting and instability associated with global finetuning.

Theoretical Significance

The results suggest that latent robustness is an inherent property of foundation VLA models. Efficient spatial adaptation unlocks this potential, obviating the need for data-centric or architecture-centric augmentation.

Prospective Research

Key directions include:

• Generalization to multi-modal and multi-agent settings: Extending lightweight spatial adaptation to handle complex multi-view, multi-agent, and semantic domain shifts.
• Algorithmic advances in adapter construction: Investigating alternative parameter-efficient adaptation mechanisms, including nonlinear adapters, conditional normalization, and domain-invariant meta-learning.
• Towards continual adaptation: Embedding on-the-fly spatial adaptation and alignment protocols for lifelong robot learning.

Conclusion

This paper demonstrates that performance brittleness of VLA models under visual perturbation arises primarily from misalignment in spatial modeling rather than limitations in visuomotor reasoning. Lightweight, one-shot adaptation—via global affine transformations (FTM) and low-rank linear corrections (FLA)—recalibrates visual representations, unlocking substantial latent robustness. Empirical and theoretical analyses indicate that scalable embodied generalization does not require larger models or more data but can be accomplished through targeted, efficient adaptation of the visual pathway. These insights materially advance the paradigm of robust, generalizable vision-language-action models and set the stage for practical deployment in dynamic real-world scenarios.