PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models

Abstract: Large-scale pretraining has made Vision-Language-Action (VLA) models promising foundations for generalist robot manipulation, yet adapting them to downstream tasks remains necessary. However, the common practice of full fine-tuning treats pretraining as initialization and can shift broad priors toward narrow training-distribution patterns. We propose PriorVLA, a novel framework that preserves pretrained priors and learns to leverage them for effective adaptation. PriorVLA keeps a frozen Prior Expert as a read-only prior source and trains an Adaptation Expert for downstream specialization. Expert Queries capture scene priors from the pretrained VLM and motor priors from the Prior Expert, integrating both into the Adaptation Expert to guide adaptation. Together, PriorVLA updates only 25% of the parameters updated by full fine-tuning. Across RoboTwin 2.0, LIBERO, and real-world tasks, PriorVLA achieves stronger overall performance than full fine-tuning and state-of-the-art VLA baselines, with the largest gains under out-of-distribution (OOD) and few-shot settings. PriorVLA improves over pi0.5 by 11 points on RoboTwin 2.0-Hard and achieves 99.1% average success on LIBERO. Across eight real-world tasks and two embodiments, PriorVLA reaches 81% in-distribution (ID) and 57% OOD success with standard data. With only 10 demonstrations per task, PriorVLA reaches 48% ID and 32% OOD success, surpassing pi0.5 by 24 and 22 points, respectively.

• The paper proposes Dual Action Experts that isolate and leverage pretrained motor and scene priors during downstream adaptation.
• It integrates learnable Expert Queries with a masked attention mechanism to selectively extract and combine priors for action generation.
• Empirical results show significant improvements in out-of-distribution success and data efficiency, particularly in few-shot robotic manipulation tasks.

PriorVLA: Preserving Pretrained Priors for Vision-Language-Action Model Adaptation

Motivation and Problem Statement

Large-scale pretraining has established Vision-Language-Action (VLA) models as effective foundations for general-purpose robot manipulation, offering broad capabilities in grounding language instructions in visual observations and synthesizing robot actions across a spectrum of tasks and embodiments. However, pretraining alone is insufficient for high-fidelity execution on diverse downstream tasks, necessitating adaptation to new domains. The canonical solution—full fine-tuning—treats the pretrained model’s parameters as simple initialization, updating all weights on limited task-specific data. This approach risks catastrophic “prior forgetting,” wherein broad generalist capabilities erode as the model overspecializes to narrow downstream distributions, causing marked degradation in out-of-distribution (OOD) generalization and data efficiency.

The paper “PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models” (2605.10925) addresses this limitation by introducing a principled framework that not only preserves but also leverages pretrained scene and motor priors during downstream adaptation. The objective is to reframe adaptation: not as a brute-force fit to new data, but as a process that learns to exploit the structural knowledge and motor regularities acquired during large-scale pretraining.

Figure 1: Overview of PriorVLA. The framework leverages broad pretrained priors by architecturally isolating and explicitly utilizing them through Dual Action Experts and Expert Queries, preventing “prior forgetting,” particularly under OOD and few-shot regimes.

PriorVLA Architecture and Mechanisms

Dual Action Experts

Central to PriorVLA is the architectural separation of pretrained prior preservation from task-specific adaptation—termed Dual Action Experts. The pretrained Action Expert (AE) becomes a frozen Prior Expert, providing a stable, read-only source of motor priors. In parallel, an Adaptation Expert—initialized from the same pretrained weights and exclusively trainable—specializes the action policy to the downstream data. Only the Adaptation Expert’s denoising output is used for actual prediction; the Prior Expert’s output is not decoded, but its internal layerwise representations expose motor priors for use in the adaptation path.

Figure 2: PriorVLA builds on a pretrained VLA by introducing Dual Action Experts (frozen Prior Expert and trainable Adaptation Expert) and Expert Queries that harvest pretrained scene/motor priors.

Expert Queries and Structured Attention

While structural separation preserves the priors, PriorVLA critically incorporates Expert Queries as learnable token-based interfaces that enable direct, layerwise access to pretrained knowledge during each denoising step. Expert Queries are divided into three functionally distinct groups:

• Scene Queries: Inserted into the Vision-LLM (VLM) input, extracting scene-level priors from the VLM backbone.
• Motor Queries: Appended to the Prior Expert, aggregating denoising representations as motor priors, without perturbing the original denoising path.
• Action Queries: Placed in the Adaptation Expert, integrating both scene and motor queries for action generation.

These are implemented via a carefully designed masked attention pattern, shown in Figure 3, ensuring strict one-way information flow from prior sources to adaptation—preventing leakage or collapse of the frozen prior representations.

Figure 3: Masked attention design in PriorVLA. Colored cells mark permitted attention: information flow is structured to segregate forward paths, harvest priors through queries, and integrate them for adaptation.

Training Strategy

Only the Adaptation Expert, Expert Queries, and the VLM vision encoder are trainable; all other components, including the bulk of the VLM and the entire Prior Expert, are kept frozen. This selective training yields a significant reduction in updated parameters—PriorVLA adapts using only 25% of the trainable weights compared to full fine-tuning, directly addressing parameter efficiency and the risk of overfitting.

The model is optimized using a flow-matching mean-squared error loss on the Adaptation Expert’s output (prediction of action chunks), with no auxiliary or regularization objectives imposed on frozen components or queries.

Empirical Evaluation

Figure 4: Experimental coverage includes RoboTwin 2.0 (bimanual, OOD-rich simulation), LIBERO (single-arm, multi-suite manipulation), and real-world dual- and single-arm robotic platforms under in-distribution/ OOD and few-shot regimes.

Simulation Results

• RoboTwin 2.0: On a challenging bimanual manipulation benchmark, PriorVLA yields 77% (Easy/ID) and 53% (Hard/OOD) task success, outperforming the strong baseline $\pi_{0.5}$ by 10 and 11 points, respectively. The largest relative gains occur in the Hard/OOD regime.
• Across few-shot (10 demonstrations), standard (50), and large-data (250) settings, PriorVLA’s OOD advantage is robust—highlighting that preserved priors especially benefit adaptation under data scarcity.
• LIBERO: On the LIBERO benchmark, PriorVLA attains 99.1% average success, outperforming the prior state-of-the-art (OpenVLA-OFT: 97.1%, $\pi_{0.5}$: 96.9%), even in highly saturated settings.

Real-World Robotics

• Across eight real-world manipulation tasks (Franka and dual-arm AC-One platforms), PriorVLA achieves 81% ID and 57% OOD success with standard data, outscoring $\pi_{0.5}$ by 12 and 16 points, respectively.
• In the few-shot setting (10 demonstrations), PriorVLA realizes 48% ID and 32% OOD success (vs. $\pi_{0.5}$: 24%/10%), thus doubling ID and more than tripling OOD performance, underlining its practical impact when demonstration data is most scarce.

Ablation Analyses

Ablations underscore that improvement is not due to structural regularization or frozen branches per se: removing the Prior Expert, substituting a random branch, or making the Prior Expert trainable all degrade OOD performance. Disabling Scene or Motor Queries also eliminates gains, confirming that both the encapsulated priors and the learnable query interfaces are necessary for robust adaptation.

Theoretical and Practical Implications

PriorVLA reconceptualizes robot adaptation as learning to utilize rather than overwriting pretrained structure. Architecturally isolating priors and offering compositional query interfaces create a robust conduit for motor and perceptual knowledge transfer under limited supervision or distributional shift. This not only advances data efficiency and generalization, but also opens pathways for fundamentally new adaptation paradigms in large foundation-model-based robotics.

The results demonstrate a strong tension between generalization and specialization: PriorVLA offers a scalable template for navigating this trade-off by enforcing architectural constraints that make forward-pass priors available and learnable post-training.

Potential directions for future work include cache-efficient or distilled prior extraction to reduce inference cost, curriculum or continual learning approaches that dynamically rebalance prior use as new data arrives, and deeper mechanistic analysis of how and where task-agnostic priors flow into downstream action synthesis.

Conclusion

PriorVLA establishes a robust and theoretically grounded approach for prior-preserving adaptation in vision-language-action models. The clear empirical gains (particularly under OOD and few-shot settings), parameter efficiency, and non-trivial architectural design elevate its significance within the trajectory of robotic foundation model research. The method provides a blueprint for adaptation strategies that exploit, rather than erase, the structural affordances of pretraining—establishing a new bar for generalization and efficiency in embodied AI.