Primitive Subspaces Mediate Few-Shot Transfer in VLAs

Abstract: Deploying vision-language-action (VLA) policies in industrial environments requires the ability to teach new tasks at low cost, a property current VLAs lack, since each new task requires fine-tuning. We investigate whether primitive-aware training produces a transferable artifact: a learned library of sub-skills that can be composed at inference time, conditioned on a small number of demonstrations, to perform tasks the policy was never trained on. We train two VLA architectures with different inductive biases, OpenVLA and $π_{0.5}$, on the REASSEMBLE contact-rich assembly dataset under matched LoRA fine-tuning recipes and locked hyperparameters, varying training between flat trajectories and primitive-segmented episodes with primitive-specific language prompts. We hold out 6 object-task combinations from training and evaluate few-shot transfer: models receive $m \in {0, 1, 3, 5, 10}$ demonstrations of a held-out task and attempt execution without weight updates. We replicate across three training seeds and validate on a second dataset (LIBERO-Long). Primitive-trained models reach 78% of fine-tuned upper-bound performance with only m=3 demonstrations, while flat-trained models require m=10 demonstrations to reach the same level -- a $3\times$ sample efficiency gap that replicates across seeds, architectures, and datasets. To establish causation, we ablate the primitive-decodable subspace of hidden states and show few-shot transfer degrades by 32 percentage points while ablating a random subspace of equal dimensionality has no effect, indicating primitive representations are causally necessary rather than incidentally correlated with transfer. We identify and correct a methodological pitfall in evaluating chunked policies: family-wise inflation of single-step action-range gates produces order-of-magnitude higher false-failure rates against ground-truth human demonstrations.

• The paper demonstrates that primitive-aware training significantly improves few-shot transfer by recombining segmented sub-skills without additional fine-tuning.
• Methodology involves comparing flat vs. primitive-segmented data across two VLA architectures and employs causal subspace ablation to validate representational impact.
• Results show up to 2–3× sample efficiency gains with primitive supervision, though generalization may falter with out-of-vocabulary tasks.

Primitive Subspaces Mediate Few-Shot Transfer in Vision-Language-Action Policies

Summary

This paper addresses the limitations of current vision-language-action (VLA) policies regarding the efficient acquisition of new tasks in industrial manipulation contexts. Existing VLA approaches typically require fine-tuning on each novel task, which is resource-intensive and impractical for deployment in dynamic industrial settings. The authors propose that explicit primitive-aware training, in which demonstrations are segmented by sub-skills (primitives) and labeled with primitive-specific language prompts, enables VLAs to form a library of compositional sub-skills. At inference, the policy can recombine these primitives to execute unseen complex tasks after being conditioned on a small number of demonstrations, without weight updates.

Two VLA architectures—OpenVLA and $\pi_{0.5}$—are trained using either flat (unsegmented, task-level) or primitive-segmented data on the REASSEMBLE assembly benchmark and validated on LIBERO-Long. The models are evaluated on six held-out tasks via a few-shot protocol, varying the number of provided demonstrations ($m$), and are compared against several strong baselines. Importantly, the paper employs a subspace-ablation intervention and probing analysis to causally dissect the representation critical for compositionality and transfer.

Methodology

Experimental Design

A $2\times 2$ training protocol is adopted, crossing architectural choices (OpenVLA—the 7B-parameter, autoregressive approach; $\pi_{0.5}$—a flow-matching, chunked-action model) with data presentation modes (flat vs primitive segmentation). Fine-tuning conditions are tightly matched (e.g., LoRA settings, data splits, and seeds). The core experimental variable is whether the models see data segmented by primitives and explicitly cued by primitive-level natural language descriptions.

Held-out tasks are selected from the REASSEMBLE dataset to ensure that sub-skills (primitives or objects) required at test time have not been jointly seen but their constituents have been observed independently. The few-shot regime explores $m \in \{0, 1, 3, 5, 10\}$, with success evaluated via automatic end-state criteria.

Baseline Comparisons

The analysis benchmarks primitive-aware training against:

• Zero-shot primitive sequencing (external planner, no demonstrations),
• Flat-trained few-shot conditioning (demonstration encoding, but without primitive segmentation),
• Octo-style demonstration-conditioned policy [octo2024],
• Diffusion Policy chi2023diffusion,
• Full fine-tuning upper bound (50 demos per held-out task).

Analysis and Probing

To directly interrogate the learned representations, the authors train linear probes to decode primitive identity from hidden network states. They apply a subspace-ablation method: projecting away the primitive-decodable directions in the hidden state, then measuring performance in the few-shot regime. Control ablation using random subspaces of equal rank is performed to attribute causality.

Additionally, careful attention is given to evaluation methodology, notably by correcting a structural bias in action-range checking for chunked-action policies that otherwise inflates false failure rates.

Key Results

Few-Shot Transfer and Sample Efficiency

Primitive-aware models exhibit substantial gains:

• At $m=3$ demonstrations, primitive-trained cells reach $62\%-66\%$ success, compared to $31\%-34\%$ for flat-trained counterparts—a $2\times$ sample efficiency advantage.
• Primitive models with $m=3$ match or exceed the sample efficiency of flat-trained models at $m$0 (demonstration-equivalence advantage of 7x).
• This effect is robust across models, seeds, and both datasets (REASSEMBLE and LIBERO-Long) with primitive/flat sample efficiency ratios of up to $m$1.

Architectural Generality

Both autoregressive (OpenVLA) and chunked (flow-matching $m$2) backbones benefit comparably from primitive-aware training, indicating that compositionality emerges from training data organization rather than model class.

Causal Mediation by Primitive Subspaces

Subspace ablation targeting primitive-decodable dimensions in the hidden state drops few-shot success rates by $m$3 percentage points (from $m$4 to $m$5 for OpenVLA-primitive, and $m$6 to $m$7 for $m$8-primitive at $m$9), while ablation of random subspaces yields negligible changes. Thus, primitive representations are causally necessary for efficient few-shot transfer, not incidentally correlated.

Limitation: Out-of-Vocabulary Tasks

For tasks requiring primitive types absent from the vocabulary seen during training, primitive-aware models underperform: $2\times 2$0 for flat vs $2\times 2$1 for OpenVLA-primitive. This implies a strong bias in primitive-segmented models to fit observations to known primitives, impeding generalization outside the primitive library.

Probing and Representation Analysis

Linear probe MACRO-F1 decoding of primitive identity from late-layer activations is $2\times 2$2 for primitive-trained models but only $2\times 2$3 for flat-trained comparators, consistent across architectures. Representational similarity analysis reveals that primitive-level supervision induces clearer conceptual separation in latent space.

Evaluation Protocol Correction

The paper identifies a substantial methodological flaw in sequence-level gating for chunked policies, resulting in up to $2\times 2$4 false-failure rates for ground truth demonstrations. The corrected evaluation protocol yields more accurate and comparable performance statistics for policies outputting temporally extended action chunks.

Implications and Future Directions

Practical Implications

For deployment in adaptive industrial settings, the demonstrated $2\times 2$5 higher sample efficiency translates to lower data collection and operational costs. Attaining within $2\times 2$6 percentage points of full fine-tuned performance with just $2\times 2$7 demonstrations per novel task has substantial practical bearing for robotic instruction under changing requirements.

Theoretical Implications

These results refine the understanding of compositionality in imitation learning: primitive-aware supervision is sufficient to induce compositional hidden state representations amenable to in-context recombination, even without hierarchical architectural scaffolding. However, the compositional generalization is confined to the span of the primitive library; expanding primitive vocabularies or developing mechanisms for on-the-fly primitive induction remains open.

Causally, this work is among the first to directly intervene on latent conceptual subspaces within VLA policies and measure their necessity for in-context adaptation, connecting ongoing threads in interpretability and policy structure [belrose2023leace, ravfogel2020null].

Limitations

All evaluation is in simulation; sim-to-real transfer is untested. Architectural and task diversity is limited to two model classes and held-out configurations constructed from known objects and primitives. Generalization to unseen objects or real-world variants remains unexplored.

Speculation and Future Work

Subsequent advances are likely to extend these findings to richer primitive vocabularies, cross-domain generalization, and policy architectures that internalize primitive hierarchies modularly. Further, real-robot experiments and broader architectural evaluations (e.g., transformer-based, diffusion-based, model-free policies) will test the universality of the core result: that the explicit incorporation of primitive-level signals during imitation learning unlocks in-context compositional adaptation.

Conclusion

This study delivers strong empirical and causal evidence that primitive-level supervision mediates significant improvements in few-shot, in-context task transfer for VLA policies. Primitive-segmented training produces compact, linearly decodable subspaces that are causally necessary for efficient compositional recombination at inference, as established by direct subspace ablation. While this mechanism confers substantial gains for recomposing known skills, extension to genuinely novel primitive behaviors remains a challenge. The findings have immediate implications for scalable robot learning and emphasize the critical role of data structure in supporting compositional generalization in high-capacity policies (2605.30695).