Semi-Supervised Vision-Language-Action Model

Abstract: Vision-Language-Action (VLA) models enable robots to predict actions directly from visual observations and language instructions, but adapting them to new environments still depends on costly action-labeled demonstrations. To reduce this dependence, we study semi-supervised VLA adaptation under limited supervision signals, where only a small portion of trajectories contain robot actions and the remaining trajectories provide action-unlabeled vision-language observations. Unlike standard semi-supervised learning, the missing supervision is an embodied action signal that must be visually grounded, language-consistent, physically feasible, and temporally stable. To address this problem, we propose SemiVLA, a self-distilled teacher-student framework that learns from reliable pseudo-actions on unlabeled trajectories. SemiVLA introduces a VLA-specific reliability controller to assess vision-language alignment, action feasibility, and temporal transition consistency, and further updates the teacher through a Bottleneck-Projected Alignment Update to avoid noisy feedback contamination. With OpenVLA as the backbone, SemiVLA consistently improves multiple PEFT strategies across LIBERO and CALVIN. Under 10\% labeled trajectories, SemiVLA with Selective LoRA achieves 89.0\% average success on LIBERO, outperforming supervised LoRA by 8.0 points without extra inference cost.

• The paper introduces SemiVLA, a self-distilled teacher-student framework that leverages limited labeled trajectories to generate reliable pseudo-actions.
• It employs a VLA-specific reliability controller integrating vision-language alignment, action feasibility, and temporal consistency to filter pseudo-actions.
• Empirical results on benchmarks like LIBERO and CALVIN show significant improvements, reducing the need for dense action supervision in robotic adaptation.

SemiVLA: Semi-Supervised Vision-Language-Action Adaptation Framework

Motivation and Problem Formulation

Vision-Language-Action (VLA) models can predict complex robotic actions directly from visual inputs and language instructions by leveraging large-scale vision-language priors. However, robust adaptation of VLA models to new environments remains contingent upon costly, dense action-labeled demonstrations—a major deployment bottleneck. This paper formalizes a semi-supervised VLA adaptation scenario: only a limited fraction of trajectories ($\mathcal{D}_l$) are action-labeled, with the majority ($\mathcal{D}_u$) comprising action-unlabeled vision-language observations. The challenge is non-trivial since unlike conventional semi-supervised settings (where missing supervision is a class label), VLA adaptation lacks embodied action sequences that must be visually grounded, language-consistent, physically feasible, and temporally coherent.

To address this, the authors propose SemiVLA—a self-distilled teacher-student paradigm explicitly designed for self-supervised pseudo-action learning under limited supervisory signals. SemiVLA introduces reliability mechanisms tailored to the VLA context, bypassing pitfalls in conventional confidence-based pseudo-labeling, which can generate ungrounded or unfeasible actions for embodied agents.

Figure 1: SemiVLA employs a VLA-specific reliability controller and bottleneck-projected alignment update, mitigating noisy pseudo-action propagation compared to purely confidence-based self-training.

SemiVLA Framework and Method Components

SemiVLA comprises two interacting VLA policies—a student and a feedback-evolved teacher—built upon parameter-efficient fine-tuning (PEFT) strategies such as LoRA or Adapter. The student is initialized via supervised warm-up on a small labeled trajectory set, ensuring initial adaptation to the target robot's embodiment and action space.

Unlabeled trajectories are processed via pseudo-action generation from the teacher. The student is trained to match these pseudo-actions, but only when they are verified by a reliability controller reflecting vision-language alignment, action feasibility, and temporal transition consistency:

• Vision-Language Alignment: Measures the groundedness of the instruction with the visual observation.
• Action Feasibility: Rejects pseudo-actions incompatible with physical constraints or exhibiting unstable dynamics.
• Temporal Transition Consistency: Ensures pseudo-actions causally explain observed state transitions.

Reliability scores are used both as training weights and for explicit alignment losses, enforcing semantic grounding in adaptation.

A distinguishing aspect of SemiVLA is the Bottleneck-Projected Alignment Update for teacher evolution. Instead of conventional EMA, student updates are decomposed into semantic and action precision components, projected through bottleneck-stable visual-language subspaces and acted upon only if supported by feasible, temporally consistent pseudo-actions. This prevents contamination of the teacher by noise-sensitive updates and maintains action fidelity.

Figure 2: High-level overview depicting the asymmetric student-teacher optimization and reliability-controlled pseudo-action distillation in SemiVLA.

Empirical Results and Analysis

Benchmark Comparison and Numerical Gains

SemiVLA was evaluated across LIBERO, SIMPLER-ENV, and CALVIN benchmarks under a 10% labeled trajectory constraint. It consistently surpassed zero-shot and supervised adaptation baselines by substantial margins:

• With Selective LoRA, SemiVLA achieves 89.0% average success on LIBERO, outperforming supervised LoRA by 8.0 points, and zero-shot OpenVLA by 19.2 points.
• Gains are pronounced in long-horizon tasks, with SemiVLA + Selective LoRA reaching 82.3% on LIBERO-Long versus 70.3% for supervised LoRA.

Figure 3: SemiVLA demonstrates superior average performance (LIBERO, SIMPLER-ENV), and heatmap analysis confirms consistent improvement across task types.

SemiVLA's improvements are orthogonal to the PEFT strategy, showing efficacy across Adapter, LoRA, QLoRA, and selective parameter tuning. It also achieves efficient adaptation, delivering peak performance at minimal parameter and compute overhead.

Task-wise and Cross-Benchmark Robustness

Task-wise evaluation (per-task success rates) indicates increased consistency, particularly for long-horizon control scenarios where compounding errors and temporal instability are prevalent.

Figure 4: SemiVLA yields more uniform task-wise success rates compared to alternatives, showing robust adaptation in spatial, object, goal, and long-horizon task suites.

Cross-benchmark experiments on CALVIN substantiate generalization, with SemiVLA + Selective LoRA achieving a notable increase in completed tasks, indicating transferability of pseudo-action learning mechanisms.

Component Ablations and Reliability Analysis

SemiVLA's reliability controller demonstrably improves pseudo-action quality. Strict filtering (combining vision-language, action feasibility, and temporal consistency) reduces accepted pseudo-action ratio but maximizes performance—a clear indicator that quality trumps quantity in embodied adaptation.

Teacher evolution strategies are critically analyzed. Bottleneck-projected updates yield lower pseudo-action errors and higher success rates than reliability-aware EMA or frozen teachers, empirically validating the information bottleneck-based alignment filtering.

Figure 5: Performance under varying labeled ratios and ablation analysis reveals largest gains from VLA reliability controller and bottleneck-projected alignment updates.

Robustness analysis shows improved generalization under task distribution shifts and environmental perturbations (lighting, object, and distractor changes), demonstrating that SemiVLA's reliable self-distillation mechanisms induce stable vision-language-action correspondences.

Figure 6: Teacher update quality, reliability controller ablation, and efficiency-performance trade-off highlight the importance of aligned teacher evolution and strict reliability filtering.

Practical and Theoretical Implications

SemiVLA advances semi-supervised learning in embodied AI by demonstrating that action-unlabeled vision-language trajectories can be reliably leveraged for robotic adaptation, if pseudo-actions are selected via specialized reliability metrics and teacher updates are constrained by information bottleneck principles. The approach markedly reduces the requirement for costly action-labeled demonstrations and enhances robustness to domain shifts.

Practically, SemiVLA enables scalable deployment of VLA models to new robot platforms and environments where dense action annotation is infeasible. Theoretical implications include the necessity of multimodal reliability assessment for embodied adaptation—confidence-based metrics suffice in classification-dominant SSL but fail for action-centric robotic settings.

The bottleneck-projection principle may inspire future research in hierarchical feedback control and robust teacher-student self-distillation, extending beyond VLA to broader agentic learning paradigms.

Conclusion

SemiVLA introduces a principled semi-supervised adaptation framework for Vision-Language-Action models under limited action supervision, establishing rigorous pseudo-action reliability controls and a bottleneck-projected teacher evolution mechanism. Empirical evaluations confirm consistent performance improvements over supervised and classical semi-supervised baselines, substantial gains in low-label and few-shot regimes, enhanced robustness, and efficient adaptation. The methodology extends the frontier in data-efficient embodied AI, offering both strong practical adaptability and novel theoretical insights for future agent learning research (2606.21493).