Unleashing More Actions via Action Compositional Training for VLA Models

Abstract: Vision-Language-Action models excel at robotic manipulation, driven by the scale and diversity of demonstration data. However, standard training paradigms often cause VLA models to severely overfit to specific behavioral patterns, rendering them unable to generalize to out-of-distribution scenarios even when those scenarios merely require novel combinations of identical sub-skills. While expanding datasets can mitigate this overfitting, acquiring high-quality robot data remains notoriously labor-intensive and cost-prohibitive. To resolve this impasse without expensive human teleoperation and to truly unleash more actions,i.e., enable VLA models to compose known sub-skills into a much broader set of executable behaviors beyond the original demonstrations-we propose ACT-VLA (Action Compositional Training for VLA Models), an offline data augmentation framework that leverages the model's latent task representations to synthesize novel, physically valid demonstrations directly from existing tasks for policy training. By eliminating additional manual data collection, our method automatically expands the training distribution and mitigates overfitting. We evaluate our approach on challenging manipulation tasks in simulation. Experiments demonstrate that while baseline VLA models generalize poorly due to original distribution overfitting, policies trained with our synthesized data achieve substantially higher success rates, validating that leveraging existing tasks for automated demonstration synthesis provides an effective, scalable, and data-efficient route to broadening VLA generalization.

• The paper introduces ACT-VLA, a novel method that synthesizes compositional demonstrations via latent interpolation to overcome the limitations of fixed training datasets.
• The methodology leverages offline augmentation with rigorous trajectory filtering to achieve significant out-of-distribution gains, including improvements of up to 52.7 percentage points.
• The approach requires no new human data or architectural changes, offering a scalable solution to enhance robotic manipulation in vision-language-action models.

Action Compositional Training for Vision-Language-Action Models: A Data-Centric Path to Compositional Generalization

Introduction

The paper "Unleashing More Actions via Action Compositional Training for VLA Models" (2607.00351) systematically addresses the compositional generalization bottleneck in Vision-Language-Action (VLA) models for robotic manipulation. Contemporary VLA policies excel at diverse tasks but are fundamentally limited by overfitting to the combinatorial sparsity present in manually acquired demonstration datasets. This work introduces ACT-VLA, a data-centric offline augmentation protocol, that produces compositional demonstrations by leveraging internal latent task structures, enabling robust generalization to unseen skill combinations without requiring any new human data acquisition or architectural modifications.

Problem Formulation and Motivation

Modern VLAs flexibly integrate perception, linguistic instruction following, and low-level control. However, their mapping from language/vision inputs to motor commands is highly sensitive to training distribution coverage. Canonical datasets consist of isolated demonstrations for a finite set of behavioral primitives (e.g., "place the mug on the circular plate") but rarely cover all possible object–target pairings or temporal compositions. As a result, existing policies fail on test cases that require novel recombinations of learned sub-skills, even when the constituent primitives are individually mastered.

The paper gives a concrete instantiation: training only on mug-to-circular-plate and cube-to-rectangular-tray leads to test-time catastrophic failure on cube-to-circular-plate or mug-to-rectangular-tray.

Figure 1: Illustration of compositional generalization via object–target re-pairing, highlighting the training distribution's inability to cover out-of-distribution object–target combinations.

ACT-VLA: Action Compositional Training Methodology

Compositional Demonstration Synthesis via Latent Interpolation

The method pivots on exploiting the latent compositional structure implicitly encoded within trained VLA policies. Instead of collecting additional demonstrations, ACT-VLA systematically synthesizes rollouts for unseen compositions by performing text latent interpolation (TLI) between task representations. This extends prior work which utilized TLI solely for inference-time control [livlas].

The key insight is to perform offline interpolation between the latent encodings (task-specific text hidden states) corresponding to two known base tasks. By interpolating these latents temporally within the model’s transformer, ACT-VLA generates trajectories that enact smooth behavioral transitions between base skills, thereby synthesizing valid compositional demonstrations for novel instructions absent from the original dataset.

Pipeline and Quality Control

The comprehensive ACT-VLA pipeline comprises:

Figure 2: The overall pipeline of ACT-VLA, featuring latent-guided demonstration synthesis, rigorous trajectory filtering, and policy retraining on the augmented dataset.

1. Representation-Guided Trajectory Synthesis: Offline generation of compositional task rollouts via latent steering.
2. Data Recording/Processing: Capturing RGB, proprioception, language instruction, and action at each timestep, where the new language instruction is deterministically recomposed from base task instructions to match the trajectory.
3. Trajectory Filtering: Strict selection of only physically valid, task-completing rollouts using simulator-based validation and episode-length constraints, ensuring no noisy labels pollute the retraining corpus.
4. Balanced Augmentation: Matching the number of synthesized demonstrations to the underlying in-distribution suite's size to avoid mode collapse or catastrophic forgetting.

Dataset Construction

The paper targets the LIBERO simulation benchmark, including out-of-distribution suites specifically constructed by object–target recombination protocols, ensuring rigorous compositional generalization assessment.

Figure 3: Construction of compositional tasks by re-pairing objects and targets from distinct base tasks, following a systematic protocol to build OOD evaluation scenarios.

Experimental Evaluation

Main Results

ACT-VLA is evaluated against a comprehensive set of SOTA VLA models, as well as approaches employing online TLI at inference (without retraining). The main results are summarized below:

• ACT-VLA achieves 88.2% success on Spatial-OOD and 95.6% on Goal-OOD, representing absolute gains of +52.7 and +49.0 percentage points, respectively, over the strongest non-compositional VLA baseline ($\pi_{0.5}$).
• Training-time data augmentation outperforms inference-time TLI (which achieves 81.0%/85.0% on the same OOD suites), both in terms of success rate and by eliminating test-time overhead.

The improvements are realized without architectural changes, external planners, or additional teleoperation, indicating that compositional policy transitions are efficiently internalized simply through optimized data exposure.

Ablation Analysis

By deploying the protocol on multiple VLA backbones and comparing to inference-time-only synthesis, the method's robustness to model capacity and TLI location is established. Direct training on synthesized compositional data yields consistent OOD gains without sacrificing in-distribution accuracy, demonstrating the significance of per-task augmentation and filtering over naive rollout aggregation.

Implications and Theoretical Considerations

ACT-VLA offers a scalable, automatable path to overcoming the combinatorial explosion of required demonstrations for real-world VLA policies. Unlike decomposition/planning-centric or hierarchical approaches, ACT-VLA requires neither explicit primitive libraries nor external planners—the compositional reasoning is embedded directly within the policy. The protocol’s efficacy depends on the semantic disentanglement capacity of the base VLA latent space: stronger pretraining yields higher-fidelity rollouts and thus greater compositional generalization.

Practically, ACT-VLA eliminates the prohibitively high marginal cost of rare-scenario coverage, accelerating both simulation-based model scaling and setting the foundation for future sim-to-real transfer without exponential data collection burden.

Limitations and Future Directions

Several open challenges persist:

• Sequential Composition Scaling: The current protocol is evaluated on two-skill compositions; extension to longer-horizon or recursively chained tasks may incur exponentially escalating rollout rejection rates and require additional filtering or curriculum strategies.
• Backbone Dependence: Synthesis quality is bottlenecked by the semantic factorization in the original model's representation. Future works could incorporate auxiliary objectives targeting improved disentanglement during VLA pretraining.
• Sim-to-Real Transfer: All validation is conducted in simulator. Further work is required to analyze the transferability and robustness of synthetic compositional demonstrations to real hardware platforms.

Conclusion

ACT-VLA demonstrates that robust compositional generalization in VLA models can be achieved not through architectural expansion or extensive manual data collection, but via principled, automated offline demonstration synthesis. By combining latent-driven policy traversal with rigorous trajectory validation and data-centric retraining, ACT-VLA establishes a scalable paradigm applicable to future generalist robot learning systems targeting the open-world combinatorial action space.

(2607.00351)