- The paper introduces Anchor-Align VLA, combining layer-wise representation anchoring with language-action alignment to reduce catastrophic forgetting and improve control generalization across architectures and action heads.
- The method raises mean success from 61.0% to 71.9% on LIBERO-PRO and from 85.1% to 90.3% on LIBERO-Plus, while achieving 22.6% on position-swap tests where most baselines score near zero.
- Real-world xArm7 experiments show success improving from 28.3% to 54.2% with VLA-Adapter and from 36.7% to 60.0% with StarVLA, while semantic errors fall and successful rollouts become 1.7 times faster.
Motivation and problem statement
Vision-Language-Action (VLA) models are typically produced by finetuning a pretrained vision-LLM (VLM) on robot demonstrations via behavior cloning (BC), using either direct regression or flow-matching action heads. The premise of this recipe is that the VLM's internet-scale semantic priors—spatial layout, color, shape, affordances—transfer to the control policy. The paper's central empirical claim is that this premise fails under standard BC: finetuning progressively overwrites the very representations that make VLMs worth adapting. On a physical xArm7, a BC-finetuned policy trained on a green mug reaches for the green mug in 90% of trials even when instructed to pick up a pink one; on LIBERO-PRO's position-swap test, it almost always replays its memorized training trajectory.
The paper further argues that the standard remedy—co-training on web image-text data (VQA, captioning, scene description)—does not fix the problem. Co-training applies language and action losses to disjoint observations, so the two heads share a backbone but are never supervised on the same robot observation. The result is language-action misalignment: when the action head moves right, the language head may say "left." Notably, co-trained VLAs score 0% on LIBERO-PRO position swap, and the paper's diagnostic shows that even state-of-the-art co-trained models (ChatVLA, MolmoAct, Magma) exhibit alignment rates as low as 7–21% on spatial axes despite moderate per-head accuracies.
Method
The proposed method, Anchor-Align VLA, augments the standard BC loss with two objectives evaluated on the same robot observation:
Vision-Language Anchoring maintains a frozen copy of the pretrained VLM (the anchor) that processes each training batch in parallel. A squared-Frobenius distillation loss matches the trainable backbone's vision- and text-token hidden states to the anchor's at every decoder layer, averaged over layers. This is a direct representation-space anti-forgetting constraint, requiring no additional data or architectural change, and differs from prior work (MAPS, single-layer patch alignment, frozen encoders) in anchoring the full backbone at all layers against itself.
Language-Action Alignment programmatically converts each continuous action chunk into a discrete motion-direction label (one of six words: up/down/left/right/forward/backward) via chunk averaging, stationarity filtering (∥vˉ∥2​<τ masked out), and dominant-axis discretization. The last-layer hidden state of the final instruction token—the pre-action state—is projected through a learned linear map and the frozen pretrained LM head, and trained with cross-entropy against the direction label. Because the LM head is frozen, gradients flow only through the projection and LoRA adapters, encouraging pre-action representations to remain readable by the original vocabulary geometry. The total loss is Laction​+λanchor​Lanchor​+λalign​Lalign​ with λanchor​=0.1, λalign​=0.02.
The default instantiation uses Prismatic-Qwen2.5-0.5B with LoRA (rank 64), DINOv2+SigLIP dual features, and a bridge-attention regression head (VLA-Adapter); generality is checked on StarVLA with Qwen2.5-VL-3B and a GR00T FM-DiT flow-matching head.
Simulation results
On LIBERO-PRO and LIBERO-Plus, Anchor-Align outperforms every baseline on every axis. The most striking result is position swap, where all baselines score at or near zero (OpenVLA-OFT and MolmoAct: 0%, VLA-Adapter: 2.3%) while Anchor-Align reaches 22.6%. Mean LIBERO-PRO success rises from 61.0% to 71.9%; mean LIBERO-Plus from 85.1% to 90.3%, with the largest gains on background texture (+8.9), sensor noise (+7.4), robot initial state (+6.5), and lighting (+5.8). Five-seed evaluation confirms statistical significance (seed variance ≤ 0.7). On standard (in-distribution) LIBERO suites, the method also achieves the best results on most suites (e.g., 100% on Object).
On CALVIN ABC→D, Anchor-Align attains the best score at every chain length, with average rollout length 4.5 versus 4.3 for VLA-Adapter and five-instruction completion of 77.9% versus OpenVLA-OFT's 66.5%. The gain widens with horizon depth (+0.8 at one instruction, +4.8 at five), consistent with compounding grounding errors being suppressed by better alignment. This is achieved without large-scale robot-action pretraining.
Ablations establish two points. First, both losses are necessary and complementary: alignment alone yields 65.9/88.6 (PRO/Plus means) and anchoring alone 68.1/87.3, versus 61.0/85.1 for BC and 71.9/90.3 combined. Second, alignment is not generic regularization: Shuffle (permuted observation-to-label mapping) and Scatter (semantically meaningless target words) controls preserve the auxiliary classification task exactly but collapse back toward baseline performance, demonstrating that the gains require genuine per-observation language-action correspondence.
Real-world results
On a UFactory xArm7 with 150 teleoperated demonstrations and seven held-out tasks (20 rollouts each), Anchor-Align improves mean success from 28.3% to 54.2% under VLA-Adapter and from 36.7% to 60.0% under StarVLA, improving every condition on both architectures—evidence the gains are not tied to a specific backbone or action head. On the semantic-perturbation test (trained only on a green mug, evaluated on "pick up the pink mug"), the BC baseline collapses to its training prior (picks green in 90% of trials), whereas Anchor-Align picks the pink mug in 100% of trials (placing it successfully in 40%). Failure-mode analysis shows semantic errors eliminated (7→0), incorrect-object approaches nearly eliminated (10→1), and grasp failures halved (15→8); the slight increase in grasp-and-drop errors reflects more successful grasps overall. Successful rollouts are also 1.7× faster on average, attributed to more decisive action vectors near the grasp point.
Analysis: preservation, decodability, and misalignment
Two representational analyses substantiate the mechanism. Standard BC loses 94% of GQA accuracy within 10K finetuning steps, while Anchor-Align retains 70%; text-token CKA between finetuned and pretrained representations drops to 0.34 under BC (collapsing in output layers) but reaches 0.91 under Anchor-Align (0.95 for anchoring alone). Simultaneously, linear-probe action decodability peaks at R2=0.60 at layer 22—the highest of any method—showing that alignment routes action information through the decoder without destroying pretrained language geometry. Direction-concept probes improve most on the OOD object split (21.4→30.3%).
The paper also introduces a programmatic four-axis diagnostic (direction, orientation, grasp, task completion) extracted from raw trajectories without annotation. Applied to co-trained VLAs on real robot images, it reveals systematic misalignment: MolmoAct's action head achieves 77.1% on grasp detection and its language head 51.8%, yet the heads agree on only 37.8% of frames; on task completion, ChatVLA shows 65.6% language accuracy and 44.3% action accuracy but only 48.1% agreement. In the authors' own training setup, raising alignment from 16.8% to 78.4% turns the per-rollout alignment–success Pearson correlation from −0.03 to +0.51. This constitutes the first direct quantification of language-action inconsistency within co-trained VLAs.
Cost and limitations
A practical advantage is efficiency: relative to co-training with Knowledge Insulation (+95% wall-clock overhead, +5 GB memory, 25K external VQA samples), Anchor-Align adds +28% wall-clock, +0.7 GB, and no external data, while substantially outperforming it (90.3 vs. 57.1 LIBERO-Plus). The frozen-anchor forward pass scales sublinearly relative to backward passes, so relative overhead decreases with larger backbones.
Several limitations are acknowledged or evident. Position-swap generalization, though far above baselines, remains low in absolute terms (22.6%), indicating that anchoring mitigates rather than solves trajectory memorization. Real-world place success on the semantic-perturbation test is modest (40%). The direction-label construction covers only translational motion and discards near-stationary chunks; rotational, grasp, and completion axes are used diagnostically but not as training targets—an open question is whether multi-axis alignment supervision would yield further gains. The method assumes access to a frozen copy of the exact pretrained backbone and was validated primarily on relatively small (0.5B–3B) backbones; scaling behavior to frontier-scale VLMs is untested. Finally, the extension of anchoring and alignment to action-conditioned video world models is proposed as a hypothesis, not demonstrated.
Conclusion
This paper recasts VLA finetuning as a problem of preserving and aligning pretrained VLM priors rather than trading them away for control. Its two contributions—a layer-wise frozen-teacher anchoring loss and a demonstration-derived language-action alignment objective—are simple, data-free additions to BC that jointly address catastrophic forgetting and head-level inconsistency. The evidence spans two architectures, three simulation benchmarks, and a physical robot, with the strongest claims being the near-doubling of real-robot success, the collapse of all baselines to near-zero on position swap contrasted with 22.6% for the method, and the first quantitative demonstration that language-action misalignment is pervasive in co-trained VLAs and predictive of task failure.
