- The paper shows that action-grounded signals, such as movement descriptions and point trajectories, improve learning more reliably than high-level reasoning alone, with full ECoT delivering an 8.2-point gain in controlled tests.
- The paper introduces ERVLA, which uses reasoning dropout, knowledge truncation, auxiliary action queries, and flow-matching control to internalize reasoning, achieving 86.9% success on LIBERO-Plus and 53.2% on VLABench.
- The paper finds that autoregressive CoT prefixes do not scale because noisy, variable-length reasoning causes exposure bias and compounding errors, while representation-level CoT training improves semantic and long-horizon real-world manipulation.
Overview
This paper presents a systematic re-examination of embodied chain-of-thought (ECoT) reasoning for vision-language-action (VLA) models, conducted at a scale substantially larger than prior work. The authors curate the largest embodied CoT corpus reported to date—978,743 trajectories, 226.3M samples, and 2,592.5 hours of data spanning BridgeData V2, Fractal, DROID, MolmoAct, and AgiBot World—and use it to answer three questions: which CoT signals actually aid action learning, whether embodied CoT serves as an effective transfer interface from VLM backbones to VLA policies, and whether CoT supervision scales as a pre-training signal. Their central finding is that embodied CoT is valuable not as a test-time verbalization channel but as a training signal that reshapes policy representations; explicit autoregressive CoT prefixes do not scale reliably and suffer compounding errors. Building on these findings, they propose ERVLA, an end-to-end trainable VLA that internalizes reasoning via CoT dropout, auxiliary action-query regression, knowledge truncation, and flow-matching continuous control (2606.03784).
Embodied CoT data construction
The corpus is organized around a hierarchical schema decomposing manipulation reasoning into four roles: task understanding, spatial grounding, subgoal planning, and action-oriented motion description. A staged auto-labeling pipeline first segments trajectories using end-effector motion statistics and gripper transitions, then queries Qwen3.5 once per episode to produce globally consistent goals, plans, and subtask assignments, avoiding frame-wise drift. Frame-level acting fields are derived from future motion: short-horizon action chunks are verbalized into thresholded movement descriptions ("move back 3 cm, keep gripper open"), suppressing controller jitter.
Two design choices are notable for their effect on label reliability. Gripper positions on static cameras are obtained by geometric projection of the calibrated end-effector pose rather than detector-based localization, since grippers are frequently occluded or blurred; invalid projections are omitted entirely. Object bounding boxes from LLMDet are annotated sparsely on keyframes rather than densely per frame. Both choices directly address what the authors term CoT contamination: dense but inaccurate grounding labels impose inconsistent supervision on semantically similar frames, degrading adaptation to control. The dataset is also, to the authors' knowledge, the first at this scale to preserve multi-view spatial correspondence (base, front, wrist views) and bimanual arm identity in its spatial fields.
Which CoT signals help action learning
Controlled ablations on VLABench with a fixed Qwen3-VL-4B backbone and autoregressive CoT+FAST interface yield three findings with direct design implications.
First, action-related fields dominate high-level understanding. Isolated descriptive fields (Goal, Planning, Subtask, Reasoning) all slightly hurt performance under direct training—for example, Goal reduces average success by 1.2 points relative to a no-CoT baseline of 19.0. In contrast, Movement adds +4.1 and point trajectory +4.8, and combinations coupling semantics with concrete motion guidance perform best: Subtask+Movement+Point trajectory reaches +7.4, and full ECoT reaches +8.2. High-level reasoning alone yields marginal or negative gains; it becomes useful only when grounded in executable cues such as end-effector movement and image-space trajectories.
Second, pre-training amplifies contamination from noisy coordinate fields. With Bridge ECoT pre-training, detector-generated gripper coordinates and bounding boxes become actively harmful, causing drops of −5.6 and −6.1 respectively. Reasoning dropout (CoT present with probability 0.5 during training) largely neutralizes this damage, shrinking those drops to −0.8 and −1.0 while preserving useful supervision—the point-trajectory gain improves from +1.4 to +3.0 with dropout. The authors argue dropout is not merely regularization but training scaffolding that reshapes internal representations.
Third, and most consequentially, explicit CoT does not scale reliably under autoregressive action decoding. Expanding the pre-training mixture from Bridge alone to include Fractal, MolmoAct, and DROID monotonically degrades performance across all VLABench tracks, reaching drops of −3.6 (in-distribution), −3.0 (cross-category), and −3.4 (texture) in the four-source setting. The mechanism is exposure bias: variable-length, noisy reasoning traces perturb the action prefix, and CoT errors propagate directly into subsequent action tokens. This is a strong negative result against naive scaling of prefix-style ECoT.
A complementary transfer study across nine VLM backbones (PaliGemma-2-3B through Qwen3-VL-8B) under a fixed StarVLA/FAST recipe shows that without CoT, backbone strength correlates only weakly with downstream control performance; with explicit embodied CoT, stronger backbones transfer more consistently, with the Qwen3-VL series dominating. Embodied CoT thus functions as a transfer interface converting pretrained semantic priors into action-relevant representations.
ERVLA architecture and training
ERVLA instantiates the conclusion that reasoning should reshape representations rather than burden decoding. It uses Qwen3-VL-4B as the reasoning backbone and a 36-layer diffusion transformer (DiT) for continuous actions via rectified flow matching, conditioned on per-layer VLM key-value caches rather than pooled features. Three components distinguish it:
- Auxiliary choice policy: trainable action-query tokens let the VLM predict N=5 candidate continuous action chunks plus scores, supervised by best-candidate MAE and stop-gradient score regression. Unlike knowledge-insulated designs, the flow loss back-propagates into the VLM, making the system fully end-to-end.
- Knowledge truncation: although the VLM processes both semantic content and appended control-query turns, the DiT conditions only on the semantic-prefix cache, preventing shortcut copying of synthetic query tokens.
- Reasoning dropout: each sample is converted to /cot or /no_cot with probability 0.5, so explicit CoT is an optional training condition; at inference, actions are predicted directly without mandatory CoT decoding, bypassing autoregressive instability entirely.
The total objective combines token-level CoT cross-entropy, rectified flow loss, choice MAE, and score regression. Pre-training uses 120,000 steps over the 226.3M-sample mixture with 60-dimensional padded action/state interfaces unifying single-arm and bimanual data.
Benchmark results
On LIBERO-Plus, evaluated zero-shot after LIBERO-only post-training, ERVLA achieves an 86.9% average success rate, surpassing π0.5 (85.5%) and far exceeding OpenVLA-OFT (69.6%), π0-FAST (61.6%), and classic ECoT-style models (~25%). It reaches 100% success on the background and lighting variations of the Spatial track. Ablations attribute the gains to the proposed interface rather than CoT alone: removing the choice branch costs roughly 10–15 points depending on configuration, and removing knowledge truncation causes large drops consistent with shortcut exploitation.
On VLABench, ERVLA attains 53.2% average success rate (70.4 intention score), outperforming π0.5 (48.1%), X-VLA, and ACoT-VLA, with the largest margins on cross-category transfer (47.0 vs. 38.2) and instruction following (58.0 vs. 48.2). Scaling curves show ERVLA improving steadily with more CoT pre-training data on both benchmarks, whereas autoregressive CoT+FAST variants degrade and isolated VLM+DiT designs saturate—directly supporting the claim that CoT scales when reasoning and action losses are jointly optimized in representation space.
Real-world evaluation
Real-robot experiments use two task families (drawer placing, tabletop clearing) across four tiers—Basic, Distractors, Semantic, Long-horizon—with 20 tasks and five trials each. On Basic tasks, ERVLA (96 SR) and π0.5 (97 SR) are comparable, indicating that strong continuous-action policies already suffice for clean manipulation. The gap emerges under distributional stress: on the Semantic tier ERVLA improves success from 31 to 42 and progress score from 45 to 58 over π0.5, and on Long-horizon it raises progress score from 38 to 55. Autoregressive ECoT performs worst overall (24 average SR), and visual-prediction (WorldVLA) and latent (UniVLA) interfaces remain limited because they do not resolve fine-grained semantic intent. These results support the paper's claim that real-world robustness derives less from low-level action quality than from whether reasoning supervision becomes grounded, action-aware representation.
Limitations and open questions
The paper is candid that the value of reasoning pre-training is bounded by annotation quality. Linguistic fields are robust but sometimes underspecified; dense grounding fields remain vulnerable to detector error, calibration bias, and occlusion despite sparse annotation and geometric projection. Reasoning dropout and knowledge truncation mitigate but do not eliminate this dependence, and the authors identify uncertainty-aware annotation—teaching the model when to trust each signal—as an open problem. Additionally, the study treats reasoning as an offline pre-training signal; how much reasoning should be dynamically refreshed during long-horizon execution, including memory of completed subtasks and recovery from failures, remains unresolved. No formal statistical significance testing or error bars accompany the results, and code, data, and checkpoints were not yet publicly available at submission time.
Conclusion
This work reframes embodied chain-of-thought as a representation-shaping training signal rather than a test-time output format. Its empirical contributions—a field-level taxonomy showing that only action-grounded linguistic signals reliably help, a demonstration that explicit autoregressive CoT prefixes fail to scale due to contamination and compounding errors, and an architecture (choice policies, knowledge truncation, reasoning dropout, end-to-end flow supervision) that makes CoT pre-training scale monotonically—are supported by state-of-the-art results on LIBERO-Plus (86.9%) and VLABench (53.2%) and by real-world gains concentrated precisely where reasoning should matter most: semantic ambiguity and long-horizon execution.