DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies

Abstract: Vision-Language-Action (VLA) models are often brittle in fine-grained manipulation, where minor action errors during the critical phases can rapidly escalate into irrecoverable failures. Since existing VLA models rely predominantly on successful demonstrations for training, they lack an explicit awareness of failure during these critical phases. To address this, we propose DreamAvoid, a critical-phase test-time dreaming framework that enables VLA models to anticipate and avoid failures. We also introduce an autonomous boundary learning paradigm to refine the system's understanding of the subtle boundary between success and failure. Specifically, we (1) utilize a Dream Trigger to determine whether the execution has entered a critical phase, (2) sample multiple candidate action chunks from the VLA via an Action Proposer, and (3) employ a Dream Evaluator, jointly trained on mixed data (success, failure, and boundary cases), to "dream" the short-horizon futures corresponding to the candidate actions, evaluate their values, and select the optimal action. We conduct extensive evaluations on real-world manipulation tasks and simulation benchmarks. The results demonstrate that DreamAvoid can effectively avoid failures, thereby improving the overall task success rate. Our code is available at https://github.com/XianzheFan/DreamAvoid.

• The paper introduces a test-time dreaming framework that intervenes at critical phases to predict and avoid failures in robotic manipulation.
• It utilizes a lightweight dream trigger alongside SDE-based action sampling and world-model evaluation to enhance policy reliability.
• Empirical results demonstrate significant success rate improvements in real-world and simulated tasks, validating the method's robust performance.

Critical-Phase Test-Time Dreaming for Failure Avoidance in Vision-Language-Action Policies

Motivation and Problem Statement

Vision-Language-Action (VLA) models have demonstrated strong generalist capabilities for robotic manipulation through direct multimodal mapping of visual and linguistic instructions to action spaces. However, these policies remain intrinsically brittle in fine-grained manipulation tasks, where brief but high-risk critical phases (e.g., insertion, alignment, or contact-sensitive interaction) can lead to irrecoverable failures upon minor local action errors. This brittleness is exacerbated by the reliance on successful demonstration data, leading to a lack of explicit failure awareness in standard VLA training paradigms.

The key insight motivating "DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies" (2605.11750) is that failures in robotic manipulation are usually localized to critical temporal phases rather than being diffuse throughout trajectories. Therefore, explicit, computation-heavy failure prediction and avoidance mechanisms need only be invoked during these sparse but sensitive phases, not throughout the entire task execution. The core hypothesis is that reflective spatiotemporal simulation—"dreaming"—performed at critical moments can bridge the gap between abstract action proposals and their downstream physical consequences.

DreamAvoid Framework

DreamAvoid introduces a novel, plug-and-play safety and correction module applied at test-time to any flow-matching VLA policy. The architecture consists of three principal components:

1. Dream Trigger: A lightweight classifier monitoring robot state and proprioceptive cues, utilizing a soft labeling scheme over hand-annotated or automated temporal landmarks to identify imminent entry into critical phases. This enables sparse, event-driven triggering with negligible overhead.
2. Action Proposer: During critical phases, instead of the standard deterministic policy rollout, the Action Proposer samples multiple diverse action chunk candidates via stochastic SDE-based inference over the flow field induced by the base policy. The use of SDE sampling over deterministic ODE trajectories dramatically increases candidate diversity, critical for exploring nuanced solution manifolds near failure boundaries.
3. Dream Evaluator: Each action candidate is rolled out via a large-scale pre-trained and post-trained action-conditioned world model (e.g., DreamDojo). The resulting video predictions are then evaluated by a value model, trained via real-dream joint optimization, to estimate progress deltas (and penalize catastrophes by enforcing negative labels on failure events). Candidate actions yielding the highest value predictions are executed.

   Figure 1: Qualitative example of the Dream Trigger, demonstrating probabilistic detection and intervention at temporally localized critical phases.

Autonomous Boundary Learning

A major advance over prior approaches is DreamAvoid's explicit inclusion of an autonomous boundary learning paradigm. This is motivated by the empirical observation that world models trained purely on successful demonstrations produce over-optimistic predictions at the boundary of failure—i.e., they cannot generate "nightmares." By deploying DreamAvoid and collecting empirical boundary and failure cases, the dataset is augmented beyond strict teleoperation success. This enables the Dream Evaluator to internalize nuanced distinctions between marginally successful and failed outcomes, essential for effective predictive policing of local errors during critical phases.

Figure 2: Visualization of boundary cases automatically collected during online deployment, showing near-boundary dynamics essential for robust failure discrimination.

Experimental Methodology and Results

Pilot Study: Necessity of Spatiotemporal Dreaming

A controlled study compared direct semantic reasoning of a VLM over "textified" action sequences (Direct Semantic Evaluation) with explicit spatiotemporal simulation using DreamDojo-generated futures (Explicit Spatiotemporal Dreaming) on a finely sensitive charger-plugging task. Direct Semantic Evaluation caused the VLM to make extremely conservative predictions, succeeding at recall (failures detected) but with unacceptably low precision due to inability to map abstract action sequences to physical outcomes. By contrast, spatiotemporal dreaming enabled precise discrimination of subtle failure cases, boosting accuracy from 57.5% to 80%, and vastly reducing false positives.

Real-World and Simulation Benchmarks

DreamAvoid and its ablated/alternative baselines were deployed on complex real-world tasks (e.g., Cup Sleeving, Charger Plugging, Cap Opening, Screw Insertion) and two simulation environments (LIBERO, SimplerEnv). Strong numerical results were observed:

• DreamAvoid-ABL (full method with autonomous boundary learning) achieved an average real-world task success rate of 72.5%, exceeding the base policy by 23.7 percentage points and outperforming GPC-RANK (always-on test-time action reranking) by 18.1 points.

  Figure 3: Real-world manipulation performance comparison across DreamAvoid variants, GPC-RANK, and baseline, showing marked gains in reliability, especially under tight critical-phase constraints.

• DA-ABL achieved 97.8% average success on LIBERO simulation tasks; on SimplerEnv, DA-ABL improved upon flow-policy and GPC baselines, with observed success rates of 63.6% (Bridge) and 80.7% (Fractal).

Ablation Analysis

• SDE-based action sampling was shown to be essential: repeated deterministic ODE rollouts suffered from candidate collapse, while SDE sampling enhanced candidate diversity and hence the chance of finding failure-avoiding actions.
• Removal of the Dream Evaluator (random action candidate selection) resulted not only in performance below DreamAvoid but even below the base policy, confirming that both diversity and accurate consequence evaluation are indispensable.

Figure 4: Example of Cup Sleeving, a failure-prone manipulation task where DreamAvoid sharply reduces lateral drift and collision failures.

Figure 5: SimplerEnv example; DreamDojo generates future videos for 4 action chunk candidates. Only candidate (4), correctly identified by DreamAvoid’s value model, yields successful task completion in simulation.

Practical and Theoretical Implications

Practically, DreamAvoid demonstrates that high-reliability, consequence-aware corrections in VLA-driven robotic manipulation can be achieved with minimal added compute by targeting intervention at sparse, automatically identified critical phases. This approach avoids the inefficiency and trajectory jitter of always-on online search/reranking methods, while offering much finer local discrimination than overtly semantic or goal-image-based verifiers.

Theoretically, the framework presents a robust architecture for temporally local risk-aware control: learning explicit decision boundaries through autonomous exploration, combined with world-model-based forward simulation and value-based selection, paves the way for safer closed-loop robotic autonomy. The methodology bridges low-level trajectory optimization and high-level semantic reasoning by translating abstract action space into interpretable, physically plausible futures for value assignment.

Future Directions

Key open problems include:

1. Extension to other forms of policies (e.g., World Action Models or more classical control architectures) as a generic plug-and-play safety layer.
2. Reducing test-time latency via fast video prediction or compressed latent dreaming.
3. Closing the loop by integrating high-fidelity dreamed corrections directly into policy updates, for continual self-improvement via generated data.

Conclusion

DreamAvoid introduces a computation-efficient, failure-aware, critical-phase intervention method for VLA policies. By decoupling intuitive execution from selective, high-fidelity test-time dreaming only at critical junctures, and by incorporating autonomous boundary learning for failure discrimination, DreamAvoid significantly improves robustness and reliability of robot manipulation both in real and simulated domains. The results highlight that targeted spatiotemporal dreaming, rather than pervasive always-on cost, is instrumental in achieving strong, interpretable failure avoidance in modern embodied policies.