Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

Abstract: Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

• The paper introduces SVA, a framework that leverages Monte-Carlo Tree Search distilled into a Q-model to enable consequence-aware decision making in frozen VLA models.
• It demonstrates up to 40-point performance gains and improved scalability compared to larger models by focusing on long-horizon action evaluation.
• Empirical results on multiple benchmarks show that decoupling action generation from evaluation preserves model generality while enhancing task success.

Distilling Tree Search for Robust Action Evaluation in Frozen VLA Models

Background and Motivation

Vision-Language-Action (VLA) models aim to endow robots with general-purpose policies by leveraging large-scale vision-language pretraining. While VLAs achieve breadth in their embodied capabilities, their generalization remains brittle—particularly when compared to their language-only (LLM) and vision-language (VLM) analogues. Existing remedies typically rely on costly post-training, such as supervised fine-tuning (SFT) or reinforcement learning (RL) with dense rewards. However, these approaches dilute the generality achieved through pretraining and require substantial compute.

The central insight of "Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models" (2607.03751) is that frozen VLAs, while failing in single-shot execution, already encode successful behaviors in their output distributions. The bottleneck is not primarily action generation, but the lack of reliable action evaluation: VLAs produce plausible actions but cannot anticipate their long-term consequences. Pass@k diagnostics reveal a stark improvement from 33% at pass@1 to 92% at pass@32, showing the latent capacity for successful behavior if only the best actions could be selected.

SVA: Search, Value, and Act Framework

The authors introduce SVA, a three-stage framework designed to equip frozen VLAs with consequence-aware decision making without further backbone optimization. The core pipeline involves:

• Search: Monte-Carlo Tree Search (MCTS) is deployed in simulation to comprehensively explore the VLA policy's output, collecting diverse trajectories annotated with empirical returns.
• Value: The data from MCTS is distilled into a lightweight Q-value model (Q-model), trained to predict the expected consequence of any candidate action from the current state. This evaluator generalizes the information obtained during search.
• Act: At inference, the frozen VLA proposes multiple candidate actions. The Q-model ranks these candidates, selecting the action expected to maximize long-term return—without requiring access to the simulator.

This decoupled proposal/evaluation design preserves the VLA backbone's generality while substantially improving task performance.

Figure 1: Overview of SVA. (a) MCTS explores the VLA policy’s output in simulation. (b) A lightweight Q-model learns to predict action consequences from search data. (c) At test-time, the Q-model selects among VLA-generated candidates without simulator calls.

Empirical Results and Ablations

SVA is comprehensively evaluated across several embodied reasoning and manipulation benchmarks (EB-Habitat, EB-Navigation, SimplerEnv, RoboTwin 2.0) and model families (GPT-4o, Qwen3.5 4B/9B/27B, Gemma 4-E4B-it, $\pi_0$, $\pi_{0.5}$, OpenVLA). Key findings include:

• Consistent Performance Gains: SVA delivers absolute improvements up to 40 points (e.g., Qwen3.5-4B on EB-Habitat), particularly on long-horizon or visually grounded tasks. Improvements are general across Benchmarks and architectures.
• Superiority over Parameter Scaling: Employing SVA with a 9B VLA outperforms a 27B VLA by 7 points at 27% lower inference latency, demonstrating that scaling test-time evaluation is more cost-effective than growing backbone size.
• Comparison with Other Verification Baselines: SVA surpasses contemporary test-time selection schemes (e.g., RoboMonkey), both in overall and task-specific success rates, due to its long-horizon value estimation rather than single-step preferences.

Figure 2: Success rates on SimplerEnv and RoboTwin show that SVA consistently surpasses both base policies and strong reranking baselines, especially on tasks demanding precise sequential manipulation.

Ablation studies confirm:

• Removing MCTS or the Q-model leads to significant drops in performance (SVA 56.11%, w/o Q-model 43.33%), explicitly highlighting the necessity of value learning grounded in structured search.
• Multi-candidate evaluation is critical; reranking single-shot proposals regresses to baseline VLA accuracy.

Analysis of Action Evaluation and Scaling Behavior

The diagnostic pass@k study crystallizes that VLA policies often contain successful actions, but lack mechanisms to disambiguate among them. SVA leverages test-time compute to mine these hidden successes. The return on scaling candidate sampling is sub-linear, which makes increasing the candidate pool practical for online agents: each doubling of $N$ yields only a modest latency increase (10.6–36.6%), but robust gains in success rate.

Moreover, SVA sidesteps the high simulation/rollout costs that burden world model-based methods or online RL by compressing search into a fast Q-model evaluator. This enables real-time deployment on actual robots, provided initial training leverages simulations supporting action evaluation.

Qualitative Robustness and Generalization

The authors provide qualitative case studies on severe VLA failure modes:

• Distractor-Aware Following: For an instruction with a salient distractor, the base policy is led astray, but SVA selects a goal-directed plan, succeeding quickly.

Figure 3: Case 1 – SVA correctly filters out a distractor and chooses the optimal plan, completing the task efficiently while the base policy fails with repeated invalid actions.

• Spatial Relation Grounding: VLAs can fail to ground spatial relations, issuing repeated invalid attempts. SVA, by evaluating long-term outcome, grounds the relation and completes the rearrangement.

Figure 4: Case 2 – SVA leverages the Q-model to correctly ground spatial relations, avoiding cycles of fruitless actions typical of the base VLA.

Implications, Limitations, and Future Directions

The SVA paradigm provides a strong practical and theoretical case for framing VLA progress as an action evaluation challenge. By preserving generalist competence and requiring only lightweight augmentation, SVA is immediately applicable to any frozen VLA backbone. Strong scaling results suggest that, for embodied agents, computation at test time should be focused on outcome-aware reranking rather than brute force model scaling.

SVA’s reliance on simulation for Q-model training and decoupled search and value learning currently limit direct applicability to domains lacking simulators or reward signals. Unification of search and value in an online loop, more sample-efficient value extraction, and validation on physical robots are suggested as impactful next steps.

Conclusion

"Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models" establishes that evaluation bottlenecks—rather than generation—are the primary limiters of frozen VLA performance. The SVA framework, by efficiently distilling tree search into a deployable Q-model, enables robust and scalable performance gains on a variety of embodied tasks, offering an effective alternative to expensive backbone updates. Given the generality and model-agnostic nature of this approach, it is likely to form a foundational strategy for future deployment and improvement of generalist robotic policies.