Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement

Abstract: Robots deployed in the real world should learn from their experience and improve over time. This requires a mechanism of practicing and learning from feedback. In this paper, we propose VERITAS, a generator-verifier framework for generalist robot policies for inference-time policy steering and self-improvement. We use a pre-trained generalist robot policy as a generator'' and pair it with a gradient-freevisual verifier'' that evaluates actions at inference time. This framework enables inference-time steering that improves policy performance without additional training. We demonstrate that inference-time verification consistently outperforms vanilla generalists without training on additional demonstration data. Additionally, we demonstrate that the verified rollouts provide effective supervision for offline policy improvement: policies fine-tuned on verified self-generated trajectories achieve consistent performance gains. Notably, we find that post-training with verified rollouts achieves comparable efficiency to expert demonstrations, while requiring no human interventions. Our results highlight inference-time verification as a practical and scalable mechanism for improving robotic policies during deployment.

• The paper introduces VERITAS, a generator-verifier framework that selects optimal action chunks at inference time to enhance task performance.
• It leverages visual reasoning to rank multiple candidate actions, mitigating covariate shift and reducing reliance on human demonstrations.
• Experimental results in simulation and real-world settings show that VERITAS significantly boosts success rates and data efficiency for robotic policies.

Visual Verification Enables Inference-Time Steering and Autonomous Policy Improvement

Introduction

"Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement" (2606.18247) introduces VERITAS, a generator-verifier framework targeting scalable self-improvement and inference-time enhancement for generalist robot policies. Rather than relying on additional human-collected demonstrations for post-deployment improvement, VERITAS leverages a plug-and-play visual verifier to select, at each decision point, from multiple candidate action chunks generated by a stochastic policy. This design allows improved task execution solely through inference-time computation. Furthermore, successful verifier-guided rollouts are logged and used as high-quality, autonomous data for iteratively fine-tuning the underlying policy. The framework mitigates the covariate shift challenges of imitation learning by ensuring fine-tuning data remains on-policy and kinematically feasible.

Figure 1: Overview of VERITAS as a generator-verifier framework, where inference-time verification steers execution and logged successful rollouts enable continual self-improvement.

VERITAS Framework and Methodology

The framework consists of two primary components:

• Generator: A pre-trained, generalist, vision-language-action (VLA) policy (e.g., $\pi_\theta$) functions as a stochastic action sampler. At each decision step, it produces $N$ candidate action chunks, each representing a feasible trajectory in the action space.
• Verifier: Implemented as a gradient-free, policy-agnostic module, the verifier provides scalar scores for each candidate action chunk based on its physical plausibility and semantic alignment with the high-level task. VERITAS emphasizes instantiating the verifier with visual reasoning, realized via a VLM-generated pixel-space waypoint trace or, alternatively, by geometric or heuristic constraints.

The control loop first generates action candidates, the verifier ranks them according to visual or semantic consistencies, and the highest-scoring action chunk is executed. The process is strictly inference-time—no parameter updates are required for performance gains over the base policy.

Figure 2: Implementation of inference-time steering; the VLM-based visual verifier proposes waypoints and evaluates candidate action traces.

Inference-time steering is realized using a best-of-$N$ selection mechanism. The amortized computational cost (linear in $N$) allows substantial performance gains with moderate impact on inference latency, and the batch sampling enables efficient candidate evaluation.

Autonomous Policy Improvement via Verified Rollouts

Successful verifier-guided task executions are retained as high-quality trajectory data. This establishes a data flywheel: deployment generates autonomous rollouts, which are then used to further offline fine-tune the policy with supervised behavioral cloning. Critically, since both the sampler and verifier are implemented without human involvement, this creates a scalable pathway for policy self-improvement.

Because autonomous trajectories remain close to the policy’s own exploration distribution, distribution shift is minimized, addressing issues typically encountered in imitation learning from off-policy or human-centric data.

Experimental Evaluation

Inference-Time Steering

Experiments span both simulation and real-world environments on diverse manipulation tasks, using multiple generalist policies (e.g., $\pi_0$-Bridge, $\pi_0$-DROID). Results consistently show that inference-time verification with VERITAS elevates baseline policy success rates.

• In simulation, across multiple tasks and verifier architectures, the largest gains are evidenced on task instances exhibiting either high ambiguity or where policy uncertainty is substantial.
• In real-world experiments, VERITAS delivers robust and statistically significant performance improvements relative to V-GPS and other baselines.

  Figure 3: Example rollouts in both simulated (WidowX) and real (FR3-DROID) settings, illustrating VERITAS inference-time steering and verifier-guided action selection.

  

Figure 4: Quantitative performance on SIMPLER tasks, showing consistent success rate improvements with VERITAS over standard execution and alternative verification schemes.

Figure 5: Real-world evaluation demonstrating the performance advantage of inference-time steering with VERITAS compared to V-GPS across two policies and four task scenarios.

Offline Self-Improvement

Fine-tuning with trajectories curated via autonomous verification further improves the baseline. The measured gains, averaging around 10% in simulation and substantial in hardware trials, highlight that verifier-selected data can approach or match the efficacy of expert-provided demonstrations. Notably, the success rate improvements are most pronounced on tasks where the policy’s action distribution is innately diverse or uncertain.

Data Efficiency and Comparison to Human Supervision

A systematic ablation (20–100 demonstration regime) shows that verifier-curated, policy-generated autonomous data produce learning curves comparable to or superior to human teleoperation rollouts.

Figure 6: Data efficiency comparison in real-world fine-tuning, demonstrating that verifier-curated autonomous rollouts can outperform human demonstrations with limited data budgets and remain competitive at larger scales.

Reference Analysis and Saturation

Ablations highlight that the advantage of inference-time steering saturates with increasing $N$. Further, the framework’s performance is robust to open-loop chunk horizons up to 10 steps, enabling practical deployment at high control frequencies.

Figure 7: Ablation on action chunk horizon demonstrating robustness of VERITAS to open-loop step parameterization.

Empirically, VERITAS outperforms naive data augmentation strategies (e.g., RoboMonkey) that do not enforce semantic or temporal consistency. Direct single-step perturbation, as in RoboMonkey, can degrade performance in tasks demanding complex sequential reasoning.

Figure 8: Comparison with RoboMonkey, showing mixed or regressive outcomes from naive data augmentation without semantic consistency, highlighting the necessity of structured inference-time verification.

Theoretical and Practical Implications

VERITAS provides empirical support for the paradigm that inference-time verification serves as a substitute for expensive human feedback in robotic policy improvement. By externalizing semantic and geometric reasoning to a verifier, it enables generalist RL/Imitation Learning policies to enhance deployment performance and continually improve with minimal supervision.

The fundamental limitation of the framework is that policy improvement is bounded by the diversity of the underlying policy’s exploration—i.e., the performance ceiling is set by candidate coverage. While inference-time compute cost grows with $N$, results show diminishing returns beyond $N=8$, offering a practical trade-off for real-world application.

In dynamic or non-stationary environments, reliance on static visual traces may limit verifier fidelity. Extensions could target adaptive trace generation, dynamic scene understanding, or distillation of verifier logic into compact learned value functions for real-time application.

Conclusion

VERITAS establishes that visual verification-based inference-time steering is an effective and scalable mechanism for robotic policy enhancement, both immediately at deployment and over continued operation. The proposed generator-verifier paradigm not only advances success rates without altering policy weights but also autonomously generates data that matches the sample efficiency of expensive expert supervision. This unlocks practical self-improving robotic systems, with substantial implications for real-world robotic autonomy and continual learning.

Figure 9: Additional qualitative results underscoring inference-time verification as a mechanism for online policy steering and task completion.