- The paper introduces Vesta, a unified embodied reasoning and planning model that integrates spatial localization, navigation, and action planning through extensive supervised fine-tuning.
- The paper demonstrates that Vesta outperforms specialist models by over 10% on average across multiple benchmarks, validating its efficiency on both simulated and real-world tasks.
- The paper validates a planner-actor architecture using a hybrid memory harness with chain-of-thought reasoning, significantly enhancing task success in complex physical environments.
Vesta: A Generalist Model for Embodied Reasoning and Planning
Unified Model Architecture and Training
Vesta represents a substantial shift from specialist Vision-LLMs (VLMs) and Vision-Language-Action (VLA) models toward generalist embodied agents capable of seamless integration across localization, spatial reasoning, navigation, and action planning. The model is architecturally grounded in Qwen3-VL-8B, undergoing comprehensive supervised fine-tuning (SFT) with a meticulously curated data mixture targeting spatial intelligence, navigation, embodied reasoning, grounding, general VLM tasks, and real robot interaction. This SFT corpus is heavily biased toward spatially grounded challenges, reflecting the requirements for high-fidelity reasoning and real-world task execution.
The memory harness, a hybrid image-text module, provides explicit context retention for long-horizon reasoning. At each step, sampled image frames and a running textual log of subtasks are injected into the prompt, enabling non-Markovian planning with efficient context compression. Ablation studies demonstrate that this multimodal harness outperforms image- or text-only variants, with uniform and recency-biased sampling strategies yielding comparable results. Vesta generates Chain-of-Thought traces for each subtask, structuring reasoning into observation, progress, reasoning, and action phases, further reinforcing temporal coherence in decision sequencing.
Vesta delivers superior performance across a spectrum of embodied cognition and spatial localization benchmarks. On widely adopted datasets (Open-X VQA, SAT, MindCube-Tiny, CV-Bench, CrossPoint, PointBench, etc.), Vesta consistently achieves the highest or second-highest scores, setting a new standard for unified reasoning capabilities at the 8B parameter scale. The model demonstrates balanced proficiency, maintaining competitiveness even on tasks historically dominated by cognition or localization specialists.
Notably, Vesta outperforms an ensemble of per-category-best baselines by an average of >10%, and outperforms individual SOTA baselines by >20%. This positive transfer effect, empirically validated via ablation, refutes the prevailing belief that task-specific training is necessary to maximize benchmark scores. The unified model generalizes across reasoning paradigms without incurring domain-specific trade-offs.
Navigation and Action Planning
Vesta's navigation module adopts a Vision-and-Language Navigation (VLN) formulation, supporting instruction-guided waypoint prediction, turn primitive generation, and STOP actions. On R2R-CE benchmarks, Vesta achieves competitive scores relative to navigation specialists, tying with SOTA InternVLA-N1 and significantly outperforming generalist models. Success Rate and Oracle Success metrics indicate robust spatial awareness and pathfinding, while SPL and Navigation Error remain within tight bounds, underscoring architectural efficiency in continuous environments.
For action planning, Vesta introduces an offline, MCQ-based evaluation benchmark comprising both standardized robotic manipulation tasks (AgiBot) and diverse, open-world episodes (Egocentric Human-Hand dataset). Vesta improves task success rates by 38.3% over actor-only baselines and by 25% over strong planner alternatives (Qwen3-VL). Its zero-shot generalization across unseen episodes, dense temporal annotation, and open-ended subtask vocabulary highlight scalable reasoning and adaptability.
Real-World Robotic Deployment
Vesta's real robot evaluation employs the bimanual YAM platform, orchestrating intricate, memory-dependent tasks: object search, fruit counting, and candy memorization. The hierarchical planner-actor interface is realized through synchronous and asynchronous coupling paradigms, with planner memory retained solely on the planner side, and actions communicated as natural-language subtasks. Empirically, Vesta increases average task success by 38.3% (statistical significance > 40) versus an actor-only setup, confirming effective reasoning augmentation in complex physical environments. The improvement is robust across all tested tasks, and is maintained with a simple, reproducible inference loop.
Generalist vs. Specialist Paradigm
Ablation experiments rigorously compare unified and specialist training regimes, fixing architecture, base VLM, and compute budget. The unified Vesta model matches or outperforms navigation- and embodied-only specialists on their respective benchmarks, demonstrating that positive cross-task transfer can be achieved without sacrificing in-domain performance. Additionally, oversampling transition-phase data during training markedly improves planning fidelity during subtask switches, suggesting efficient strategies for resolving inherent class imbalance.
Limitations and Future Directions
Vesta's evaluation scope is restricted to bimanual manipulation tasks on a single robotic platform, with broader generalization to mobile, multi-modal, and dexterous embodiments remaining unaddressed. Scaling effects at larger model sizes and the integration of adaptive, learned memory modules are unexplored. Future work should investigate lifelong memory, consolidation, and retrieval strategies, as well as adaptation to platforms with varied subtask abstraction and controllability profiles.
Practical and Theoretical Implications
Vesta's design and empirical performance have substantial implications for the deployment and scaling of embodied AI systems:
- Simplification of Robotic Architectures: Replacing ensembles of specialists with a single generalist planner reduces system latency, inference complexity, and cascading error risks, enhancing robustness in both academic and production settings.
- Rapid Benchmarking and Prototyping: The offline planning benchmark allows cost-effective evaluation, shrinking model development cycles for research groups with limited physical platform access.
- Safety and Auditing: The planner-actor split provides a layer for safety filtering and auditing, as natural-language subtasks can be reviewed prior to physical execution, mitigating risks associated with embodied AI.
At a theoretical level, Vesta validates the feasibility of unified generalist training, positive transfer across embodied reasoning axes, and scalable memory harnesses. Continued exploration of multimodal memory architectures, lifelong learning, and scaling laws for spatially grounded data will be essential for advancing the field.
Conclusion
Vesta establishes a new precedent for generalist embodied planning, unifying localization, navigation, reasoning, and action sequencing within a single model. Its empirical superiority across multiple benchmarks and real-robot tasks demonstrates that generalist architectures are viable and scalable alternatives to collections of task-specific specialists. This paradigm shift enables practical and theoretical advancements in embodied AI, paving the way for more robust, adaptive, and efficient deployment in increasingly complex physical environments (2606.20905).
