ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

Abstract: Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address this loop only in parts: end-to-end policies generate actions but often lack spatial reasoning, planning, and execution assessment, while robot-agent systems orchestrate tools or specialists but do not learn a shared representation. This fragmentation limits general Physical Agentic AI. We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. Built on ACE-Brain-0, which established spatial intelligence as a shared scaffold across robot platforms, ACE-Brain-0.5 extends an understanding-centric model into a closed-loop foundation model. A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery. To unify these capabilities without cross-task interference, we introduce SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector merging. The fifth function, self-improvement, is realized by a companion framework that updates external execution state, including task schemas, spatial memory, and failure-recovery cases, from rollouts. Across fifteen benchmarks, ACE-Brain-0.5 improves over ACE-Brain-0 on 14 of 18 spatial perception and grounding benchmarks, achieves competitive navigation and manipulation performance, and provides strong progress estimation in ID and OOD settings. Together, these results mark an early step toward general Physical Agentic AI.

• The paper presents a unified embodied foundational model that integrates spatial perception, decision making, action, and self-monitoring into a closed-loop cognitive system.
• It utilizes a hybrid transformer architecture with an omni-modal vision encoder and a four-stage SSR+ training process to minimize cross-task interference.
• Empirical results show significant improvements across spatial, planning, navigation, manipulation, and self-monitoring benchmarks, advancing deployment-ready adaptability.

ACE-Brain-0.5: Toward Unified Embodied Foundation Models for Agentic Physical Intelligence

Motivation for Unified Physical Agentic AI

The field of embodied AI is advancing beyond modular Sense–Plan–Act robotics and specialized end-to-end robot policies towards architectures that integrate perception, reasoning, planning, action, and self-monitoring in a closed cognitive loop. Current approaches—vision-language-action (VLA) models, robot-agent system orchestrations, and spatial reasoning models—excel at specific capabilities but lack unified shared representations and bidirectional integration between high-level cognition and executable actions. Fragmentation in embodiment capability coverage precludes the emergence of general-purpose, continually self-improving agentic robots.

ACE-Brain-0.5 introduces a new paradigm, building on ACE-Brain-0's spatial-intelligence scaffold, to instantiate five tightly coupled cognitive functions in a single foundation model: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. This design closes the perception–planning–action–evaluation loop and offers a scalable agentic AI system capable of deployment-time adaptation and continual improvement.

Figure 1: Evolution from modular, pipeline-based robot systems to unified foundation models capable of closed-loop cognition, integrating spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement within a single architectural backbone.

Model Architecture and Cognitive Functions

ACE-Brain-0.5 is an all-in-one 8B-parameter mixture-of-transformer model grounded in a spatial-intelligence prior. The architecture comprises:

• Omni-modal Vision Encoder: Processes single/multi-view images and videos; provides visual tokens projected into the LLM decoder's space.
• LLM Decoder: Fuses tokenized language instructions, visual information, and optional proprioceptive state into a shared embodied state representing contextual semantics used throughout downstream decoding.
• Fast Vision Encoder and Action Expert: A lightweight DINOv3 encoder feeds real-time visual features to a policy head designed for low-latency continuous (manipulation) and discrete (navigation) action generation via flow-matching.

From this representation, ACE-Brain-0.5 decodes into four output modalities:

• Spatial Perception: Autoregressive prediction of bounding boxes, segmentation masks, and spatial coordinates.
• Decision Making: Instruction decomposition into executable natural language subgoals and sequential task plans.
• Embodied Interaction: Goal- and observation-conditioned actions for both navigation (discrete) and manipulation (continuous joint/end-effector).
• Self-Monitoring: Framewise progress estimation and trajectory pairwise preferences, enabling robust execution evaluation and recovery.

Figure 2: Overview of the ACE-Brain-0.5 model architecture highlighting multi-stream vision/language fusion, fast vision and action expert pathways for closed-loop control, and joint multimodal decoders for spatial perception, task planning, action, and self-monitoring.

SSR+ Training: Scaffold, Specialize, Reconcile, Reactivate

Unifying diverse embodiment supervisions leads to severe cross-task interference if naively mixed. ACE-Brain-0.5 leverages a four-stage SSR+ training regime:

1. Scaffold: Initialize from ACE-Brain-0 (spatial cognition expert).
2. Specialize: Independently train on QA, grounding, navigation, and progress estimation to avoid interface interference.
3. Reconcile: Use vector-based task merging in weight space to embed all specialized capabilities into a single parameterization, minimizing expert divergence.
4. Reactivate: Lightweight SFT on a mixed-format dataset to synchronize output interface conventions, exploiting fast recovery enabled by structurally aligned intermediate representations.

This approach consistently outperforms single-stage or purely isolated multi-task training, enabling robust multimodal interface switching within one model.

Self-Improvement Framework

ACE-Brain-0.5 operationalizes self-improvement via an external execution state $\mathcal{H}$ (task schemas, spatial memory, recovery cases) incrementally updated from robot rollouts. By converting failure/trajectory corrections into new supervision and episodically updating execution memory, the system supports deployment-time adaptation—critical in dynamic physical environments.

Empirical Evaluation Across Cognitive Functions

Benchmarks target the complete pipeline: spatial perception, decision making, embodied navigation/manipulation, and execution monitoring. Notable results:

• Spatial Perception: Strong improvements over ACE-Brain-0 in 14/18 spatial Q/A-grounding benchmarks, e.g., MindCube (86.3% vs. prev. 82.1%), ScanRefer (70.2%), and Multi3DRef (72.4%).
• Planning-Oriented Reasoning: Outperforms prior open models and closed-source MLLMs (Gemini-2.5-Pro, GPT-5.4) on complex spatial and referential reasoning—e.g., RoboSpatial (60.1%), RefSpatial (55.6%), PointArena (68.5%), and RoboAfford (75.1%).
• Decision Making (Driving): Maintains competitive performance on autonomous driving scene and action benchmarks, including MME-RealWorld (66.6%) and MAPLM (71.3%).
• Navigation/Manipulation: Achieves high navigation SR and OS in RxR/R2R with SOTA alignment, for both general purpose and navigation-specialized variants. On LIBERO manipulation, achieves 98.2% avg. success (superior to prior VLA models) and SOTA performance on SimplerEnv-Bridge (82.3% avg. SR).
• Self-Monitoring: Surpasses reward modeling baselines on RBM-EVAL tasks (standard and refined OOD splits), showing VOC $r$ up to 0.96, and demonstrates robustness to temporal inversion controls.
• Self-Improvement: Closed-loop action corrections directly reduce navigation error (NE 5.7 $\to$ 4.8), increase SR/SPL, and robustly address off-policy distribution errors.

Figure 3: ACE-Brain-0.5 progress estimation visualizations; model-predicted progress closely tracks ground truth for both successful and failure trajectories, including action reversals.

Figure 4: Navigation self-improvement mechanism leads to error correction and completion of instruction under previously ambiguous intersections.

Figure 5: Dataset composition shows ACE-Brain-0.5 was trained on a long-tail distribution combining large-scale, broad spatial reasoning data with targeted manipulation and progress-annotation samples.

Figure 6: Integration of detection-based object localization with transition to closed-loop low-level manipulation in the embodied brain framework.

Architectural and Training Innovations

A distinct technical feature is the hybrid attention mask, which preserves autoregressive VLM causal attention for upstream perception and full attention for downstream action tokens, thus ensuring retention of visual reasoning capabilities while enabling efficient policy learning for low-level control.

Figure 7: The specialized attention mask design balances VLM reasoning preservation with efficient action/head-specific attention for policy learning.

Theoretical and Practical Implications

The ACE-Brain-0.5 paradigm provides evidence that spatial intelligence can serve as a robust universal scaffold for integrating disparate embodied tasks within a single parameter sharing model without catastrophic interference. The empirical success of SSR+ confirms that task interface heterogeneity can be reconciled in weight space and interface conventions rapidly recovered, underlining the practical viability of weight-merge-based integration for large multimodal models.

This approach deprecates the need for orchestration-based agent pipelines and paves the way for AI systems with persistent, updatable cognitive states, i.e., enabling lifelong skill refinement and continual improvement from closed-loop experience. The exposure of progress estimation and self-monitoring as primary model outputs further bridges the gap between reward modeling and primary policy execution, offering unified architectures for both analysis and control.

Future Directions

While the self-improvement framework is presently lightweight and concentrated on navigation, a promising avenue is extending online adaptation and self-evolution to the model-level, encompassing manipulation and decision-making in longer-horizon, open-world tasks. Scaling ACE-Brain-0.5 across further embodiment morphologies, broader perceptual modalities, and more challenging multi-agent and real-world deployments remains an open area for foundational research. Techniques presented here are directly applicable to robotic systems requiring robust, closed-loop generalization, modular extension, and rapid policy re-synchronization.

Conclusion

ACE-Brain-0.5 establishes a unified foundation model for embodied agentic AI, with architectural and training strategies that jointly optimize spatial perception, decision making, continuous/discrete control, execution self-assessment, and continual improvement. The model advances the state of the art on a diverse spectrum of physical, spatial, and agentic intelligence benchmarks. The methodology represents a concrete step towards generalizable, self-adaptive robots with persistent and decomposable cognitive faculties. Future progress will hinge on advancing deployment-time adaptation, scaling to broad embodiments, and deeper integration of online self-evolution into foundation model parameter spaces.

---

References: ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI (2607.04426)