Embodied-R1.5: Unified Multimodal EFM

Updated 4 July 2026

The paper presents Embodied-R1.5 as a unified 8B-parameter embodied foundation model that integrates cognition, planning, correction, and spatial reasoning in one multimodal transformer.
It achieves state-of-the-art performance on 16 of 24 benchmarks by using a Planner–Grounder–Corrector framework that enables closed-loop, long-horizon robotic task execution.
The model is trained on 34 datasets with over 15B tokens and refined with reinforcement learning, demonstrating robust generalization across diverse embodied AI tasks.

Embodied-R1.5 is an 8B-parameter embodied foundation model (EFM) that aims to internalize the full “see–think–act” loop for robotics within a single multimodal transformer, and then use that model as the backbone for efficient VLA policies and closed-loop robot autonomy. It integrates embodied cognition and spatial reasoning, task planning and correction, and embodied pointing and location; is trained on a 34-dataset corpus of over 15B tokens; and is coupled to a Planner–Grounder–Corrector (PGC) framework for long-horizon, self-correcting execution (Yuan et al., 9 Jun 2026).

1. Conceptual scope and lineage

Embodied-R1.5 is presented as a response to a recurrent fragmentation in embodied AI: some systems specialize in cognition and spatial QA, some in planning, some in grounding, and many rely on multiple models and scales within a pipeline. Its stated objective is general physical intelligence, operationalized as a single model that can perceive and understand 3D physical scenes, reason about tasks and progress over long horizons, ground decisions into actionable spatial commands, monitor and correct mistakes in a closed loop, generalize across tasks and robots, and serve as a strong backbone for downstream VLA policies (Yuan et al., 9 Jun 2026).

The model explicitly builds on Embodied-R1, which treated “pointing” as an embodiment-agnostic intermediate representation and centered four abilities—referring expression grounding, region referring grounding, object functional grounding, and visual trace generation—inside a 3B VLM (Yuan et al., 19 Aug 2025). Embodied-R1.5 extends that scope from a pointing specialist to a general EFM by unifying cognition, planning, correction, and pointing inside one architecture (Yuan et al., 9 Jun 2026).

Its framing is also legible within broader embodiment theory. Classical embodied cognition emphasizes that behavior is not merely the outcome of an internal controller, but is shaped by morphology, material properties, the ecological niche, and closed-loop sensorimotor interaction (Hoffmann et al., 2012). A plausible implication is that Embodied-R1.5 represents an attempt to shift from narrowly scoped embodied VLM capabilities toward a more integrated control-and-reasoning substrate, even though its main computational core remains a large multimodal model rather than a morphology-first design.

The model also sits naturally inside recent taxonomic discussions of embodied AGI, which place current systems largely between L1 and L2 and reserve L3+ for broader multimodal, conditionally general-purpose, and more autonomous robotic intelligence (Wang et al., 20 May 2025). This suggests that Embodied-R1.5 is best understood as a unification-oriented EFM rather than as a complete solution to embodied AGI.

2. Unified architecture and capability dimensions

The core backbone is Qwen3-VL-8B-Instruct. Inputs include images and videos, together with textual instructions, questions, and task context. Outputs are always token sequences: free-form reasoning, <answer> blocks for evaluation, and coordinates or trajectories expressed as plain-text numeric tokens. Points, boxes, and traces are normalized to $[0,1000]$ , and the design deliberately avoids special coordinate tokens; the paper argues that this yields more stable predictions and preserves “visual anchoring” through long reasoning chains (Yuan et al., 9 Jun 2026).

Architecturally, Embodied-R1.5 does not instantiate separate modules for separate capabilities. Instead, it uses prompting and output formatting over a shared VLM to realize three major dimensions:

Embodied cognition and spatial reasoning: scene understanding, 3D structure, metric depth, spatial relations, appearance order, and robot-centric scene interpretation.
Task planning and correction: long-horizon decomposition, next-step planning, process detection, error localization, and corrective replanning.
Embodied pointing and location: referring expression grounding, region grounding, functional-part grounding, and 2D/3D end-effector or object traces.

The model’s VLA derivative, Embodied-R1.5-VLA, adds a DiT-based flow-matching action expert to intermediate VLM features. The action head uses 2048-dimensional features, 32 learnable future query tokens, and predicts 7D continuous action sequences via flow matching. Inference uses 4-step Euler integration over the flow field (Yuan et al., 9 Jun 2026). The intended separation is that embodied reasoning is first internalized in the EFM, after which a comparatively lightweight action head maps that internalized structure to continuous control.

3. Data system and multi-task training

Embodied-R1.5 is trained on 34 datasets totaling over 15B tokens. Its data system is organized around three automated pipelines, each targeting a different capability bottleneck (Yuan et al., 9 Jun 2026).

Pipeline	Function	Resulting data
ER1.5-Spatial	3D scene annotation for tabletop spatial reasoning	~20K QA pairs
ER1.5-Correction	Failure-aware planning and correction	~800K+ QA pairs
ER1.5-Pointing	Affordance and trajectory data for pointing	~400K samples

The ER1.5-Spatial pipeline starts from RGB images in datasets such as Fractal, BridgeData V2, and DROID; uses Qwen3-VL and RAM++ for semantics, MoGe-2 for metric depth and normals, Grounded-SAM for open-vocabulary instance segmentation, inverse projection for 3D lifting, and plane alignment to construct 3D semantic scene graphs from which QA pairs are programmatically generated. The ER1.5-Correction pipeline injects planning failures and execution failures into correct plans and successful trajectories, creating balanced detection, localization, and correction data. The ER1.5-Pointing pipeline aggregates REG, RRG, OFG, and VTG data from both real and synthetic sources, including functional-part annotations and projected 2D/3D traces (Yuan et al., 9 Jun 2026).

Training proceeds in two stages. Stage 1 is standard causal SFT over the heterogeneous corpus. Stage 2 is reinforcement fine-tuning with a GRPO-like recipe on a filtered subset of roughly 200K medium-difficulty samples. The RL stage is designed around three stabilizers: difficulty-aware filtering, dynamic group filtering, and global batch reward normalization. The normalization is

$\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$

where the group-level mean preserves within-prompt ranking and the batch-level standard deviation normalizes across heterogeneous tasks (Yuan et al., 9 Jun 2026).

Reward functions are task-specific and include exact match, IoU, point-distance reward, trajectory-RMSE reward, semantic-similarity reward, and a format reward. For continuous quantities, the paper uses a piecewise-linear decay:

$\phi(d;\,\tau_p,\tau_z) = \mathrm{clip}\!\left(\frac{\tau_z - d}{\tau_z - \tau_p},\;0,\;1\right).$

The final reward mixes accuracy and output validity as

$R = (1 - \lambda)\,R_{\mathrm{acc}} + \lambda\,R_{\mathrm{fmt}}, \qquad \lambda = 0.1.$

The RL stage uses 8 rollouts per prompt, AdamW, a learning rate of $3 \times 10^{-6}$ , and keeps the vision encoder unfrozen (Yuan et al., 9 Jun 2026).

4. Planner–Grounder–Corrector closed-loop execution

The PGC framework is the execution harness through which Embodied-R1.5 acts as a closed-loop autonomous system. It invokes the same model in three prompt-defined roles rather than three separate models (Yuan et al., 9 Jun 2026).

The Planner receives the high-level task instruction, current observations, and optional documents such as SOPs, and generates either an initial long-horizon decomposition or an updated next-step plan. The Grounder converts each subtask into actionable spatial outputs by invoking the model’s grounding capabilities: object points, functional parts, free-space regions, or 2D/3D traces. The Corrector monitors execution and returns one of three statuses—SUCCESS, PROCESS, or FAIL—together with error type and corrective strategy when failure is detected (Yuan et al., 9 Jun 2026).

PGC uses a minimal FIFO memory containing sampled frames over the whole task and annotations of subtask status. Execution loops as follows: a plan is generated, the current subtask is grounded into spatial commands, low-level execution occurs, the Corrector is polled, and the system either proceeds, waits, or replans. The paper emphasizes that PGC is stateless apart from this memory and contains no heuristic rules; all planning, grounding, and correction decisions are delegated to the EFM (Yuan et al., 9 Jun 2026).

This framework is demonstrated on long-horizon tasks such as making milk tea at roughly 10 steps, stacking cups into three layers in 6 steps, sweeping garbage in a cyclic detect–localize–sweep pattern, and goods picking from shelves under open-vocabulary instructions. A notable perturbation experiment repeatedly moves the target plate during a “move corn to plate” task; the Corrector detects the mismatch, triggers re-grounding, and the system eventually completes the task (Yuan et al., 9 Jun 2026).

5. Benchmark performance as an embodied VLM

Embodied-R1.5 is evaluated on 24 embodied VLM benchmarks grouped into planning and correction, pointing and location, and cognition and spatial reasoning. The paper reports SOTA on 16 of the 24 embodied benchmarks, with an average of 70.4% across the 21 main accuracy-based benchmarks, surpassing Gemini-Robotics-ER-1.5 by 17.0 percentage points and GPT-5.4 by 21.7 percentage points (Yuan et al., 9 Jun 2026).

On planning and correction, the average over four benchmarks is 65.3 for Embodied-R1.5, versus 53.8 for GPT-5.4, 41.3 for Gemini-Robotics-ER-1.5, and 54.2 for the Qwen3-VL-8B base model. On pointing and location, the average over nine pointing benchmarks is 72.8, versus 59.2 for Qwen3-VL-8B, 58.7 for Embodied-R1, 55.2 for Molmo-D-7B, and 52.4 for Gemini-Robotics-ER-1.5. On cognition and spatial reasoning, the average over eight benchmarks is 70.2, versus 68.8 for Gemini-2.5-Pro, 67.3 for GPT-5.4, and 67.0 for Qwen3-VL-8B (Yuan et al., 9 Jun 2026).

The specialization does not erase general visual competence. On seven general-vision benchmarks, the model remains close to the base Qwen3-VL-8B and even improves on some tasks, such as MMMU and SciQA. The paper’s SFT-vs-RFT ablation also shows that RL refinement is consistently beneficial: planning improves from 62.6 to 65.3, pointing from 69.0 to 72.8, and cognition from 68.9 to 70.2 when moving from Embodied-R1.5-SFT to the RL-refined model (Yuan et al., 9 Jun 2026).

The benchmarking infrastructure is itself part of the contribution. EmbodiedEvalKit standardizes benchmark ingestion, coordinate normalization, answer parsing, and task-specific metrics for 25+ embodied benchmarks and 20+ models, using a unified interface (Yuan et al., 9 Jun 2026).

6. VLA adaptation and real-robot results

Embodied-R1.5 is explicitly designed to function as a VLA backbone with comparatively little action data. In SimplerEnv on Google Robot Visual Matching, Embodied-R1.5-VLA reaches 92.4% overall success, compared with 72.7% for $\pi_{0.5}$ , 71.4% for $\pi_0$ , and 67.7% for GR00T-N1.6. On Variant Aggregation, it reaches 71.5%, versus 68.4% for $\pi_{0.5}$ . On WidowX Visual Matching, it reaches 74.0%, versus 57.1% for $\pi_{0.5}$ and 62.0% for GR00T-N1.5 (Yuan et al., 9 Jun 2026).

On LIBERO, the comparison is especially pointed because the paper separates models with and without large-scale action pretraining. Embodied-R1.5-VLA reaches 97.3% without action pretraining, compared with 93.6% for $\pi_{0.5}$ and 91.9% for OpenVLA-OFT in the same regime. On LIBERO-Plus robustness, it reaches 76.0% overall, compared with 69.6% for OpenVLA-OFT, 61.6% for $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 0-FAST, and 53.6% for $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 1. On ManiSkill-Affordance, average success is 0.76 on seen categories and 0.66 on unseen categories, exceeding ManipLLM’s 0.59 and 0.54, respectively (Yuan et al., 9 Jun 2026).

The RoboTwin2.0 comparison is structurally different: Embodied-R1.5 is zero-shot, whereas RDT, $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 2, and $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 3 are fine-tuned with 400 demonstrations per task. Even so, Embodied-R1.5 reaches 65.0% average success, compared with 72.9% for fine-tuned $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 4, and achieves 99% on “Click Bell,” versus 66% for $\hat{A}_i = \frac{R_i - \mu_{\text{group}}}{\sigma_{\text{batch}} + \epsilon},$ 5 (Yuan et al., 9 Jun 2026). This suggests that strong grounding and trace generation can substitute, at least partially, for task-specific low-level imitation.

Real-robot experiments on Xarm6, ARX Lift2s, and Realman RM75 further test zero-shot transfer. The reported success rates are 100.0% on general pick-and-place, 100.0% on tool affordance, 83.3% on an ordinal spatial reasoning task (“Put the [X]-th duck toy from the left on the plate”), 66.6% on cup disassembly, and 16.6% on door opening (Yuan et al., 9 Jun 2026). The door result is low, but it is also the experiment that most directly demands curved 3D trajectory reasoning around an articulated hinge.

7. Limitations and significance within embodied AI

The paper identifies several limitations. First, perception remains largely 2D-centric: depth is inferred rather than natively sensed, and future work is explicitly directed toward point clouds and depth maps. Second, the VLA coupling is intentionally lightweight, which makes the action model efficient but leaves room for tighter integration between reasoning tokens and control. Third, PGC has been validated mainly in tabletop manipulation and limited bimanual settings, not yet in broad mobile manipulation or navigation. Fourth, some affordance and trajectory supervision depends on simulated and synthetic content, leaving open the question of broader real–sim gaps. Fifth, there is no formal safety guarantee, and training an 8B EFM with RL over 15B tokens remains computationally demanding (Yuan et al., 9 Jun 2026).

Within the larger field, Embodied-R1.5 occupies a distinctive position. Surveys of large-model embodied AI emphasize hybrid stacks that combine hierarchical planning, end-to-end VLA control, and world models (Liang et al., 14 Aug 2025). Related self-improvement systems approach the problem from different angles: SEEA-R1 uses Tree-GRPO with Monte Carlo Tree Search and a Multi-modal Generative Reward Model for self-evolving embodied agents (Tian et al., 26 Jun 2025), whereas EmbodiSkill externalizes adaptation into a revisable skill document and skill-aware reflection rather than into model parameters (Ju et al., 11 May 2026). Embodied-R1.5 differs in trying to internalize cognition, planning, correction, and grounding inside one EFM and then export that internalization into a compact VLA head (Yuan et al., 9 Jun 2026).

A plausible implication is that Embodied-R1.5 marks a shift from evaluating embodied VLMs as isolated QA or grounding systems toward evaluating them as reusable infrastructural models for downstream control, closed-loop correction, and benchmark standardization. Its open release of model weights, datasets, training code, and EmbodiedEvalKit reinforces that role (Yuan et al., 9 Jun 2026).