---
title: 'WLA-0: Prototype for World-Language-Action Models'
url: https://www.emergentmind.com/topics/wla-0
type: topic
---

# WLA-0: Prototype for World-Language-Action Models

WLA-0 is a prototype **world-language-action (WLA)** model, introduced as an embodied foundation model that jointly performs **world modeling**, **language reasoning**, and **action synthesis** from multimodal robot inputs. It takes **textual instructions, images, and robot states** as inputs, and predicts **textual subtasks, subgoal images, and robot actions**, combining the *world modeling interface* associated with world-action models and the *language reasoning* capacities associated with vision-language-action models. Its core design choice is an **autoregressive Transformer backbone** that predicts the **next state** at both the **semantic level** and the level of **fine-grained physical dynamics**. The reported WLA-0 prototype has **2B active parameters**, achieves **40 ms per inference on an NVIDIA RTX 5090**, and is evaluated on both simulated and real-world robotic benchmarks, including **92.94\% success rate on RoboTwin2.0 Clean** and **56.5\% success rate on RMBench** [2606.05979].

## 1. Concept and scope

WLA-0 was proposed as the first instantiation of a new class of **World-Language-Action** models intended to unify three functions that had previously been emphasized separately: predicting physical dynamics, reasoning over instructions and subgoals, and synthesizing executable robot actions [2606.05979]. The model is described as conjoining the **world modeling interface** used to learn from extensive egocentric videos with the **language reasoning** capacities needed for complex long-horizon tasks.

A central feature is that WLA-0 predicts the *next state* in two complementary forms. One is a **semantic-level textual intention**, expressed as **textual subtasks**. The other is a **fine-grained physical dynamics** representation that supports future-state prediction and action generation. This dual representation distinguishes the model from systems that only emit low-level controls or only perform high-level task decomposition.

The paper situates WLA-0 relative to **WAMs** and **VLAs** rather than treating it as a minor variant of either family. Its backbone is explicitly **autoregressive**, in contrast to the **bidirectional diffusion Transformer** used in WAMs. This design is intended to support both text generation and multimodal causal prediction in a single model [2606.05979].

A common misconception would be to view WLA-0 as merely a vision-language-action policy with an auxiliary prediction head. The reported formulation is stronger: the model is trained so that world modeling supervises a dedicated representation of physical dynamics, and that representation is then used to ease the characterization of the **state-action correlation** for the **Action Expert** [2606.05979].

## 2. Architectural organization

The architecture comprises three principal modules: an **Autoregressive Transformer Backbone**, a **World Expert**, and an **Action Expert** [2606.05979].

The **Autoregressive Transformer Backbone** is the core multimodal model. It is initialized from a large pretrained vision-language model such as **RynnBrain-2B**, and processes **images, text, robot states, and memory**. According to the reported design, it sequentially predicts **Textual Intention** and **Physical Dynamics**. The backbone therefore serves both as a **language reasoning engine** and as a **dynamics rollouter**.

The **World Expert** is a lightweight diffusion Transformer, exemplified by **SANA-600M**, that receives the physical dynamics representation and the current state to predict a latent feature representation of the future observation. The model uses **VAE** features for this prediction. The paper states that the world modeling objective is auxiliary at train time and optional at inference time [2606.05979].

The **Action Expert** consumes the same latent dynamics and the current proprioceptive state to synthesize an **action chunk**. It is implemented as a **flow-matching head**. The reported decomposition implies that WLA-0 does not generate actions directly from raw observations alone; instead, it interposes a latent representation of physical dynamics shared with world modeling.

The model’s inputs and outputs are explicitly multimodal:

| Component | Reported modalities |
|---|---|
| Inputs | Textual instructions, images, robot states, memory |
| Intermediate predictions | Textual intention, physical dynamics, meta-queries |
| Outputs | Textual subtasks, future images, action chunk |

The architecture also introduces **meta-queries**, described as special tokens appended to the Transformer context. These tokens extract and aggregate relevant context through attention and generate the **physical dynamics vector** used by both the World Expert and the Action Expert [2606.05979].

## 3. Representation, objectives, and inference modes

The model predicts a window of subtasks according to
\[
\mathcal{S}_t = f(\mathbf{o}_{t-h}, \mathbf{o}_t, \ell, \mathcal{M})
\]
where \(\ell\) is the original instruction and \(\mathcal{M}\) is the memory buffer [2606.05979]. The latent dynamics are then produced via meta-queries:
\[
\mathbf{h}_t = f(\mathbf{o}_{t-h}, \mathbf{o}_t, \ell, \mathcal{M}, \mathcal{S}_t, \mathbf{Q})
\]
and this latent code is used by both experts.

The paper defines the action and world-model outputs as
\[
\mathbf{a}_{t:t+n} = f_{\mathrm{act}}(\mathbf{h}_t, \mathbf{q}_t)
\]
and
\[
\mathbf{o}_{t+n} = f_{\mathrm{wm}}(\mathbf{h}_t, \mathbf{o}_t).
\]
This organization makes the latent physical-dynamics representation a shared interface between semantic reasoning and control [2606.05979].

Training uses a three-term objective:
\[
\mathcal{L} = \mathcal{L}_\mathrm{act} + \alpha\,\mathcal{L}_\mathrm{wm} + \beta\,\mathcal{L}_\mathrm{lang}
\]
with \(\mathcal{L}_\mathrm{act}\) the flow-matching loss for the Action Expert, \(\mathcal{L}_\mathrm{wm}\) the flow-matching loss for world modeling, and \(\mathcal{L}_\mathrm{lang}\) the cross-entropy loss for subtask generation. The reported example values are \(\alpha=0.1\) and \(\beta=0.005\) [2606.05979].

Two inference regimes are described. In **Efficient Mode**, the **World Expert is disabled**, leaving only the AR Backbone and Action Expert active; this is the mode associated with the reported **~40 ms per call on RTX 5090**. In **Test-Time Scaling (TTS) Mode**, the model samples \(K\) candidate action chunks, uses the World Expert to imagine future images for each candidate, and scores them with a learned value model:
\[
v_t = y \cdot \gamma^{T-t}.
\]
The best action chunk is then selected for execution. The paper further states that imagined frames can be fed back to extend the planning horizon [2606.05979].

A common misconception is that WLA-0 must always run explicit world prediction online. The reported design states the opposite: **world prediction can be disabled during inference**, while still allowing it to be activated for **test-time scaling** when improved control is desired [2606.05979].

## 4. Benchmark performance

The reported empirical results cover **RoboTwin2.0**, **LIBERO**, **RMBench**, and real-world robotic tasks [2606.05979].

On **RoboTwin2.0 Clean**, WLA-0 achieves **92.94\%** average success. On **RoboTwin Rand.**, the summary table reports **90.02\%**. On **LIBERO**, WLA-0 attains **98.6\%** average success, and with **TTS** this rises to **98.9\%**. The paper describes these as state-of-the-art results for the reported settings [2606.05979].

On **RMBench**, which emphasizes long-horizon behavior and memory, WLA-0 achieves **56.5\%** average success, compared with **28.5\%** for the best prior **Mem-0** baseline. The paper also reports that removing the **language subtask loss** reduces performance to **17.3\%**, which is presented as evidence for the importance of language-decomposed subtasks in long-horizon execution [2606.05979].

The provided comparison table includes the following entries:

| Method | RoboTwin Clean | LIBERO Avg |
|---|---:|---:|
| WLA-0 | 92.94 | 98.6 |
| Motus | 88.66 | 97.7 |
| Fast-WAM | 91.88 | 97.6 |

The real-world experiments emphasize dynamic and out-of-distribution tasks. On **Dispose Trash**, which is described as latency-sensitive and dynamic, WLA-0 is reported to outperform **Motus** and \(\pi_{0.5}\), with the explanation that those systems lag due to higher latency or lack of history dependence. The paper also states that WLA-0 reduces inference latency by **~40× vs. Motus** [2606.05979].

These results support a specific interpretation of WLA-0’s contribution: the model is not only optimized for broad benchmark coverage, but also for the interaction between **reasoning depth**, **history dependence**, and **control latency**.

## 5. Long-horizon reasoning and memory

WLA-0 is explicitly designed for **complex long-horizon tasks**, and the reported mechanism for this capability is the joint use of **textual subtasks**, **memory**, and **physical dynamics** [2606.05979].

The backbone receives a **memory buffer** and outputs a **window of intermediate subtasks** aligned with the planning horizon. This means the model does not treat the instruction as a single immutable prompt; instead, it incrementally reformulates the task into local semantic intentions. The paper associates this with WLA-0’s strong performance on **RMBench**, where language-level memory and progress tracking are credited with the nearly twofold improvement over the best prior baseline [2606.05979].

The ablation in which the **language subtask loss** is removed is particularly important because it isolates the role of semantic decomposition. The drop from **56.5\%** to **17.3\%** is reported as highlighting the critical role of language-decomposed subtasks. This makes WLA-0 notable not simply for incorporating language tokens, but for making subtask prediction a supervised and quantitatively consequential component of the architecture.

A plausible implication is that WLA-0 treats language not merely as a conditioning signal but as an internal planning substrate. That interpretation is consistent with the model’s explicit prediction of **Textual Intention** before downstream action synthesis, though the paper’s concrete claim is limited to the reported module structure and ablation outcomes [2606.05979].

## 6. Learning from robot videos and cross-embodiment transfer

A prominent claim of WLA-0 is that it holds the promise to learn novel tasks directly from **cross-embodiment robot videos without action annotations** [2606.05979]. The experiments addressing this point combine **action-labeled seen tasks** with **action-free videos of unseen tasks**, drawn from both the **same** and **cross-embodiment** robots.

For unseen tasks, adding **same-embodiment action-free videos** increases the baseline unseen-task success rate **from ~13\% to 34.4\%**. Using **cross-embodiment action-free videos** yields **28.8\%**, which the paper characterizes as retaining most of the performance [2606.05979]. These numbers are presented as evidence that WLA-0 can extract and transfer actionable knowledge from diverse robot videos even without explicit action supervision.

This claim should be interpreted carefully. The reported setup does not state that WLA-0 is trained without any action labels at all; rather, it is trained on **action-labeled seen tasks** and additionally on **action-free videos of unseen tasks**. The stronger statement is prospective: WLA-0 “holds the promise” of learning novel tasks directly from cross-embodiment robot videos without action annotations [2606.05979].

The broader significance of this result lies in the model’s intermediate representation. Because the AR backbone predicts **semantic-level** subtasks and **fine-grained physical dynamics**, and because the World Expert supervises future-state prediction while the Action Expert maps the same latent dynamics to control, the architecture is designed to separate task semantics from embodiment-specific motor trajectories. This suggests why cross-embodiment transfer is plausible, although the exact mechanism is described in the paper through the reported modules and results rather than through a formal invariance theorem.

## 7. Position within embodied model design

WLA-0 is positioned as a unification of two previously distinct emphases in embodied learning: the **world modeling interface** associated with WAMs and the **language reasoning** capacities associated with VLAs [2606.05979]. Its defining architectural commitment is the use of an **autoregressive Transformer backbone** in place of the **bidirectional diffusion Transformer** used in WAMs.

This substitution matters because the AR backbone is described as capable of both **sequential text** modeling and **latent distribution** modeling, allowing a causal chain of the form “predict subtask \(\rightarrow\) predict physical dynamics \(\rightarrow\) predict action.” The World Expert remains diffusion-based, but it is moved into an expert head rather than used as the central sequence model [2606.05979].

The model also departs from approaches that bind world prediction to online control at every step. Through **meta-queries**, world prediction can **implicitly** influence action generation, even when the World Expert is disabled during inference. This yields a hybrid design: world modeling shapes the learned control interface during training, while low-latency action execution remains available at test time [2606.05979].

Several boundaries of the current evidence should be stated neutrally. The paper reports a **WLA-0 prototype** with **2B active parameters**, not a universal or final WLA system. The strongest claims are supported on the listed benchmarks and real-world tasks. The statement regarding direct learning from **cross-embodiment robot videos without action annotations** is framed as a promise rather than as a fully closed experimental conclusion [2606.05979]. Within those bounds, WLA-0 is presented as a unified embodied model in which **semantic reasoning**, **future-state modeling**, and **low-level action synthesis** are trained as a single coordinated system.

Source: https://www.emergentmind.com/topics/wla-0