Primitive Embodied World Models (PEWM)

• PEWM is a framework that decomposes long-horizon tasks into short, atomic motion primitives to enhance language-action-video alignment and control.
• The architecture couples a modular Vision-Language Model with Start-Goal heatmap Guidance, enabling precise primitive-scale prediction and closed-loop replanning.
• PEWM reduces data acquisition and annotation costs while mitigating video-generation errors, yielding significant improvements in inference latency and real-time prediction.

Searching arXiv for the core PEWM paper and closely related embodied world model work. First, locating the primary PEWM paper. Searching for related survey and evaluation papers on embodied world models. Primitive Embodied World Models (PEWM) are a paradigm for embodied world modeling in which prediction and control are organized around primitive, short-horizon, atomic skills rather than long, monolithic video rollouts. The central claim is that the diversity of embodied data far exceeds the relatively small space of possible primitive motions, so world modeling should focus on fixed short horizons that align more tightly with reusable robotic actions. In this formulation, PEWM restricts video generation to primitive-scale windows, couples a modular Vision-LLM (VLM) planner with Start-Goal heatmap Guidance (SGG), and uses closed-loop replanning to compose primitive policies into extended tasks, with the stated goals of improved alignment granularity, reduced learning complexity, higher data efficiency, and lower inference latency (Sun et al., 28 Aug 2025).

1. Definition, motivation, and formal framing

PEWM was proposed against a background in which video-generation-based embodied world models were gaining attention but remained constrained by the reliance on large-scale embodied interaction data. The stated bottlenecks are the scarcity, difficulty of collection, and high dimensionality of embodied data, together with the brittleness of long-horizon video generation under compounding error, loss of fine-grained correspondence, sparse rewards, and high inference latency. PEWM addresses these issues by restricting generation explicitly to short, fixed horizons corresponding to low-level action primitives such as reach, pick, or move, and by decoupling long-term plan synthesis from short-term video modeling (Sun et al., 28 Aug 2025).

In the formal treatment given for the paradigm, a primitive is defined as a non-constant, finite-duration trajectory $p \in C^0([0,T], X)$ that is indecomposable under concatenation. The primitive set $\mathcal{P}$ is assumed to be finite, and embodied behavior is said to be arbitrarily well approximated by compositions of these primitives. The appendix-level guarantee reported for the method states that any continuous embodied behavior can be approximated arbitrarily well by compositions of finitely many primitives (Sun et al., 28 Aug 2025).

This framing distinguishes PEWM from approaches that attempt to synthesize full long-horizon futures in a single generative process. The short-horizon restriction is not merely an engineering heuristic; it is presented as the key mechanism by which language-action-video alignment becomes finer, local dynamics become easier to model, and compositional control becomes tractable (Sun et al., 28 Aug 2025).

2. Core architecture and control loop

The PEWM architecture is modular. In its main formulation, it consists of a VLM planner, primitive-level world models, and a closed-loop execution scheme. The planner receives high-level task specifications such as language instructions, semantic goals, and images, then decomposes them into a sequence of primitives while grounding them in semantic and spatial knowledge. In the more concrete system description, the planner is implemented with Qwen2.5-VL-7B together with lightweight LoRA adapters, and is described as including a reasoner, planner and grounder, and verifier (Sun et al., 28 Aug 2025).

A second core component is Start-Goal heatmap Guidance. For each primitive, the planner identifies start and goal positions in the image, renders Gaussian-smoothed heatmaps $H_s$ and $H_g$, and forms $H_{s\to g} = H_g - H_s$. These heatmaps condition the primitive video model together with the current frame and text prompt. In the generic goal-conditioned formulation, the primitive world model generates

$x_{t+1:t+N} \sim p(x_{t+1:t+N}\mid x_t, g),$

where $x_t$ is the current observation, $g$ is the spatial goal, and $N$ is the short primitive horizon (Sun et al., 28 Aug 2025).

The primitive-conditioned video generator is described as a video diffusion model, instantiated in one formulation with DynamiCrafter at approximately 1.4B parameters and trained to generate short 32-frame primitive-scale sequences. Its input is the current frame, primitive description, and SGG heatmap, and its output is a short rollout constrained by language and spatial cues. Discrete or binary gripper actions are executed symbolically rather than through the video model. For each primitive, the rollout is summarized as

$$x_{1:T}^{\text{img}} \sim \mathcal{D}(x_0^{\text{img}}, a_k, H_{s\to g}),$$

with $\mathcal{P}$0 denoting the diffusion model and $\mathcal{P}$1 the short horizon (Sun et al., 28 Aug 2025).

Execution is explicitly closed loop. After each primitive is executed, the system re-observes the environment, updates the goal or plan, and selects the next primitive. In the reported implementation, 6-DoF pose is extracted from generated video via Gen6D and then executed in the real world, after which planning proceeds to the next primitive. This contrasts with open-loop long-horizon prediction and is presented as a direct source of robustness to drift and perturbation (Sun et al., 28 Aug 2025).

3. Training protocol, data organization, and efficiency claims

PEWM’s data representation is primitive-centric. An episode $\mathcal{P}$2 is segmented into primitive units,

$\mathcal{P}$3

so that each training example couples a local visual state, a primitive instruction, a spatial start-goal signal, and a short future segment. Temporal boundaries are set on-the-fly by teleoperation signals, and multi-view capture is used to increase spatial density (Sun et al., 28 Aug 2025).

The reported training protocol is three-stage and sim-real hybrid: sim-only for quick embodiment priming, balanced sim-real for domain alignment, and real-heavy for fidelity. Visual Detail Guidance is added in the final stage for improved visual consistency. A generic primitive-level diffusion objective is written as

$\mathcal{P}$4

describing reconstruction or denoising over short goal-conditioned video snippets (Sun et al., 28 Aug 2025).

The efficiency claims attached to this decomposition are unusually explicit. Primitive segmentation is reported to make collection denser, with up to $\mathcal{P}$5 faster data acquisition. VLM-based auto-labeling is reported to reduce annotation cost by up to $\mathcal{P}$6 per primitive. At inference, causal distillation and self-forcing reduce denoising from 50+ steps to 4, yielding 12 FPS real-time prediction, a $\mathcal{P}$7–$\mathcal{P}$8 speedup versus large models, and operation on a single low-memory GPU at approximately 11GB VRAM (Sun et al., 28 Aug 2025).

These claims are tied to four canonical PEWM benefits: improved fine-grained language-action-video alignment, reduced learning complexity, improved data efficiency, and decreased inference latency. The compositional structure is also explicitly linked to zero-shot generalization over new verb-object combinations by recombining known primitives (Sun et al., 28 Aug 2025).

4. Relation to the broader world-model literature

Within the broader embodied AI literature, world models are described as internal representations that enable agents to predict, plan, and act without relying solely on raw sensory input or external simulation. One survey organizes their roles into neural simulators, dynamic models, and reward models, and traces representative architectures from early predictive models through latent state-space models, transformer-based sequence models, and diffusion-based world models (Long et al., 1 Jul 2025).

A second survey formalizes world models in embodied AI through a POMDP-style decomposition with dynamics prior $\mathcal{P}$9, filtered posterior $H_s$0, and reconstruction $H_s$1, and proposes a three-axis taxonomy: functionality, temporal modeling, and spatial representation. In that taxonomy, “primitive” usage can denote foundational, minimal, and computationally efficient world models that are decision-coupled, use sequential simulation and inference, and often compress state into a global latent vector. This broader usage differs from the specific PEWM paradigm of primitive-scale video generation, but it is adjacent in emphasizing efficiency, controllability, and online decision support (Li et al., 19 Oct 2025).

This terminological spread matters. In the specific 2025 PEWM proposal, the primitive is the unit of short-horizon video prediction and composition (Sun et al., 28 Aug 2025). In survey usage, “primitive world models” may instead refer to simple stepwise latent transition models inherited from model-based RL traditions (Li et al., 19 Oct 2025). A plausible implication is that PEWM now names both a concrete architecture family and a wider design preference for minimal, compositional, action-conditioned internal simulation.

5. Variants, extensions, and adjacent formulations

Several later systems extend or reinterpret primitive embodied world modeling by modifying the state representation, action interface, or feedback mechanism. PIVOT-R is a primitive-driven waypoint-aware world model for manipulation in which a Waypoint-aware World Model performs primitive action parsing and primitive-driven waypoint prediction, and a lightweight action prediction module decodes low-level actions. Its Asynchronous Hierarchical Executor runs primitive parsing, scene prediction, and action prediction at different frequencies; the reported result is an average relative improvement of 19.45% on the SeaWave benchmark and a 28-fold execution-efficiency increase with only a 2.9% performance drop relative to synchronous execution (Zhang et al., 2024).

Other work addresses geometric persistence and viewpoint coverage. “Learning 3D Persistent Embodied World Models” augments a video diffusion world model with a persistent 3D feature map built from RGB-D and DINO features, explicitly targeting the myopia of models conditioned only on currently visible images. Generated RGB-D futures are aggregated into a 3D spatial map and then used to condition subsequent rollouts, with applications to planning and policy learning (Zhou et al., 5 May 2025). Embody4D moves further from 2D PEWM by proposing a generalist 4D world model that synthesizes arbitrary novel views from monocular video. Its 3D-aware compositional synthesis pipeline, confidence-aware adaptive noise injection, and interaction-aware attention are reported to yield a 74% average task success rate for policies trained with Embody4D-augmented data, versus 32–46% for other data models (Tu et al., 3 May 2026).

A different line changes the control representation itself. iMaC replaces low-dimensional action vectors with motion images and contact images, treating raw visual images as native action representations for embodied world models. The paper reports that world-model-estimated success is strongly correlated with real-world outcomes, with $H_s$2–$H_s$3 in 6/8 tasks, and attributes failure cases to inadequate visual coverage rather than to the action representation itself (Wu et al., 8 Jun 2026). In $H_s$4-EVA, the world model becomes an active action-feedback module inside an Envision–Verify–Act loop: a latent dynamics model predicts proposal-conditioned future features, a language-conditioned flow policy proposes an action chunk, and a tri-branch refiner uses the imagined consequence to produce the final action. The complete model is reported at approximately 1.2B parameters, with the central design choice being latent consequence reasoning rather than future video generation at inference (Sun et al., 8 Jun 2026).

Physical grounding has also motivated more radical departures. Hamiltonian World Models encode observations into a structured latent phase space $H_s$5 and evolve them via Hamiltonian-inspired dynamics with control, dissipation, and residual terms, explicitly targeting physically reliable, action-controllable, and long-horizon stable rollouts (Cui et al., 1 May 2026). RoboStereo similarly emphasizes geometry, using a symmetric dual-tower 4D world model over RGB video and 3D pointmap sequences together with unified policy optimization procedures; it reports state-of-the-art generation quality and greater than 97% average relative improvement on fine-grained manipulation tasks (Zhang et al., 13 Mar 2026).

6. Evaluation, misconceptions, and open problems

A central misconception in the area is that better-looking generative rollouts imply better embodied utility. WorldArena was introduced specifically because evaluation had become fragmented and overly centered on perceptual fidelity. It evaluates embodied world models with 16 metrics across six sub-dimensions—Visual Quality, Motion Quality, Content Consistency, Physics Adherence, 3D Accuracy, and Controllability—and also evaluates functional utility through three roles: data engine, policy evaluator, and action planner. Its main empirical conclusion is a significant perception-functionality gap: high visual quality does not necessarily translate into strong embodied task capability (Shang et al., 9 Feb 2026).

World-in-World pushes this point further by replacing open-loop evaluation with a closed-loop world that mirrors real agent-environment interactions. Across four closed-loop environments, it prioritizes task success as the primary metric and reports three “surprises”: visual quality alone does not guarantee task success and controllability matters more; scaling post-training with action-observation data is more effective than upgrading pretrained video generators; and allocating more inference-time compute substantially improves closed-loop performance (Zhang et al., 20 Oct 2025). For PEWM specifically, these results reinforce the original design emphasis on action fidelity, primitive-level controllability, and replanning rather than on unconstrained visual realism.

Open problems remain extensive. A survey of embodied world models identifies the learning challenge of grounding a minimal primitive world model from raw experience, the representation bottleneck of finding a latent abstraction that is both expressive and compact, the difficulty of enforcing causality and physicality, and the computational bottlenecks of sampling efficiency, long-term stability, and scalability across agents and environments. The same survey also emphasizes sim-to-real transfer, robustness in unfamiliar environments, and the interpretability of latent predictions as continuing obstacles (Long et al., 1 Jul 2025).

Recent reinforcement-learning work adds another failure mode: broader exploration without reliable verification becomes highly susceptible to reward hacking. “Reward as an Agent for Embodied World Models” argues that the core limitation is not exploration itself but the lack of reliable verification strategies, and introduces an agentic reward framework together with dynamic-aware rollout diversification to mitigate reward hacking under distribution shifts (Li et al., 18 Jun 2026). In the context of PEWM, this suggests that primitive decomposition alone is insufficient: scalable embodied learning also requires evaluation procedures that verify physical compliance, task completion, and action-grounded correctness under closed-loop use.