Riemann-1.0 Causal World Action Model

• Riemann-1.0 is a causal, autoregressive World Action Model (WAM) designed to predict both future actions and visual states from past observations and actions, suitable for robot manipulation and embodied intelligence.
• The model utilizes a Progressive Embodied Pretraining framework to enhance robotic skill transfer using heterogeneous datasets, including egocentric human videos and heterogeneous robot trajectories.
• Riemann-1.0 demonstrates strong performance on tasks such as cube stacking, kitchen organization, and long-horizon desk rearrangement, achieving notable success rates in multiple real-world and simulation environments.

Riemann-1.0 is a fully causal autoregressive World Action Model (WAM) for embodied intelligence and robot manipulation. It jointly models multi-view visual observations, robot states, and embodiment-specific actions within a unified causal sequence, representing action and subsequent visual evolution as causal state transitions. The same model supports online robot-policy execution and action-conditioned visual simulation. Its associated Progressive Embodied Pretraining framework transfers information from egocentric human videos, handheld-gripper demonstrations, wearable demonstrations, and heterogeneous robot trajectories into executable manipulation capabilities. The system is described in “Riemann-1.0: An Embodied World Action Model for Physical AI” (Sun et al., 27 Aug 2026).

1. Conceptual scope and design objectives

Riemann-1.0 is designed around the causal ordering

$$\text{past observations and states}\rightarrow \text{current action}\rightarrow \text{resulting visual state}.$$

This distinguishes it from conventional vision-language-action policies, which are principally optimized as observation-to-action predictors,

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$

Riemann-1.0 additionally predicts the visual consequences of actions:

$$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$

The architecture consequently operates in two modes. In online policy execution, it predicts an action chunk, executes the action on a robot, receives new observations and state measurements, and feeds the resulting observations back into the model. In action-conditioned visual simulation, it receives or samples a candidate action trajectory and autoregressively generates the corresponding visual consequences.

The model is positioned against three WAM paradigms. Joint-generation systems model visual latents and actions simultaneously, but must jointly denoise variables with different dimensionalities, temporal resolutions, and optimization characteristics. Video-first systems generate future visual latents before inferring actions, producing additional inference latency and an ordering mismatch with online interaction. Decoupled action-video systems use separate models for visual and action generation, improving modularity but retaining dependence on predicted visual features for action inference.

Riemann-1.0 instead predicts the action from preceding observations, states, actions, and conditioning information, then predicts the resulting visual latent conditioned on that action. This ordering is intended to support real-time control, action-conditioned simulation, action-plan comparison, long-horizon imagination, and causal consistency in autoregressive rollouts.

The model does not generate robot states. States are directly observed from the physical environment, replayed from recorded trajectories, or otherwise externally supplied during simulation and evaluation. Its generated variables are primarily action chunks and future visual latents, while robot states remain conditioning variables.

2. Unified multimodal architecture

Let $z_t$ denote a visual latent, $s_t$ a robot state, $a_t$ an action chunk, and $c$ the task and embodiment condition. Riemann-1.0 factorizes a rollout as

$$p(a_{1:T},z_{1:T}\mid z_0,s_0,c) = \prod_{t=1}^{T} p(a_t\mid z_{<t},s_{<t},a_{<t},c)\, p(z_t\mid z_{<t},s_{<t},a_{\leq t},c).$$

At each transition, the action is predicted from preceding context; the action is appended to that context; the visual latent resulting from the action is predicted; and the resulting visual observation and state become context for the next transition.

Language specifies the task, embodiment, and camera configuration. The reported prompt template is:

$s_t$9

A T5 encoder processes the language condition and injects it into the transformer through cross-attention. A numerical embodiment ID selects embodiment-specific action projections, state projections, and action-prediction heads. The transformer backbone is shared across robots, while the input and output interfaces remain embodiment-specific.

Multi-view images are arranged on a unified embodiment-specific canvas, resized to the model resolution, encoded with the Wan VAE, and converted into transformer tokens through a 3D patch embedding. The initial visual latent $z_0$ provides context. During simulation, future visual latents are generated; during robot execution, they are replaced by real observations.

Raw states and actions are reordered into an embodiment-specific canonical order, normalized using per-embodiment statistics, projected into shared model dimensions, and padded to fixed maximum dimensions. If the raw action and state dimensions are $d_a$ and $$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$0, the padded dimensions satisfy

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$1

This padding does not impose a common physical action space. It permits a shared temporal and visual backbone while embodiment-specific projections and heads preserve the semantics of end-effector motion, gripper commands, joint commands, hand-pose changes, handheld-gripper motion, and latent transition actions. Validity masks identify meaningful action channels and time steps.

Training serializes the modalities into a unified sequence containing context latent, clean latent, noisy latent, state, clean action, and noisy action tokens. The context latent is the first observed frame; clean latent and action tokens represent teacher-forced history; noisy target tokens are used for flow matching; and state tokens represent observed initial and intermediate states. One visual latent corresponds to 16 low-level action steps in the reported simulation interface. The state sequence contains one additional state relative to the number of visual transitions:

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$2

3. Flow matching and causal training

Riemann-1.0 applies conditional flow matching to both visual latents and continuous actions. For a data variable $$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$3, Gaussian noise $$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$4, and noise level $$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$5,

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$6

with target velocity

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$7

The generic training loss is

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$8

For a future visual latent, the visual head predicts

$$(\text{image},\text{instruction},\text{state})\rightarrow\text{action}.$$9

and is trained with a visual-latent loss $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$0. The current action $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$1 is included because $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$2 represents the visual consequence of that action.

The action head predicts

$$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$3

with action loss $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$4. Crucially, the action prediction is conditioned only on preceding history and does not use the future visual latent caused by the action.

The unified objective is

$$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$5

where $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$6 controls the emphasis on executable action prediction. The reported curriculum uses $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$7 in Stage I, $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$8 in Stage II, $$\text{history}\rightarrow\text{action}\rightarrow\text{future visual state}.$$9 in Stage III, and $z_t$0 during post-training.

A structured causal attention mask prevents future-observation leakage. Clean tokens attend only to clean tokens at current or preceding time steps. Noisy target tokens attend to preceding clean tokens and may attend to other noisy tokens in the same generation block, but cannot attend to future clean observations. Padding, including text padding, is masked. Local generation blocks may use bidirectional attention internally while the global sequence remains causal.

Separate latent and action masks exclude padded frames, invalid latents, padded action channels, invalid low-level action steps, and episode-boundary regions. Losses are normalized only over valid entries. Inactive padded regions remain zero in clean samples, injected noise, noisy samples, and velocity targets, avoiding a train-test mismatch between noisy training channels and zero-valued deployment channels.

4. Progressive Embodied Pretraining

The Progressive Embodied Pretraining framework organizes supervision as

$z_t$1

The reported corpus descriptions refer to more than 200K hours of interaction data; the infrastructure section reports more than 230K hours comprising more than 200K hours of egocentric human videos, more than 12K hours of handheld-gripper and wearable demonstrations, and more than 20K hours of heterogeneous robot trajectories. The source also notes that these quantities are reported approximately and are not presented as an exactly deduplicated aggregate.

All data sources are converted into a common trajectory representation containing language instruction, embodiment identity, multi-view observations, robot or hand states, actions, and semantic metadata. Processing includes visual normalization, task and action segmentation, semantic annotation, quality filtering, 3D hand reconstruction, geometric filtering, action calibration, temporal alignment, per-embodiment normalization, and semantic-aware data balancing.

For human videos, 3D hand trajectories are reconstructed using hand tracking and MANO. Camera intrinsics and poses are estimated using VGGT-$z_t$2, allowing hand motion to be represented in a consistent coordinate system despite egocentric camera motion. Handheld-gripper and robot data receive temporal alignment, action normalization, gripper-state boundary refinement, camera-shake filtering, stationary-end-effector rejection, and abnormal-control filtering. Semantic-aware sampling preserves human-video diversity while increasing exposure to long-tail skills and low-resource embodiments.

Stage I: LAM-Action Bootstrap

Stage I uses unlabeled egocentric human manipulation videos. A frozen Latent Action Model (LAM) is trained on adjacent frames $z_t$3. The LAM patchifies each frame into $z_t$4 image patches, prepends an action-prompt token, processes both frames with a spatio-temporal transformer, projects the future-frame action-prompt hidden state to Gaussian moments, samples a 32-dimensional latent action, and reconstructs the next frame.

Its objective is

$z_t$5

The decoder cannot directly observe $z_t$6, so the bottleneck must encode the transition. After training, the posterior mean $z_t$7, rather than a sampled latent, is used as a deterministic pseudo-action.

Adjacent-frame latent actions are grouped into chunks matching the WAM temporal interface. Video frames are subsampled and encoded using the Wan VAE. With $z_t$8, Stage I emphasizes broad visual dynamics and latent action-motion associations. It is not intended to produce a deployable robot policy.

Stage II: Trajectory-Grounded Alignment

Stage II combines UMI or handheld-gripper demonstrations, heterogeneous robot trajectories, and human videos with reconstructed 3D hand poses and keypoints. Its action weight is $z_t$9.

Each trajectory is converted into VAE visual latents, action chunks aligned to visual latents, and a state sequence containing the initial state and post-transition states. Actions and states are canonicalized, normalized, padded, and masked according to embodiment. The stage aligns the shared backbone across different robot bodies, camera layouts, action spaces, and state interfaces.

Stage III: Robot Policy Enhancement

Stage III uses high-quality robot demonstrations with real action-state supervision. Human video, LAM pseudo-actions, UMI trajectories, and 3D hand-pose human trajectories are excluded. The action weight is increased to $s_t$0.

Training emphasizes state-conditioned action prediction, trajectory consistency, executable robot behavior, contact-rich manipulation, and actuator-specific conventions. The visual objective remains active as a regularizer and preserves action-conditioned visual modeling.

Post-training

For real-world adaptation, one aggregated dataset covers four tasks on the Tianji Marvin dual-arm robot: ordered four-cube stacking, clothes folding, desk organization, and kitchen organization. The paper reports both three hours of teleoperation per task and 15 demonstrations per task. These descriptions are not further reconciled in the source. The four datasets are jointly used to train one generalist model, with $s_t$1 increased to $s_t$2.

5. Robot execution and visual simulation

During online execution, the initial multi-view observation is encoded as $s_t$3, and the initial state $s_t$4 is embedded. Text and embodiment information initialize the transformer and KV cache. The action head samples noisy action chunks, flow denoising produces a clean action chunk, and the chunk is executed for a specified number of low-level control steps. The robot then returns new camera observations and an updated state. The new observation is encoded, and the visual latent, state, and executed action are appended to the cache.

The deployment distinction is

$s_t$5

whereas simulation uses

$s_t$6

Thus predicted visual states are not used as the next context during real execution; actual environment observations close the feedback loop. For long horizons, the KV cache is maintained within a fixed window. When the window is full, the cache is reset and the latest observation becomes the new context frame.

In simulation mode, the model receives the current visual observation, task prompt, robot state, embodiment identity, and a candidate action trajectory. The action trajectory is embedded through the same embodiment-specific interface used for policy learning. The model predicts future visual latents according to

$s_t$7

which are decoded into RGB video. Generated observations can be fed back as context for additional autoregressive steps. Candidate actions may be generated by Riemann-1.0, sampled from a planner, or taken from a recorded trajectory.

This provides a common interface for visual rollout inspection, action-plan comparison, future-state imagination, model-based planning research, and multi-embodiment simulation. The simulator is not a complete physical simulator because robot states are externally supplied rather than predicted. Generated visual consequences therefore do not constitute a complete learned state-space transition model.

6. Evaluation, reported performance, and limitations

Riemann-1.0 is evaluated on RoboCasa-365, RoboTwin 2.0, and LIBERO. In the reported simulation experiments, one visual latent corresponds to 16 low-level action steps.

On RoboCasa-365, the model obtains 74.2% on Atomic-Seen, 56.0% on Composite-Seen, 56.3% on Composite-Unseen, and 62.6% average success. It improves over the reported ABot-M0.5 results by 3.6 points on Atomic-Seen, 11.7 points on Composite-Seen, 10.7 points on Composite-Unseen, and 8.4 points overall.

On RoboTwin 2.0, Riemann-1.0 obtains 94.6% on clean scenes, 94.0% on randomized scenes, and 94.3% average success. It has the highest reported average, although ABot-M0.5 is slightly higher in the randomized condition by 0.2 points. The training configuration uses 50 bimanual tasks, 50 clean demonstrations per task, 25,000 randomized demonstrations in total, and 500 randomized demonstrations per task.

On LIBERO, Riemann-1.0 obtains 99.6% on Spatial, 100.0% on Object, 97.6% on Goal, 98.6% on Long, and 99.0% average success. It is not the best reported model on Goal, where Being-H0.5 reaches 99.4%.

Real-world evaluation uses the Tianji Marvin dual-arm robot and covers ordered rigid-object stacking, deformable-object manipulation, long-horizon desk rearrangement, and cluttered kitchen organization. Riemann-1.0 obtains 85.0% average Success Rate (SR) and 94.43% average Progress Success Rate (PSR). SR measures final task completion, whereas PSR measures the fraction of task-specific intermediate milestones completed.

Reported task-level results include 85.0% SR and 91.6% PSR for cube stacking; 90.0% SR and 98.4% PSR for kitchen organization; 85.0% SR and 92.5% PSR for clothes folding; and 80.0% SR and 95.2% PSR for desk organization. The model exceeds the strongest open-source baseline by 15 percentage points in average SR according to the paper. The higher PSR relative to SR suggests that failures often occur late in long-horizon sequences rather than at initial task execution.

Compositional evaluation uses 10 real-robot trials per task. For cube-to-bowl or cube-to-plate transfer, Riemann-1.0 obtains 8/10 successes; under added color and ordering constraints, it obtains 5/10. The average is 65.0%, compared with 45.0% for $s_t$8 and 15.0% for LingBot-VA. In out-of-domain evaluation, it obtains 10/10 on Rubik’s cube to storage box and 7/10 on towel to basin, averaging 85.0%.

The evidence has several limitations. The model generates visual latents but not robot states, so hidden physical variables, contact forces, joint states, and state estimation remain external. Visual-simulation assessment is primarily qualitative: the paper does not report pixel reconstruction error, FVD, PSNR, SSIM, object-state prediction accuracy, or action-conditioned consistency metrics.

The real-world evaluation uses four post-trained tasks on one robot platform, with out-of-domain testing based on 10 trials per task. The paper does not report a conventional numerical ablation isolating progressive-pretraining stages, individual data sources, action-loss weights, causal masking, predicted versus observed visual feedback, number of embodiments, model scale, context length, denoising steps, KV-cache size, throughput, control frequency, GPU requirements, or training duration.

The exact training scale is also underspecified: the source reports both more than 200K hours and more than 230K hours without clarifying the deduplicated total. It does not report the final parameter count, transformer depth, hidden dimension, image resolution, optimizer settings, batch size, learning rate, hardware, or training duration. Comparisons may also involve differing model sizes, training datasets, compute budgets, and tuning conditions; the paper explicitly states that the DreamZero* baseline was retrained at 5B scale on the authors’ collected large-scale manipulation dataset.

Riemann-1.0 therefore represents a unified causal WAM architecture combined with a progressive supervision curriculum. Its defining technical property is the action-before-consequence factorization: actions are inferred from past context, while future visual states are conditioned on those actions. Its reported results indicate strong performance on simulation, compositional generalization, long-horizon manipulation, and selected out-of-domain tasks, while the absence of quantitative simulator metrics, causal ablations, complete model specifications, and predicted-state dynamics leaves the relative contribution of each architectural and data component unresolved.