DreamX-World 1.0 Interactive World Model

• DreamX-World 1.0 is an interactive world model that generates streaming video from text/image prompts and user-defined camera trajectories across photorealistic, game-style, and stylized domains.
• It employs novel techniques like E-PRoPE for efficient camera-aware conditioning, geometry-based memory retrieval, and few-step autoregressive distillation to sustain long-horizon scene consistency.
• The system integrates reinforcement learning and optimized pipeline parallelism to enhance camera control and visual fidelity, achieving interactive streaming up to 16 FPS on eight RTX 5090 GPUs.

DreamX-World 1.0 is a 5B-parameter, text/image-to-video world model built for interactive, long-horizon camera-controlled generation across multiple visual domains. It takes a prompt plus a camera trajectory, and optionally an initial image, and produces a streaming video in which the camera can move, revisit places, and trigger complex events while the scene remains consistent over tens of seconds. In the reported system, these capabilities are coupled with camera navigation, revisits to previously observed regions, and promptable events across photorealistic, game-style, and stylized domains; with mixed-precision DiT execution, residual reuse, 75\%-pruned VAE decoding, and asynchronous pipeline parallelism, the system reaches up to $16\,\text{FPS}$ on eight RTX 5090 GPUs (Team et al., 15 Jun 2026).

1. Scope, task definition, and target capabilities

DreamX-World 1.0 is defined as a general-purpose interactive text/image-to-video world model rather than a short-form passive video generator. In the provided formulation, “world model” denotes a system that simulates a persistent 3D world that can be explored interactively. Concretely, it supports text/image-to-video generation, user-specified camera trajectories, online event instructions, streaming generation over tens of seconds to $\sim 1$ minute, and operation across photorealistic, game-style, and stylized domains. Its target capabilities are explicit camera navigation in $6$-DoF, revisits to previously seen regions, promptable events and composable event logic, and chunk-wise autoregressive streaming generation (Team et al., 15 Jun 2026).

The control problem is correspondingly broader than conventional text-to-video conditioning. “Controllable long-horizon generation” requires following explicit camera trajectories rather than inventing arbitrary camera motion, maintaining scene state over long spans, supporting promptable events that modify world state in a structured way, and preserving revisit consistency when the camera returns to a location. The stated comparison point is prior open systems such as HY-WorldPlay 1.5 and LingBot-World, which are described as having weaker geometry-aware camera control, more limited geometry-guided memory for revisits, and less structured support for composable multi-entity events than DreamX-World 1.0 (Team et al., 15 Jun 2026).

A plausible implication is that DreamX-World 1.0 occupies a specific point in the world-model design space: it is not a simulator-grounded renderer in the Dreamland sense, and it is not primarily a token-masked visual sequence model in the WorldDreamer sense, but an interactive video world model whose principal abstractions are camera geometry, autoregressive rollout, memory retrieval, and text-mediated event composition (Mo et al., 9 Jun 2025, Wang et al., 2024).

2. Data engine and camera-aware conditioning

The system’s data engine combines three sources: camera-accurate Unreal Engine rendering, action-rich gameplay recordings, and real-world videos with recovered camera geometry. The UE5 pipeline supplies first-person free-camera exploration, third-person character-following camera sequences, and event subsets with visible state change, together with per-frame camera pose, discrete action vectors, and, for third-person data, character world position and heading. Real-world and game data are drawn from SpatialVID, RealEstate10K, Sekai, DL3DV, Sekai-Game, and OmniWorld-Game. For real-world clips, sparse camera poses are obtained via MegaSaM, rotations are interpolated with SLERP, translations are linearly interpolated, and trajectories are normalized and cleaned by removing inconsistent intrinsics, translation spikes, rapid rotations, vertical jitter, and invalid orientations (Team et al., 15 Jun 2026).

Camera modeling is explicit. For a camera with intrinsics $K$ and extrinsics $[R \mid t]$, the projection matrix is

$P = K [R \mid t].$

DreamX-World inherits PRoPE-style projective camera geometry but introduces E-PRoPE, a lightweight variant that preserves projective structure while reducing cost. The summarized PRoPE decomposition is

$$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$

where $D_{\text{Proj}}$ encodes camera geometry and $D_{\text{ROPE}}$ encodes standard rotary positional embeddings. E-PRoPE retains $D_{\text{Proj}}$ and drops $\sim 1$0, relying on the base DiT’s spatiotemporal inductive bias (Team et al., 15 Jun 2026).

Its efficiency derives from spatial token reduction and projective-only attention. Instead of applying PRoPE over all latent tokens, DreamX-World downsamples along spatial dimensions to a reduced token set, computes camera-aware attention there, upsamples, and adds the result back to the full-resolution DiT stream. The reported example reduces tokens from $\sim 1$1 to $\sim 1$2. In the cited evaluation, E-PRoPE achieves camera control $\sim 1$3 versus $\sim 1$4 for full PRoPE with approximately $\sim 1$5 lower inference latency, which suggests that most of the controllability benefit is retained after the efficiency reduction (Team et al., 15 Jun 2026).

At inference, camera conditioning is chunk-relative. The first chunk encodes poses relative to the first frame of that chunk, whereas later chunks encode poses relative to the last frame of the previous chunk. This is described as preventing signal weakening under autoregressive rollout and keeping camera conditioning consistent with the streaming context (Team et al., 15 Jun 2026).

3. Conversion into a few-step autoregressive world model

DreamX-World 1.0 is built by adapting Wan 2.2 TI2V, a DiT-based text-to-video/image-to-video model, into a camera-aware, memory-augmented, few-step autoregressive world model. Video is encoded by a VAE into latent tokens, diffusion is applied in latent space with a rectified flow training objective, text prompts are injected through the standard text encoder and cross-attention path, optional initial images are handled by conditioning on the first latent frame, and camera trajectories are injected through E-PRoPE (Team et al., 15 Jun 2026).

The central architectural transition is from a bidirectional generator to a causal streaming model. Causal forcing trains an autoregressive student that conditions on past tokens via a rolling KV cache and uses only past context during generation. Distillation is then used to preserve generative quality while reducing sampling cost. DreamX-World adopts a DMD-style distillation regime over local temporal windows of long videos, in which the student few-step autoregressive model is trained to match a bidirectional teacher. The provided formulation writes the distillation term as

$\sim 1$6

This is implemented over short temporal windows sampled from long rollouts, with both text-to-video and image-to-video conditions, including camera control (Team et al., 15 Jun 2026).

Long-horizon stability is addressed with long-rollout training. The student generates long sequences, sub-windows are sampled from those sequences, and DMD plus rectified-flow losses are applied while the model is conditioned on its own generated history. Infinity-RoPE is used to extend positional encodings for long contexts. The reported purpose is to expose the model to its own generated history and reduce the style and color drift that accumulates across autoregressive chunks. A plausible implication is that DreamX-World treats rollout mismatch as a first-class training problem, rather than only a sampling-time artifact (Team et al., 15 Jun 2026).

4. Memory-conditioned scene persistence and event instruction tuning

Long-horizon revisit consistency is handled through Memory-Conditioned Scene Persistence. Given a latent sequence $\sim 1$7 with camera signals $\sim 1$8, the model packs retrieved memory frames, recent history, and current targets into a unified self-attention stream:

$\sim 1$9

Here, $6$0 denotes current target latents to be denoised, $6$1 denotes recent history latents, and $6$2 denotes clean latent frames retrieved from earlier history through geometry-based retrieval. The DiT processes this concatenated sequence in self-attention, while the loss is applied only on the target frames $6$3 under a rectified flow denoising objective (Team et al., 15 Jun 2026).

The retrieval mechanism is geometry-based rather than purely temporal. Earlier frames are selected according to camera pose proximity and view overlap, and memory frames retain their original temporal RoPE when inserted into the packed sequence. For large time gaps, long-range RoPE treatments such as NTK-aware scaling, YaRN, and randomized positional encodings are applied. During training, DreamX-World injects error into conditioning frames, following Stable Video Infinity, so that the model learns to use memory when it is helpful and to fall back to its prior when memory is noisy. Residual recycling further makes the conditioning path less sensitive to imperfect memory latents. The reported outcome is stronger revisit consistency, with gains on pixel-level, perceptual, semantic, and place-recognition metrics relative to non-revisit baselines (Team et al., 15 Jun 2026).

Event control is introduced through Event Instruction Tuning. The event dataset uses a hierarchical annotation format comprising a global description of static scene context and overall temporal evolution, plus entity-level event records with an entity reference, event predicate, spatial anchor, and temporal interval. Interactions are represented compositionally by assigning separate event records to each participating entity while describing inter-object relations in the global caption. The architecture is unchanged in this stage: structured event records are rendered into natural-language prompts, text-conditioning remains the only interface, and the full DiT is fine-tuned conservatively with strict gradient clipping to preserve pretrained visual and motion priors (Team et al., 15 Jun 2026).

This design indicates that DreamX-World does not introduce explicit symbolic event slots inside the backbone. Instead, compositionality is delegated to the text encoder and cross-attention pathway. The intended operating regime includes multi-entity, interaction-level prompts in combination with free camera motion and global scene prompts, which differentiates it from systems that expose only coarse event prompting (Team et al., 15 Jun 2026).

5. Reinforcement learning alignment and systems implementation

After distillation, DreamX-World applies reinforcement learning post-training to recover camera control and visual quality. The diffusion denoising process is treated as a policy, long-horizon autoregressive rollouts are generated from text-image-camera conditions, short clips are sampled from those rollouts for tractable optimization, and two reward models score each clip: a camera-control reward, based on horizontal translation and rotation accuracy versus the commanded trajectory using pose estimation, and a video-quality reward that measures perceptual quality and artifact presence. The update uses DiffusionNFT with KL regularization:

$6$4

The update is described as soft, with a gradual schedule, small learning rates, and strong KL regularization to maintain long-horizon stability while improving camera following and visual fidelity (Team et al., 15 Jun 2026).

The inference path is optimized for chunk-wise autoregressive streaming. Each chunk is denoised with a few-step sampler conditioned on the text prompt, a chunk-relative camera trajectory through E-PRoPE, and a rolling KV cache storing history across chunks. Image-to-video uses the same interface except that the first chunk’s first frame is replaced by the encoded input image. The deployment stack combines INT8 SageAttention in attention layers, FP8 quantization of FFN layers via AngelSlim, sequence parallelism for long token streams, fused Triton kernels, residual reuse across timesteps following TeaCache, and ParaVAE-style distributed decoding (Team et al., 15 Jun 2026).

On the decoding side, the system uses Matrix-Game 3.0 VAE with $6$5 pruning, reported at approximately $6$6 per chunk decoding, with additional acceleration after the first iteration through torch.compile. Asynchronous pipeline parallelism overlaps decoding of chunk $6$7 with control reception for chunk $6$8, KV-cache updates, and DiT denoising for chunk $6$9. The aggregate result is interactive streaming with up to $K$0 on $K$1 RTX 5090 GPUs (Team et al., 15 Jun 2026).

6. Evaluation, benchmark results, limitations, and research context

DreamX-World 1.0 is evaluated on a 5-second basic evaluation, a long-horizon $K$2-second evaluation, a memory/revisit benchmark, and a human preference study. Camera control follows a WorldScore-style metric with MegaSaM for pose recovery and an enriched trajectory set including tilt and diagonal movement. The camera error is defined as

$K$3

where $K$4 is rotation error and $K$5 is translation error; this is normalized into a $K$6 score. Additional metrics include imaging quality, temporal flicker, motion smoothness, dynamic degree, transition detection, and frame-level artifact pass rate as judged by Gemini-3.1-Pro on sampled frames at $K$7 (Team et al., 15 Jun 2026).

On the 5-second benchmark, DreamX-World-1.0-5B reports camera $K$8, quality $K$9, flicker $[R \mid t]$0, motion smoothness $[R \mid t]$1, dynamic degree $[R \mid t]$2, artifact $[R \mid t]$3, and overall $[R \mid t]$4. The compared systems HY-WorldPlay 1.5 and LingBot-World report overall scores of $[R \mid t]$5 and $[R \mid t]$6, respectively. On the $[R \mid t]$7-second benchmark, DreamX-World-1.0-5B reports camera $[R \mid t]$8, quality $[R \mid t]$9, artifact $P = K [R \mid t].$0, and overall $P = K [R \mid t].$1, again exceeding the overall scores of HY-WorldPlay and LingBot. These results support the paper’s characterization of the model as smaller than the compared systems while achieving the highest camera control and overall score in the 5-second evaluation, and the best overall score in the long-horizon evaluation (Team et al., 15 Jun 2026).

Memory evaluation uses revisit trajectories such as out-and-back, translation-rotation, and closed-loop motion. Revisit pairs satisfy

$P = K [R \mid t].$2

with $P = K [R \mid t].$3 and $P = K [R \mid t].$4, together with a minimal temporal gap. The reported gains relative to non-revisit baselines are $P = K [R \mid t].$5, $P = K [R \mid t].$6, $P = K [R \mid t].$7, $P = K [R \mid t].$8, $P = K [R \mid t].$9, $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$0, and CLIP-V $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$1. In side-by-side human comparison, DreamX-World is preferred overall against HY-WorldPlay 1.5 by $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$2 win, $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$3 tie, $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$4 lose, and against LingBot-World by $$D_{\text{PROPE}} = \begin{bmatrix} D_{\text{Proj}} \ D_{\text{ROPE}} \end{bmatrix},$$5 win, corroborating the quantitative emphasis on visual quality, artifact robustness, and strong revisit behavior (Team et al., 15 Jun 2026).

The stated limitations remain substantive. Despite Memory-Conditioned Scene Persistence and long-rollout training, worlds can still drift in object appearance and change layout over very long interactions. Caption, camera, and event instructions can conflict, leading the model to prefer one control source or to produce an incoherent compromise. The authors also note that automatic metrics for world models remain incomplete, and that domain and resolution limits persist outside the trained operating range. The future directions explicitly listed are character-centric world models and native audio-visual world models, alongside stronger memory, stronger physical reasoning, and better physics-aware simulation (Team et al., 15 Jun 2026).

Within the adjacent literature, multiple world-model paradigms remain active. WorldDreamer formulates world modeling as masked visual token prediction over a spatio-temporal token field (Wang et al., 2024). Dreamland combines a physics-based simulator with pretrained generative models through a Layered World Abstraction to obtain controllable world creation (Mo et al., 9 Jun 2025). DreamWorld integrates temporal, semantic, and spatial feature targets into video generation through a joint world modeling paradigm (Tan et al., 28 Feb 2026). DreamX-World 1.0 is situated differently: its defining contribution in the provided material is the integration of camera-aware conditioning, few-step autoregressive rollout, geometry-based memory retrieval, composable event prompting, and systems-level streaming efficiency into a single interactive world model (Team et al., 15 Jun 2026).