minWM: Interactive Video World Models
- minWM is a full-stack framework that transforms pretrained bidirectional video diffusion models into interactive, camera-controllable video world models.
- It employs camera-controllable fine-tuning, autoregressive diffusion training, and few-step causal initialization to enable real-time streaming inference.
- The framework achieves significant latency reduction and precise action conditioning through innovative distillation strategies and modular design.
Searching arXiv for minWM and closely related papers to ground the article. minWM is a full-stack open-source framework for building real-time interactive video world models from existing bidirectional video diffusion backbones (Zhao et al., 28 May 2026). It is designed to convert text-to-video or text/image-to-video foundation models into camera-controllable, few-step autoregressive generators suitable for low-latency rollout. The framework combines camera-controllable fine-tuning, autoregressive diffusion training, causal initialization through causal ODE or causal consistency distillation, asymmetric distribution-matching distillation, and streaming inference within a single end-to-end recipe (Zhao et al., 28 May 2026). A closely related extension describes minWM as the action-conditioned adaptation of the Causal Forcing++ pipeline, emphasizing frame-wise autoregressive rollout under 1–2 diffusion steps per frame and explicit per-frame action conditioning such as camera pose (Zhao et al., 14 May 2026).
1. Definition and scope
minWM denotes a *m*iniature *W*orld *M*odel built for real-time, interactive, camera-controllable video generation (Zhao et al., 14 May 2026). In the framework formulation, its core purpose is to transform a pretrained bidirectional video diffusion model into a controllable few-step autoregressive world model with low first-frame latency and streaming rollout (Zhao et al., 28 May 2026). The framework is positioned as full-stack because it spans data construction, controllable fine-tuning, autoregressive training, few-step distillation, and inference code, rather than providing only an isolated distillation or sampling method (Zhao et al., 28 May 2026).
Two descriptions of minWM appear in the supplied literature. One emphasizes the complete open-source system and modular training recipe across multiple backbones (Zhao et al., 28 May 2026). The other presents minWM as the action-conditioned instantiation of Causal Forcing++, where a bidirectional latent video diffusion model is fine-tuned to accept per-frame action inputs and then distilled into a fully causal student through a three-stage autoregressive pipeline (Zhao et al., 14 May 2026). Taken together, these descriptions indicate that minWM functions both as a concrete world-model implementation and as a reusable recipe for adapting bidirectional video generators into low-latency interactive systems.
A central technical distinction is that minWM targets interactive rollout. This requires causality, controllability, and low-latency generation, constraints that standard bidirectional video diffusion models do not directly satisfy (Zhao et al., 28 May 2026). The framework therefore inserts explicit control signals, enforces autoregressive factorization, and compresses multi-step denoising into few-step generation.
2. Pipeline architecture
minWM consists of two broad phases in the framework presentation: camera-controllable fine-tuning of a bidirectional diffusion backbone, followed by autoregressive diffusion distillation into a few-step generator (Zhao et al., 28 May 2026). In the action-conditioned presentation, the same conversion is organized into three distillation stages after the base model has been fine-tuned for action input (Zhao et al., 14 May 2026). The resulting overall pipeline can be summarized as a progression from bidirectional controllable diffusion to causal few-step autoregressive generation.
The framework first fine-tunes a bidirectional video diffusion model with camera control (Zhao et al., 28 May 2026). Data construction may use videos with known camera poses , either from reconstructed 3D scenes such as DL3DV with rendered trajectories or from sampled images processed by WorldPlay to synthesize videos with prescribed camera motion (Zhao et al., 28 May 2026). This stage preserves the multi-frame, multi-step denoising ability of the backbone while conditioning it on arbitrary camera trajectories through PRoPE-based attention modifications (Zhao et al., 28 May 2026).
The second major phase is autoregressive diffusion distillation. In the framework description, this contains three stages: AR diffusion training under teacher forcing, causal ODE or causal consistency initialization, and asymmetric DMD (Zhao et al., 28 May 2026). In the Causal Forcing++ formulation, the same structure is presented as: Stage 1 AR diffusion pre-training by teacher forcing, Stage 2 causal consistency distillation initialization replacing costly PF-ODE trajectory distillation, and Stage 3 asymmetric distribution-matching distillation with student self-rollout (Zhao et al., 14 May 2026).
The training stages are not merely incremental refinements. Stage 1 establishes a causal model that predicts each frame from a clean or noisy prefix. Stage 2 provides a few-step initialization target aligned with the autoregressive teacher’s conditional flow map. Stage 3 then corrects distributional mismatch that emerges under self-rollout, where the student conditions on its own generated history rather than ground-truth prefixes (Zhao et al., 14 May 2026). This suggests that minWM is organized around progressively reducing the gap between teacher-forced denoising and online interactive rollout.
3. Camera and action conditioning
A defining component of minWM is camera-controllable conditioning. In the full-stack framework, camera information is injected with PRoPE into both cross-attention and self-attention, depending on the backbone architecture (Zhao et al., 28 May 2026). For frame , the lifted projective matrix is defined as
and for token at frame with spatial coordinates , the block-diagonal transform is
This is injected into every self-attention layer through generalized attention:
For cross-attention-based backbones such as Wan2.1-T2V, camera tokens are concatenated with text tokens after embedding, and separate cross-attention heads attend to text and camera-pose embeddings (Zhao et al., 28 May 2026). For MMDiT-style backbones such as HY1.5-TI2V, the same PRoPE transform is injected into every self-attention block inside each MMD block, and cross-attention to the conditioning image similarly adds camera RoPE on spatial latents of the image frame (Zhao et al., 28 May 2026).
In the action-conditioned Causal Forcing++ presentation, the action is the camera pose of the next frame, denoted , described as “6-DOF or similar” (Zhao et al., 14 May 2026). The bidirectional Wan2.1-1.3B model is fine-tuned to accept per-frame action inputs via PRoPE-style positional encoding, and inference generates frame from the student using the generated prefix 0 and a user-provided or policy-generated action 1 (Zhao et al., 14 May 2026). During all training stages, the network input at frame index 2 is 3 (Zhao et al., 14 May 2026). This indicates that control is not a post hoc steering mechanism; it is embedded into both teacher and student training throughout the distillation stack.
4. Training objectives and distillation mechanics
The framework’s Stage 1 autoregressive diffusion training uses a causal attention mask so that the model sees only prefix frames 4, and optimizes the standard diffusion denoising loss under causal masking,
5
The action-conditioned presentation gives an equivalent flow-matching style objective for Stage 1:
6
with
7
Stage 2 can be implemented either as causal ODE initialization or as causal consistency distillation. In the framework paper, causal ODE initialization uses PF-ODE trajectories 8 and trains a few-step student by regressing 9 under prefix 0:
1
The Causal Forcing++ formulation emphasizes that previous causal ODE distillation required storing full PF-ODE trajectories, for example 48 steps per sample, with a cost of approximately 2 K GPU·h and 3 TB storage (Zhao et al., 14 May 2026). Causal consistency distillation replaces this with a local pairing scheme. At each iteration and frame index 4, a real frame 5 and prefix 6 are sampled, the frame is forward-diffused to level 7 as 8, one online PF-ODE step is taken with the frozen teacher to obtain
9
and the student flow 0 is trained to be locally consistent between 1 and 2 (Zhao et al., 14 May 2026). The corresponding objective is
3
where 4 under the velocity parameterization and 5 is an EMA copy of 6 (Zhao et al., 14 May 2026).
The paper states that causal ODE distillation and consistency distillation share the same target, namely the AR-conditional flow map or “consistency function” of the teacher model (Zhao et al., 14 May 2026). It further states that under standard consistency-model theory, the global ODE-to-0 target and the local one-step CD target coincide up to 7 error for a 8th-order solver (Zhao et al., 14 May 2026). This suggests that minWM’s Stage 2 is not simply an engineering simplification; it is motivated by target equivalence combined with a reduced optimization gap from local timestep pairing.
Stage 3 is asymmetric distribution-matching distillation. In the framework paper, self-rollout generates full sequences 9 autoregressively, noise is added at diffusion time 0, and the DMD gradient is
1
The action-conditioned presentation gives the same stage in conditional form with student self-rollout producing 2 from 3 and two critics or score models 4 and 5 based on bidirectional Wan2.1-14B (Zhao et al., 14 May 2026). This final stage explicitly addresses the discrepancy between conditioning on real prefixes during training and generated prefixes during inference.
5. Backbones, modularity, and implementation profile
minWM is described as modular and architecture-extensible (Zhao et al., 28 May 2026). The framework is instantiated on representative open backbones including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, thereby covering both cross-attention-based condition injection and MMDiT-style architectures (Zhao et al., 28 May 2026). It also supports adapting existing video world models such as HY-WorldPlay to new data distributions, training recipes, and latency targets (Zhao et al., 28 May 2026).
For Wan2.1-T2V-1.3B, the backbone is a cross-attention-heavy U-Net with approximately 6 B parameters, latent resolution 7, 8 frames total, and chunk size 9 latents (Zhao et al., 28 May 2026). The listed training recipe is bidirectional fine-tune 0K steps, Stage 1 AR 1K steps, Stage 2 2K steps, and Stage 3 3 steps (Zhao et al., 28 May 2026). For HY1.5-TI2V-8B, the backbone is MMDiT-style with approximately 4 B parameters and the recipe is fine-tune 5K, Stage 1 6K, Stage 2 7K, and Stage 3 8 steps (Zhao et al., 28 May 2026).
The framework allows control signals such as pose, depth, and segmentation to be swapped by replacing PRoPE with another positional encoding such as DepthPE (Zhao et al., 28 May 2026). Latency targets are configured through the number of few-step timesteps 9, chunk size, and distillation hyperparameters, and a recipe JSON is described as specifying backbone, data, 0, and training-stage hyperparameters (Zhao et al., 28 May 2026). An example YAML snippet uses:
The project also releases runnable scripts, checkpoints, documentation, and inference code, with example training commands such as train_bidirectional.py, train_ar_stage1.py, train_distill_ode.py, and train_distill_dmd.py, and inference through demo_interactive.py (Zhao et al., 28 May 2026). These details indicate that minWM is intended as a reproducible research framework rather than only a conceptual model family.
6. Performance, ablations, and operational characteristics
The open-source framework reports a few-step AR model using only 1 timesteps per chunk of 2 latent frames (Zhao et al., 28 May 2026). End-to-end first-frame latency on A800, excluding VAE encode/decode, is reported as 3 s for HY1.5 and 4 s for Wan2.1, corresponding to 5 and 6 speed-up versus bidirectional baselines of 7 s and 8 s, respectively (Zhao et al., 28 May 2026). Throughput at batch 9 is stated as 0–1 fps on A800 for 4-step AR (Zhao et al., 28 May 2026).
The Causal Forcing++ action-conditioned presentation reports frame-wise 2-step minWM performance as 2 FPS on A800, compared with 3 FPS for 4-step chunk-wise baselines, and first-frame latency of 4 s versus 5 s, described as a 6 reduction (Zhao et al., 14 May 2026). It reports VBench Total 7, Quality 8, Semantic 9, VisionReward 0, and Instruction-follow 1, compared with VisionReward 2 and Instruction-follow 3 for Causal Forcing (Zhao et al., 14 May 2026). The same source states that Stage 2 cost drops from 4 GPU·h plus 5 GiB storage for causal ODE initialization to 6 GPU·h and 7 GiB for causal consistency initialization, approximately a 8 speedup with no extra disk (Zhao et al., 14 May 2026). Its ablation states that across 1-, 2-, and 4-step settings, causal CD is at least as good as causal ODE and better than causal DMD, multi-step AR initialization, and Self-Forcing ODE (Zhao et al., 14 May 2026).
The framework paper also provides practical ablations for controllability (Zhao et al., 28 May 2026). For data quality, SpatialVid with perception-estimated poses is described as yielding unreliable controllability, while DL3DV with re-rendering and OpenVid with WorldPlay trajectories are described as reliable or strong (Zhao et al., 28 May 2026). For the number of fine-tuning steps on HY1.5, 9–0K steps show no controllability, about 1K steps show camera response that is unstable, and at least 2K steps yield stable high-fidelity controllability (Zhao et al., 28 May 2026). For minimal batch size on Wan2.1, batch 3 fails to learn control, batch 4 gives partial unstable control, and batch 5 yields stable full controllability (Zhao et al., 28 May 2026). A quantitative metric is the percentage of held-out videos where generated camera-trajectory error is at most 6 average: 7 for batch 8, 9 for batch 00, and 01 for batch 02 (Zhao et al., 28 May 2026).
A concise comparison of selected reported characteristics is useful:
| Aspect | minWM framework | Action-conditioned minWM / Causal Forcing++ |
|---|---|---|
| Backbone examples | Wan2.1-T2V-1.3B, HY1.5-TI2V-8B | Wan2.1-1.3B, critics based on Wan2.1-14B |
| Few-step regime | 03 timesteps per chunk of 4 latent frames | frame-wise 2-step setting |
| First-frame latency | 1.14 s (Wan2.1), 3.45 s (HY1.5) on A800, excluding VAE encode/decode | 0.27 s vs. 0.60 s baseline on A800 |
| Stage 2 distillation cost | causal ODE or causal consistency | 04 GPU·h + 05 GiB to 06 GPU·h + 07 GiB |
These numbers come from different experimental settings and should not be merged into a single unified benchmark. They nevertheless show that minWM is consistently framed around latency reduction and few-step rollout.
7. Limitations, related developments, and interpretation
The supplied materials identify several limitations. In the action-conditioned Causal Forcing++ account, the action-conditioned results are shown only under a chunk-wise 4-step setup, and extending them to fully frame-wise 1–2 steps is explicitly described as future work (Zhao et al., 14 May 2026). The same source notes that long-horizon consistency still degrades gradually and suggests memory modules or hierarchical schemes as potential remedies (Zhao et al., 14 May 2026). In the framework paper, chunk size is fixed, and dynamic chunking or memory networks such as WorldCompass are listed as future directions (Zhao et al., 28 May 2026).
A later comparison with BiWM frames minWM as a four-stage causal pipeline whose limitations include error accumulation, long-horizon quality degradation, camera-control drift, and training/inference overhead from per-layer camera encoder and residual PRoPE modules (Rui et al., 8 Jun 2026). According to that comparison, once generated frames are frozen into the key/value cache, mismatches become permanent and can lead to washed-out geometry, desaturated colors, and vanishing object details over longer rollouts (Rui et al., 8 Jun 2026). It further contrasts minWM’s four-stage recipe with a two-stage bidirectional autoregressive alternative (Rui et al., 8 Jun 2026). Because these claims are presented from the perspective of a subsequent system, they should be read as a comparative characterization rather than as the original minWM authors’ own summary.
At the same time, the BiWM comparison provides useful context for situating minWM within the emerging design space of interactive world models. It identifies minWM as representative of a fully causal approach in which control fine-tuning, autoregressive training, causal initialization, and few-step distillation are performed sequentially (Rui et al., 8 Jun 2026). This suggests that minWM occupies an important position in the transition from bidirectional video diffusion to interactive world modeling: it formalizes a reproducible causal pipeline, but also exposes the trade-offs of strict causality, especially for long-horizon fidelity and controllability.
A plausible implication is that minWM’s lasting significance lies less in a single architecture than in its codification of a modular recipe. The framework demonstrates that open bidirectional T2V and TI2V models can be converted into controllable world models through a structured stack of PRoPE-based control injection, autoregressive training, causal or consistency-based initialization, and DMD refinement (Zhao et al., 28 May 2026). The Causal Forcing++ extension further suggests that the most computationally expensive part of this stack—few-step initialization—can be substantially simplified by replacing offline PF-ODE trajectory storage with online local consistency supervision (Zhao et al., 14 May 2026).