---
title: 'minWM: Interactive Video World Models'
url: https://www.emergentmind.com/topics/minwm
type: topic
---

# minWM: Interactive Video World Models

Searching arXiv for minWM and closely related papers to ground the article.
minWM is a full-stack open-source framework for building real-time interactive video world models from existing bidirectional video diffusion backbones [2605.30263]. It is designed to convert text-to-video or text/image-to-video foundation models into camera-controllable, few-step autoregressive generators suitable for low-latency rollout. The framework combines camera-controllable fine-tuning, autoregressive diffusion training, causal initialization through causal ODE or causal consistency distillation, asymmetric distribution-matching distillation, and streaming inference within a single end-to-end recipe [2605.30263]. A closely related extension describes minWM as the action-conditioned adaptation of the Causal Forcing++ pipeline, emphasizing frame-wise autoregressive rollout under 1–2 diffusion steps per frame and explicit per-frame action conditioning such as camera pose [2605.15141].

## 1. Definition and scope

minWM denotes a *m*iniature *W*orld *M*odel built for real-time, interactive, camera-controllable video generation [2605.15141]. In the framework formulation, its core purpose is to transform a pretrained bidirectional video diffusion model into a controllable few-step autoregressive world model with low first-frame latency and streaming rollout [2605.30263]. The framework is positioned as full-stack because it spans data construction, controllable fine-tuning, autoregressive training, few-step distillation, and inference code, rather than providing only an isolated distillation or sampling method [2605.30263].

Two descriptions of minWM appear in the supplied literature. One emphasizes the complete open-source system and modular training recipe across multiple backbones [2605.30263]. The other presents minWM as the action-conditioned instantiation of Causal Forcing++, where a bidirectional latent video diffusion model is fine-tuned to accept per-frame action inputs and then distilled into a fully causal student through a three-stage autoregressive pipeline [2605.15141]. Taken together, these descriptions indicate that minWM functions both as a concrete world-model implementation and as a reusable recipe for adapting bidirectional video generators into low-latency interactive systems.

A central technical distinction is that minWM targets *interactive* rollout. This requires causality, controllability, and low-latency generation, constraints that standard bidirectional video diffusion models do not directly satisfy [2605.30263]. The framework therefore inserts explicit control signals, enforces autoregressive factorization, and compresses multi-step denoising into few-step generation.

## 2. Pipeline architecture

minWM consists of two broad phases in the framework presentation: camera-controllable fine-tuning of a bidirectional diffusion backbone, followed by autoregressive diffusion distillation into a few-step generator [2605.30263]. In the action-conditioned presentation, the same conversion is organized into three distillation stages after the base model has been fine-tuned for action input [2605.15141]. The resulting overall pipeline can be summarized as a progression from bidirectional controllable diffusion to causal few-step autoregressive generation.

The framework first fine-tunes a bidirectional video diffusion model with camera control [2605.30263]. Data construction may use videos with known camera poses $\{K_i, T_i^{cw}\}$, either from reconstructed 3D scenes such as DL3DV with rendered trajectories or from sampled images processed by WorldPlay to synthesize videos with prescribed camera motion [2605.30263]. This stage preserves the multi-frame, multi-step denoising ability of the backbone while conditioning it on arbitrary camera trajectories through PRoPE-based attention modifications [2605.30263].

The second major phase is autoregressive diffusion distillation. In the framework description, this contains three stages: AR diffusion training under teacher forcing, causal ODE or causal consistency initialization, and asymmetric DMD [2605.30263]. In the Causal Forcing++ formulation, the same structure is presented as: Stage 1 AR diffusion pre-training by teacher forcing, Stage 2 causal consistency distillation initialization replacing costly PF-ODE trajectory distillation, and Stage 3 asymmetric distribution-matching distillation with student self-rollout [2605.15141].

The training stages are not merely incremental refinements. Stage 1 establishes a causal model that predicts each frame from a clean or noisy prefix. Stage 2 provides a few-step initialization target aligned with the autoregressive teacher’s conditional flow map. Stage 3 then corrects distributional mismatch that emerges under self-rollout, where the student conditions on its own generated history rather than ground-truth prefixes [2605.15141]. This suggests that minWM is organized around progressively reducing the gap between teacher-forced denoising and online interactive rollout.

## 3. Camera and action conditioning

A defining component of minWM is camera-controllable conditioning. In the full-stack framework, camera information is injected with PRoPE into both cross-attention and self-attention, depending on the backbone architecture [2605.30263]. For frame $i$, the lifted projective matrix is defined as

\[
\tilde P_i =
\begin{bmatrix}
[K_i\;0]\,T_i^{cw}\\
e_4^\top
\end{bmatrix},
\qquad
e_4=(0,0,0,1)^\top,
\]

and for token $t$ at frame $i(t)$ with spatial coordinates $(x_t,y_t)$, the block-diagonal transform is

\[
D_t^{\mathrm{PRoPE}} =
\begin{bmatrix}
I_{d/8}\otimes \tilde P_{i(t)} & 0 \\
0 &
\begin{bmatrix}
\mathrm{RoPE}_{d/4}(x_t) & 0\\
0 & \mathrm{RoPE}_{d/4}(y_t)
\end{bmatrix}
\end{bmatrix}.
\]

This is injected into every self-attention layer through generalized attention:

\[
\mathrm{Attn}_{\mathrm{PRoPE}}(Q,K,V)
=
D^{\mathrm{PRoPE}} \odot
\mathrm{Attn}\bigl(
(D^{\mathrm{PRoPE}})^\top \odot Q,\,
(D^{\mathrm{PRoPE}})^{-1} \odot K,\,
(D^{\mathrm{PRoPE}})^{-1} \odot V
\bigr)
\]
[2605.30263].

For cross-attention-based backbones such as Wan2.1-T2V, camera tokens are concatenated with text tokens after embedding, and separate cross-attention heads attend to text and camera-pose embeddings [2605.30263]. For MMDiT-style backbones such as HY1.5-TI2V, the same PRoPE transform is injected into every self-attention block inside each MMD block, and cross-attention to the conditioning image similarly adds camera RoPE on spatial latents of the image frame [2605.30263].

In the action-conditioned Causal Forcing++ presentation, the action is the camera pose of the next frame, denoted $a^i$, described as “6-DOF or similar” [2605.15141]. The bidirectional Wan2.1-1.3B model is fine-tuned to accept per-frame action inputs via PRoPE-style positional encoding, and inference generates frame $i$ from the student using the generated prefix $\hat x^{<i}$ and a user-provided or policy-generated action $a^i$ [2605.15141]. During all training stages, the network input at frame index $i$ is $(x_t^i, x^{<i}, a^i)$ [2605.15141]. This indicates that control is not a post hoc steering mechanism; it is embedded into both teacher and student training throughout the distillation stack.

## 4. Training objectives and distillation mechanics

The framework’s Stage 1 autoregressive diffusion training uses a causal attention mask so that the model sees only prefix frames $<i$, and optimizes the standard diffusion denoising loss under causal masking,

\[
L_{AR} = E_{x_0,\epsilon,t,i}\bigl[\|\epsilon - \epsilon_\theta(x_t^i \mid x_0^{<i})\|^2\bigr]
\]
[2605.30263].

The action-conditioned presentation gives an equivalent flow-matching style objective for Stage 1:

\[
\mathcal L_{\mathrm{diff}}
=
\mathbb{E}_{x_0,x_0^{<i},t,\epsilon}
\Big\|
v_\theta\bigl(x_t^i,x_0^{<i},t,a^i\bigr)
-\tfrac{\mathrm{d}}{\mathrm{d}t}x_t^i
\Big\|_2^2,
\]

with

\[
x_t^i = (1-t)x_0^i + t\epsilon,
\qquad
\frac{\mathrm{d}}{\mathrm{d}t}x_t^i = -(x_0^i - x_t^i)
\]
[2605.15141].

Stage 2 can be implemented either as causal ODE initialization or as causal consistency distillation. In the framework paper, causal ODE initialization uses PF-ODE trajectories $x_s^i$ and trains a few-step student by regressing $x_s^i \to x_0^i$ under prefix $x_0^{<i}$:

\[
\theta^* = \arg\min_\theta E_{gt^{<i},s,i,x_s^i}\bigl[\|G_\theta(x_s^i,x_0^{<i},s) - x_0^i\|^2\bigr]
\]
[2605.30263].

The Causal Forcing++ formulation emphasizes that previous causal ODE distillation required storing full PF-ODE trajectories, for example 48 steps per sample, with a cost of approximately $11.6$ K GPU·h and $1.9$ TB storage [2605.15141]. Causal consistency distillation replaces this with a local pairing scheme. At each iteration and frame index $i$, a real frame $x_0^i$ and prefix $x_0^{<i}$ are sampled, the frame is forward-diffused to level $t$ as $x_t^i=(1-t)x_0^i+t\epsilon$, one online PF-ODE step is taken with the frozen teacher to obtain
\[
\hat x^i_{t-\Delta t} = x_t^i - \Delta t\, v_\phi(x_t^i,x_0^{<i},t),
\]
and the student flow $G_\theta$ is trained to be locally consistent between $(x_t^i,x_0^{<i},t)$ and $(\hat x^i_{t-\Delta t},x_0^{<i},t-\Delta t)$ [2605.15141]. The corresponding objective is

\[
\mathcal L_{\mathrm{CD}}(\theta)
=
\mathbb{E}_{x_0^{<i},t,\epsilon}
\Big[
w(t)\,
\big\|
G_\theta(x_t^i,x_0^{<i},t)
-
G_{\theta^-}(\hat x^i_{t-\Delta t},x_0^{<i},t-\Delta t)
\big\|_2^2
\Big],
\]

where $G_\theta(x_t,\cdot,t)=x_t - t\,v_\theta(x_t,\cdot,t)$ under the velocity parameterization and $\theta^-$ is an EMA copy of $\theta$ [2605.15141].

The paper states that causal ODE distillation and consistency distillation share the same target, namely the AR-conditional flow map or “consistency function” of the teacher model [2605.15141]. It further states that under standard consistency-model theory, the global ODE-to-0 target and the local one-step CD target coincide up to $\mathcal O((\Delta t)^p)$ error for a $p$th-order solver [2605.15141]. This suggests that minWM’s Stage 2 is not simply an engineering simplification; it is motivated by target equivalence combined with a reduced optimization gap from local timestep pairing.

Stage 3 is asymmetric distribution-matching distillation. In the framework paper, self-rollout generates full sequences $\tilde x$ autoregressively, noise is added at diffusion time $t$, and the DMD gradient is

\[
\nabla_\theta D_{KL}\bigl(p_{\theta,t}(\tilde{x}_t)\,\|\,p_{data,t}(\tilde{x}_t)\bigr)
=
-\mathbb{E}_{\tilde{x},t,\tilde{x}_t}
\Big[
\bigl(s_{real}(\tilde{x}_t,t)-s_{fake}(\tilde{x}_t,t)\bigr)\,
\tfrac{\partial \tilde{x}}{\partial\theta}
\Big]
\]
[2605.30263].

The action-conditioned presentation gives the same stage in conditional form with student self-rollout producing $\tilde x^i$ from $\tilde x^{<i}$ and two critics or score models $s_{\mathrm{real}}$ and $s_{\mathrm{fake}}$ based on bidirectional Wan2.1-14B [2605.15141]. This final stage explicitly addresses the discrepancy between conditioning on real prefixes during training and generated prefixes during inference.

## 5. Backbones, modularity, and implementation profile

minWM is described as modular and architecture-extensible [2605.30263]. The framework is instantiated on representative open backbones including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, thereby covering both cross-attention-based condition injection and MMDiT-style architectures [2605.30263]. It also supports adapting existing video world models such as HY-WorldPlay to new data distributions, training recipes, and latency targets [2605.30263].

For Wan2.1-T2V-1.3B, the backbone is a cross-attention-heavy U-Net with approximately $1.3$ B parameters, latent resolution $480\times 832$, $77$ frames total, and chunk size $4$ latents [2605.30263]. The listed training recipe is bidirectional fine-tune $5$K steps, Stage 1 AR $4$K steps, Stage 2 $2$K steps, and Stage 3 $200$ steps [2605.30263]. For HY1.5-TI2V-8B, the backbone is MMDiT-style with approximately $8$ B parameters and the recipe is fine-tune $8$K, Stage 1 $4$K, Stage 2 $1.5$K, and Stage 3 $500$ steps [2605.30263].

The framework allows control signals such as pose, depth, and segmentation to be swapped by replacing PRoPE with another positional encoding such as DepthPE [2605.30263]. Latency targets are configured through the number of few-step timesteps $K$, chunk size, and distillation hyperparameters, and a recipe JSON is described as specifying backbone, data, $K$, and training-stage hyperparameters [2605.30263]. An example YAML snippet uses:

```yaml
backbone: "wan2.1-t2v-1.3b"
data: {type:"worldplay", resolution:[480,832], frames:77}
camera_pe: "prope"
distill: {steps_AR:4000, steps_ODE:2000, steps_DMD:200, K:4}
```
[2605.30263]

The project also releases runnable scripts, checkpoints, documentation, and inference code, with example training commands such as `train_bidirectional.py`, `train_ar_stage1.py`, `train_distill_ode.py`, and `train_distill_dmd.py`, and inference through `demo_interactive.py` [2605.30263]. These details indicate that minWM is intended as a reproducible research framework rather than only a conceptual model family.

## 6. Performance, ablations, and operational characteristics

The open-source framework reports a few-step AR model using only $K=4$ timesteps per chunk of $4$ latent frames [2605.30263]. End-to-end first-frame latency on A800, excluding VAE encode/decode, is reported as $3.45$ s for HY1.5 and $1.14$ s for Wan2.1, corresponding to $223.8\times$ and $236.6\times$ speed-up versus bidirectional baselines of $771$ s and $269$ s, respectively [2605.30263]. Throughput at batch $=8$ is stated as $20$–$30$ fps on A800 for 4-step AR [2605.30263].

The Causal Forcing++ action-conditioned presentation reports frame-wise 2-step minWM performance as $14.1$ FPS on A800, compared with $10.4$ FPS for 4-step chunk-wise baselines, and first-frame latency of $0.27$ s versus $0.60$ s, described as a $50\%$ reduction [2605.15141]. It reports VBench Total $84.14$, Quality $84.89$, Semantic $81.13$, VisionReward $6.661$, and Instruction-follow $51$, compared with VisionReward $6.326$ and Instruction-follow $56$ for Causal Forcing [2605.15141]. The same source states that Stage 2 cost drops from $11\,600$ GPU·h plus $1\,900$ GiB storage for causal ODE initialization to $2\,900$ GPU·h and $0$ GiB for causal consistency initialization, approximately a $4\times$ speedup with no extra disk [2605.15141]. Its ablation states that across 1-, 2-, and 4-step settings, causal CD is at least as good as causal ODE and better than causal DMD, multi-step AR initialization, and Self-Forcing ODE [2605.15141].

The framework paper also provides practical ablations for controllability [2605.30263]. For data quality, SpatialVid with perception-estimated poses is described as yielding unreliable controllability, while DL3DV with re-rendering and OpenVid with WorldPlay trajectories are described as reliable or strong [2605.30263]. For the number of fine-tuning steps on HY1.5, $1$–$2$K steps show no controllability, about $5$K steps show camera response that is unstable, and at least $8$K steps yield stable high-fidelity controllability [2605.30263]. For minimal batch size on Wan2.1, batch $<4$ fails to learn control, batch $=8$ gives partial unstable control, and batch $\ge 16$ yields stable full controllability [2605.30263]. A quantitative metric is the percentage of held-out videos where generated camera-trajectory error is at most $3^\circ$ average: $12\%$ for batch $=4$, $57\%$ for batch $=8$, and $92\%$ for batch $=16$ [2605.30263].

A concise comparison of selected reported characteristics is useful:

| Aspect | minWM framework | Action-conditioned minWM / Causal Forcing++ |
|---|---|---|
| Backbone examples | Wan2.1-T2V-1.3B, HY1.5-TI2V-8B | Wan2.1-1.3B, critics based on Wan2.1-14B |
| Few-step regime | $K=4$ timesteps per chunk of 4 latent frames | frame-wise 2-step setting |
| First-frame latency | 1.14 s (Wan2.1), 3.45 s (HY1.5) on A800, excluding VAE encode/decode | 0.27 s vs. 0.60 s baseline on A800 |
| Stage 2 distillation cost | causal ODE or causal consistency | $11\,600$ GPU·h + $1\,900$ GiB to $2\,900$ GPU·h + $0$ GiB |

These numbers come from different experimental settings and should not be merged into a single unified benchmark. They nevertheless show that minWM is consistently framed around latency reduction and few-step rollout.

## 7. Limitations, related developments, and interpretation

The supplied materials identify several limitations. In the action-conditioned Causal Forcing++ account, the action-conditioned results are shown only under a chunk-wise 4-step setup, and extending them to fully frame-wise 1–2 steps is explicitly described as future work [2605.15141]. The same source notes that long-horizon consistency still degrades gradually and suggests memory modules or hierarchical schemes as potential remedies [2605.15141]. In the framework paper, chunk size is fixed, and dynamic chunking or memory networks such as WorldCompass are listed as future directions [2605.30263].

A later comparison with BiWM frames minWM as a four-stage causal pipeline whose limitations include error accumulation, long-horizon quality degradation, camera-control drift, and training/inference overhead from per-layer camera encoder and residual PRoPE modules [2606.10135]. According to that comparison, once generated frames are frozen into the key/value cache, mismatches become permanent and can lead to washed-out geometry, desaturated colors, and vanishing object details over longer rollouts [2606.10135]. It further contrasts minWM’s four-stage recipe with a two-stage bidirectional autoregressive alternative [2606.10135]. Because these claims are presented from the perspective of a subsequent system, they should be read as a comparative characterization rather than as the original minWM authors’ own summary.

At the same time, the BiWM comparison provides useful context for situating minWM within the emerging design space of interactive world models. It identifies minWM as representative of a *fully causal* approach in which control fine-tuning, autoregressive training, causal initialization, and few-step distillation are performed sequentially [2606.10135]. This suggests that minWM occupies an important position in the transition from bidirectional video diffusion to interactive world modeling: it formalizes a reproducible causal pipeline, but also exposes the trade-offs of strict causality, especially for long-horizon fidelity and controllability.

A plausible implication is that minWM’s lasting significance lies less in a single architecture than in its codification of a modular recipe. The framework demonstrates that open bidirectional T2V and TI2V models can be converted into controllable world models through a structured stack of PRoPE-based control injection, autoregressive training, causal or consistency-based initialization, and DMD refinement [2605.30263]. The Causal Forcing++ extension further suggests that the most computationally expensive part of this stack—few-step initialization—can be substantially simplified by replacing offline PF-ODE trajectory storage with online local consistency supervision [2605.15141].

Source: https://www.emergentmind.com/topics/minwm