Video-Mirai: Foresight in Video Diffusion

• Video-Mirai is a training-only foresight objective that adjusts causal representations to incorporate future information for enhanced long-horizon consistency.
• It employs a distinct training pipeline using a causal generator, a frozen bidirectional foresight encoder, and a lightweight predictor to align internal states with future segments.
• Empirical results show improved 5-second VBench scores and significant gains in subject and background consistency during long-horizon rollouts.

Video-Mirai is a training-only foresight objective for causal or streaming autoregressive video diffusion models. It alters representation supervision rather than inference, so a generator still rolls out segment by segment under causal masking and temporal KV-cache constraints, but its internal states are trained to retain information that later segments will need for identity, layout, and motion consistency. In the reported implementation, Video-Mirai is applied to a strong Causal-Forcing baseline distilled from a bidirectional teacher into a causal student, improving 5-second VBench Total Score from 83.82 to 84.62 and, on 30-second rollouts beyond the training horizon, improving subject consistency from 84.93 to 88.47 and background consistency from 90.22 to 91.94 (Yu et al., 2 Jun 2026).

1. Problem formulation and the representation-level planning gap

Video-Mirai is motivated by a specific failure mode of causal video generation: a model that predicts each present segment plausibly from the past may still construct internal states that are insufficient for maintaining future consistency. The underlying causal factorization is

$$p_\theta(\mathbf{x})=\prod_{i=1}^{N} p_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i}),$$

where a video is decomposed into temporal segments $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$. Standard training applies a present-segment loss

$$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$

which constrains output quality for the current segment but does not directly constrain the internal causal state $\mathbf{h}_i^L$ to preserve future-relevant information such as subject identity, scene layout, or motion phase (Yu et al., 2 Jun 2026).

This mismatch is termed the representation-level planning gap. The central claim is not that causal inference is flawed, but that causal states are under-supervised when they are trained only to explain the present. A state that is sufficient for a locally plausible current chunk may still discard information required by later chunks. The practical consequence is long-horizon drift: subject identity changes, backgrounds mutate, and motion phases reset or teleport.

A recurrent misconception is that any future-conditioned signal necessarily violates causality. Video-Mirai rejects that premise. Future frames are used only to define training targets for representations; they are never used as generator inputs during causal rollout. This distinction between causal inference and non-causal representation supervision is the method’s organizing principle.

2. Architecture and foresight supervision mechanism

Video-Mirai introduces three components during training: a causal generator $G_\theta$, a frozen foresight encoder $E_\phi$, and a lightweight predictor $\phi_\omega$. The causal generator is the deployed model; the encoder and predictor are discarded after training.

Component	Function	Inference status
Causal generator $G_\theta$	Rolls out segments causally and produces hidden states $\mathbf{h}_i^L$	Kept
Foresight encoder $E_\phi$	Reads the completed rollout non-causally and produces future-informed targets $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$0	Discarded
Predictor $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$1	Maps causal hidden states into the encoder feature space	Discarded

The training pipeline proceeds in four stages. First, the generator performs a standard causal rollout, producing segments $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$2 and storing a mid-depth hidden state $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$3 for each segment. Second, the frozen bidirectional foresight encoder processes the completed clean rollout non-causally and yields $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$4, whose segment-level features already contain future information because temporal attention is bidirectional. Third, Video-Mirai forms a fused future-aware target over a foresight window $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$5: $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$6 Fourth, the predictor aligns the causal state to this stopped-gradient target through cosine similarity: $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$7 and the sequence loss is

$\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$8

The use of $\mathbf{x}=\{\mathbf{X}_1,\dots,\mathbf{X}_N\}$9 is essential. The target does not backpropagate into the frozen encoder and does not create a training path by which future frames become generator inputs. Gradients flow only into the predictor and the causal generator’s representation $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$0. In this sense, Video-Mirai distills hindsight into foresight: the encoder observes the whole rollout, but only the causal state is updated.

3. Integration with causal autoregressive diffusion training

Video-Mirai is attached to an existing autoregressive video diffusion distillation pipeline rather than replacing it. In the reported system, the baseline is Causal-Forcing: a causally correct distillation recipe in which a bidirectional Wan-14B teacher is distilled into a causal Wan-1.3B student through autoregressive rollout and asymmetric Distribution Matching Distillation. Video-Mirai adds its foresight term to the existing generation objective: $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$1 with default $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$2 (Yu et al., 2 Jun 2026).

The concrete implementation choices are tightly specified. The causal student is a Wan2.1-T2V-1.3B DiT with 30 transformer blocks, hidden dimension 1536, FFN dimension 8960, 12 heads, QK-normalization, and causal temporal attention. The foresight encoder is a frozen Wan2.1-T2V-14B with 40 blocks, hidden dimension 5120, FFN dimension 13824, 40 heads, and bidirectional temporal attention. The student’s layer $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$3 is aligned with the encoder’s layer $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$4, matching relative depth $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$5. The predictor is itself a small DiT-style network with 3 blocks, hidden dimension 1536, FFN ratio 4, 12 heads, and a final linear projection to 5120 dimensions.

The default foresight window in the chunk-wise setting is $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$6, meaning that the target for the current causal state averages the encoder features of the current and immediately following chunk. This is not equivalent to a current-only target, because even $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$7 is already future-informed by bidirectional attention. Training overhead is incurred only during optimization: the paper reports roughly $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$8 per-cycle FLOP overhead and about $$\mathcal{L}_{\text{present}}(\theta)=\mathbb{E}_{\mathbf{x}}\sum_{i=1}^{N}\ell\!\left(G_\theta(\mathbf{X}_i \mid \mathbf{X}_{<i})\right),$$9 wall-clock overhead, while inference retains the original architecture, per-step FLOPs, and KV-cache behavior (Yu et al., 2 Jun 2026).

4. Empirical performance and ablation results

On the 5-second VBench protocol, Video-Mirai raises the chunk-wise Causal-Forcing baseline from Quality 84.54, Semantic 80.93, and Total 83.82 to Quality 85.38, Semantic 81.59, and Total 84.62. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.93 to 88.47, background consistency from 90.22 to 91.94, and overall consistency from 24.93 to 25.03 (Yu et al., 2 Jun 2026).

The method also transfers across training regimes and segmentation granularities. In the chunk-wise Self-Forcing setting, Total Score improves from 83.67 to 84.15. In the frame-wise Causal-Forcing setting, Total Score improves from 82.27 to 83.60. These results support the paper’s claim that the foresight objective targets a representation problem rather than a specific rollout protocol.

The ablations identify several dominant ingredients. For the foresight window, $\mathbf{h}_i^L$0 is सर्वोत्तम in the chunk-wise case, outperforming $\mathbf{h}_i^L$1, $\mathbf{h}_i^L$2, and $\mathbf{h}_i^L$3. For predictor design, a fused-target 3-block DiT is better than MLP variants and per-offset predictor variants, indicating that token mixing and single-target fusion are more effective than separate offset heads. For depth selection, matched mid-depth supervision at 15→20 is better than shallower 9→12 or deeper 24→32 alignment. For encoder choice, a frozen Wan-14B outperforms a bidirectional Wan-1.3B and an EMA causal teacher, which the paper attributes to both bidirectionality and higher capacity. For loss choice, cosine similarity outperforms MSE and SIGReg-based alternatives.

Stage placement is more subtle. The foresight loss improves results when attached at Stage 2 or Stage 3, but naively combining both does not stack gains, because Stage 2 and Stage 3 supervise different futures—ground-truth future versus rollout future. The recommended practical recipe is Stage-3-only attachment during the final 100 DMD steps.

5. Probing, interpretation, and common misconceptions

Video-Mirai’s central empirical interpretation is that future information becomes more decodable from current causal features. The paper evaluates this with probes: a small MLP is trained, after generator training is complete, to reconstruct future RGB frames from frozen hidden states $\mathbf{h}_i^L$4. Across layers and prediction horizons, Video-Mirai yields better PSNR, lower MSE, and lower LPIPS than the baseline, indicating that its causal states preserve more future-relevant structure (Yu et al., 2 Jun 2026).

Qualitative probe visualizations reinforce the point. Baseline readouts remain close to the current frame, whereas Video-Mirai readouts more faithfully approximate future appearance. The paper further reports that these readouts resemble the average over multiple stochastic future rollouts, which suggests that the learned representation does not collapse to a single deterministic continuation. A plausible implication is that the causal state is being shaped into a distributional plan over near futures rather than a brittle next-frame template.

Two misconceptions are explicitly addressed by the method’s design. The first is that future-conditioned supervision breaks causality. It does not: future frames supervise representations, never generator inputs, and the encoder targets are stopped-gradient. The second is that better long-horizon consistency must come from a heavier inference stack. It does not here: the foresight encoder and predictor are training-only, so the deployed generator has unchanged architecture, unchanged per-step FLOPs, and unchanged KV-cache behavior.

6. Position within long-horizon video generation research

Video-Mirai occupies a specific niche within the broader study of long-horizon video generation. It does not redesign factorization, planner structure, or inference-time control; instead, it changes how causal states are trained. This distinguishes it from several neighboring lines of work.

VideoMAR is a decoder-only autoregressive video generator over continuous tokens that emphasizes temporal causality and spatial bi-directionality, next-frame diffusion loss, temporal short-to-long curriculum learning, and progressive temperature scheduling (Yu et al., 17 Jun 2025). MiLA addresses long-horizon multi-view driving video generation through a Coarse-to-(Re)fine pipeline, Joint Denoising and Correcting Flow, and Temporal Progressive Denoising (Wang et al., 20 Mar 2025). MIND-V pursues long-horizon robotic manipulation synthesis through hierarchical planning, Staged Visual Future Rollouts, and GRPO post-training with a Physical Foresight Coherence reward derived from V-JEPA2 (Zhang et al., 7 Dec 2025). Against that backdrop, Video-Mirai is notable for leaving causal inference untouched and localizing foresight entirely to training-time representation alignment (Yu et al., 2 Jun 2026).

Its limitations are correspondingly narrow but clear. The default foresight window is short, the added supervision incurs training-time overhead, and the main empirical setting is text-to-video at 832×480 with evaluation up to 30 seconds. Even so, the method establishes a broader point: for causal video diffusion, local output correctness is not sufficient supervision for long-horizon consistency. The paper’s formulation recasts future data as a supervisory signal for present hidden states, arguing that visual autoregressive models require not only causality, but foresight.