Motif-Video 2B: Efficient Text-to-Video Diffusion

• Motif-Video 2B is a 2B-parameter text-to-video diffusion transformer designed to optimize prompt alignment, temporal consistency, and high-frequency visual details within a micro-budget regime.
• Its architecture features a three-stage backbone—dual-stream for early fusion, single-stream for joint representation, and a DDT decoder for detail refinement—demonstrating efficiency with 7× fewer parameters than larger models.
• The model leverages innovations like shared cross-attention, dynamic token routing (TREAD), and a progressive curriculum to match or exceed benchmark scores on VBench while minimizing compute and data requirements.

Searching arXiv for the cited papers to ground the article and confirm metadata. Motif-Video 2B is a 2B-parameter text-to-video diffusion transformer developed to test the claim that strong video generation quality need not come only from brute-force scaling in model size, data volume, and training compute, but can instead come from organizing capacity more carefully (Lim et al., 14 Apr 2026). The model is presented as a micro-budget video model trained on fewer than 10M clips and under 100,000 H200 GPU hours, yet it reaches 83.76% on VBench, surpassing Wan2.1-T2V-14B at 83.69% while using 7× fewer parameters. Its central thesis is that video generation is a three-way optimization problem involving prompt alignment, temporal consistency across frames, and high-frequency visual detail, and that these objectives interfere with one another when they are forced through a single pathway (Lim et al., 14 Apr 2026).

1. Positioning and design thesis

The report frames Motif-Video 2B as an explicit test of capacity organization in video generation. Rather than arguing that single-stream video transformers cannot work, it argues that, under a small compute and data budget, specialization matters more. The intended contribution is therefore not merely that a small model performs well, but that architectural specialization plus efficiency-oriented training can narrow or even exceed the benchmark gap usually associated with much larger models (Lim et al., 14 Apr 2026).

That thesis is unusually concrete in the paper’s setup. The training system used 64 H200 GPUs on Azure, arranged as 8 nodes × 8 GPUs, with training orchestrated via Kubernetes and SkyPilot. The model is positioned relative to contemporary open video generators trained on hundreds of millions of clips and 5B–14B parameters, but the claim is not that scale is irrelevant. A more precise reading is that the report treats scale and organization as separable variables, and argues that better role separation can partly substitute for brute-force scale when the budget is constrained.

A common misconception is to interpret the benchmark result as a claim of unconditional superiority over larger models. The paper does not make that claim. It repeatedly narrows the conclusion to the idea that careful specialization can be highly competitive, especially on semantic and compositional axes, without erasing the remaining gap in human-perceived fidelity.

2. Backbone structure and role specialization

Motif-Video 2B takes compressed video latents from the Wan2.1 VAE, which applies 8×8 spatial and 4× temporal compression, and patchifies them with a 2×2×1 kernel (Lim et al., 14 Apr 2026). Text is encoded with T5Gemma2, chosen because the authors prefer an encoder-decoder text encoder for bidirectional contextual representations in visual generation.

The backbone has 28 encoder layers plus 8 decoder layers, all with QK normalization, 12 attention heads of dimension 128, and hidden size 1536. Its defining structural feature is a three-part decomposition:

• 12 dual-stream layers for early modality fusion
• 16 single-stream layers for joint text-video representation learning
• 8 DDT decoder layers for detail refinement

The first stage preserves separate text and video streams, each with its own self-attention, and exchanges information through cross-attention. The stated purpose is early fusion without entanglement: text and video begin interacting while retaining modality-specific processing. This is inherited from FLUX-style dual-stream multimodal transformers.

The second stage merges the streams into a single joint sequence. These 16 single-stream blocks are where most cross-modal integration occurs. The report treats this as the locus of compositional text-conditioned generation, but also as the stage where token imbalance becomes most damaging because video token count grows while text token count stays fixed.

The third stage uses DDT (Decoupled Diffusion Transformer) as a decoder over the 28-layer encoder stack. The paper’s interpretation is that diffusion transformers face an optimization conflict when the same layers must both encode low-frequency semantics and decode high-frequency details. The DDT decoder is therefore assigned the downstream role of detail reconstruction and temporal cleanup. Attention heatmaps are reported to show clear inter-frame attention structure in the DDT decoder, with each frame attending more strongly to temporally adjacent frames than in the single-stream encoder blocks. The paper is careful to note that whether this stronger structure comes from DDT specifically or just depth remains an open question (Lim et al., 14 Apr 2026).

In the authors’ decomposition, the backbone assigns three responsibilities to different regions of the network: early layers stabilize text and video features before full mixing, middle layers construct the joint multimodal representation, and late decoder layers focus on detail and temporal coherence. This suggests a deliberately modular answer to interference among prompt alignment, temporal consistency, and fine-detail recovery.

3. Shared Cross-Attention and the text-dilution diagnosis

One of the report’s central technical claims is that prompt alignment degrades in long-sequence video transformers because text tokens are structurally diluted inside a joint self-attention denominator. For a video query token $i$ attending to text token $j$, the paper writes

$$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$

Here $\mathcal{V}$ is the set of video tokens and $\mathcal{T}$ the set of text tokens. Since $|\mathcal{V}| \gg |\mathcal{T}|$, the report describes a structural dilution effect: text influence is mathematically diluted by competition with many more video tokens (Lim et al., 14 Apr 2026). The paper states that prompt-following degrades when scaling to 720p.

Motif-Video 2B’s answer is Shared Cross-Attention, attached after self-attention in each single-stream block. Let $\mathbf{h}_v$ be the post-self-attention video hidden state and let $\mathbf{x}_{\mathrm{txt}}$ be the text hidden states entering that block’s self-attention. Shared Cross-Attention is defined as

$$\mathbf{Q} = W_Q^{\mathrm{cross}} \mathbf{h}_v,\qquad \mathbf{K} = W_K \mathbf{x}_{\mathrm{txt}},\qquad \mathbf{V} = W_V \mathbf{x}_{\mathrm{txt}},$$

$$\mathbf{h}_v \leftarrow \mathbf{h}_v + W_O^{\mathrm{cross}} \cdot \mathrm{Attn}(\mathbf{Q}, \mathbf{K}, \mathbf{V}).$$

Only $j$0 and $j$1 are new parameters. The key and value projections are shared with the enclosing self-attention block, so the text-side geometry used by cross-attention is exactly the one self-attention already learned. The implementation reuses the same $j$2 tensors already computed inside self-attention, adding zero key/value projection FLOPs. $j$3 is zero-initialized, so the block is functionally identical to the base model at initialization and the additional path grows gradually during training (Lim et al., 14 Apr 2026).

The paper distinguishes this mechanism from both standard cross-attention and fully joint single-stream attention. Compared with standard cross-attention, it is residual, post-self-attention, and shares text-side $j$4 with the enclosing self-attention block. Compared with single-stream attention, it creates a dedicated text-conditioning path that does not compete with video tokens in a joint softmax denominator.

The report’s interpretation is that this realizes sequential refinement: after self-attention has already aggregated local spatiotemporal structure, the model asks a second question of the text conditioned on that refined state. Empirically, the paper reports four forms of evidence. First, single-stream intermediate layers allocate substantially less attention mass to text tokens than dual-stream ones. Second, when inserted into a pretrained checkpoint and trained for 1,000 steps, a SkyReels-V4-style raw-$j$5 variant collapses, whereas Shared Cross-Attention remains stable. Third, during inference across all 16 single-stream blocks and 50 denoising steps at 1280×736, 121 frames, and CFG 8, the cross-attention residual has a global mean magnitude equal to 7.6% of the self-attention residual, with a maximum of 21.7%, no block below 5.2%, and average cosine similarity about $j$6. Fourth, disabling the module at runtime produces qualitative failures including action reversal, noun dropping, and object merging (Lim et al., 14 Apr 2026).

4. Training objective, curriculum, and efficiency mechanisms

Training uses rectified flow matching. Given a clean sample $j$7 and Gaussian noise $j$8, the interpolation is

$j$9

and the model predicts the velocity field

$$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$0

with loss

$$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$1

Classifier-free guidance training uses prompt dropout probability $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$2. The report states that there are no modifications to the noise schedule or loss weighting relative to the conventional setup, although appendices add resolution-dependent timestep sampling details (Lim et al., 14 Apr 2026).

The curriculum is progressive. It begins with text-to-image at 144p, 1 frame, then proceeds through joint image-video and progressively higher spatiotemporal regimes: 144p, 33 frames; 144p, 65 frames; 360p, 65 frames; 480p, 65 frames; 480p, 121 frames; a 480p supervised fine-tuning stage; 720p, 121 frames; then back to 360p, 121 frames with Shared Cross-Attention added; and finally 720p, 121 frames again for final pretraining plus SFT. The switch to T5Gemma2 occurs at the 360p stage. Shared Cross-Attention is introduced at Stage 9 after semantic degradation was observed at 720p.

Two efficiency techniques are central. The first is dynamic token routing via TREAD. In Motif-Video 2B, routing is applied from layer 4 to layer 25 with a token drop ratio of 0.5. The report excludes layers 1–3 because they are early dual-stream blocks and layers 26–36 because they are DDT decoder layers where token dropping was more likely to hurt high-frequency detail. At 720p, 121 frames, and 512 text tokens, the full transformer requires about 4,913 TFLOPs per forward pass; with TREAD, this falls to about 3,563 TFLOPs, a 27.5% reduction with estimated 1.38× speedup and measured 1.31× throughput increase in videos/sec. Routing is disabled at inference (Lim et al., 14 Apr 2026).

The second technique is early-phase feature alignment to a frozen pretrained video encoder, adapting REPA to video. The teacher is V-JEPA 2.0. Alignment is applied only in Stages 1–3 and disabled from Stage 4 (360p) onward. The report presents this as phase-constrained alignment: early in training, a frozen visual encoder accelerates convergence, but later it becomes a bottleneck that restricts adaptation to the target distribution. The aligned layer is layer 8 of the encoder; following iREPA, the projection is a 3×3 Conv2D with spatial normalization, and student and teacher features are aligned with trilinear interpolation. The REPA loss weight $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$3 is set between 0.1 and 0.5 across early stages (Lim et al., 14 Apr 2026).

Several implementation details are specified: AdamW with $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$4, $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$5, $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$6, and weight decay 0.0; global gradient clipping at 1.0; bfloat16 mixed precision with communication and optimizer-sensitive quantities in float32. For VBench, sampling uses 1280×736, 121 frames, 24 fps, the Video APG sampler with momentum 0, $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$7, $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$8, CFG scale 8, and a fixed negative prompt (Lim et al., 14 Apr 2026).

5. Data construction, captioning, and distributed systems

The report repeatedly states that the model is trained on fewer than 10M clips. It also mentions a 2.8M-clip proprietary collection over which a learned preference model is used, alongside public datasets. Data construction combines an internal web crawl with public image and video datasets, then applies sanitation for broken files, tiny files, near-duplicates, NSFW and watermark filtering, black-bar cropping, scene boundary detection with conservative over-segmentation and SigLIP-based stitch detection, and discarding of clips under 2 seconds (Lim et al., 14 Apr 2026).

Quality filtering includes aesthetic scoring with Aesthetic Predictor V2.5, luminance filtering, a model-based suitability score inspired by Koala-36M, technical quality filtering with DOVER, and motion filtering using optical flow statistics from UniMatch. Admission criteria are progressively tightened as training moves from 144p to 360p to 480p to 720p. The final 720p SFT stage adds domain balancing and a dynamic-motion criterion, and synthetic video is injected only at 720p.

Deduplication uses an SSCD-based pipeline with a 512-dimensional descriptor, tenth frame as representative for videos, cuVS IVF-PQ retrieval with $$\alpha_{ij} = \frac{ \exp\!\left(q_i^\top k_j / \sqrt{d}\right) }{ \displaystyle \sum_{v \in \mathcal{V}} \exp\!\left(q_i^\top k_v / \sqrt{d}\right) \;+\; \sum_{t \in \mathcal{T}} \exp\!\left(q_i^\top k_t / \sqrt{d}\right) }.$$9, nprobe=16, and cosine threshold 0.9. Within duplicate groups, the retained sample maximizes the weighted score

$\mathcal{V}$0

Captioning is tightly integrated into training. Using Qwen3-VL-30B-A3B, each clip receives structured JSON containing free-text descriptions and tags such as subject, style, action, camera motion, quality, watermark, and NSFW status. Three caption variants are retained: caption_long with 150–250 words, caption_short with 15–25 words, and caption_truncated as the leading sentence of the long caption. During training, these are sampled with probabilities $\mathcal{V}$1. The stated purpose is to reduce train-test mismatch because user prompts at inference are typically shorter than VLM-generated training captions (Lim et al., 14 Apr 2026).

On the systems side, the paper introduces an offline bucket-balanced sampler for WebDataset. The problem is load imbalance across distributed workers when frame counts and resolutions vary. The report formalizes per-bucket progress as

$\mathcal{V}$2

and optimizes shard assignment to minimize the coefficient of variation of per-rank bucket counts:

$\mathcal{V}$3

With simulated annealing over 30,000 iterations, throughput per epoch improves from baseline $\mathcal{V}$4 at about 20% utilization to about 5.4N at about 90% utilization, and dataloader latency drops from 0.05 s/step to <0.001 s/step. This is presented as an enabling systems contribution for the micro-budget training regime (Lim et al., 14 Apr 2026).

6. Empirical performance, interpretation, and limitations

The main quantitative evidence comes from VBench. Motif-Video 2B achieves Total 83.76, Quality 84.59, and Semantic 80.44. Against Wan2.1-T2V-14B, the corresponding scores are 83.69, 85.59, and 76.11 (Lim et al., 14 Apr 2026).

Model	VBench Total	Semantic
Motif-Video 2B	83.76	80.44
Wan2.1-T2V-14B	83.69	76.11
Wan2.1-T2V-1.3B	83.31	75.65

The detailed VBench dimensions reported for Motif-Video 2B are: Subject Consistency 95.38, Background Consistency 95.74, Temporal Flickering 98.16, Motion Smoothness 98.81, Dynamic Degree 65.56, Aesthetic Quality 65.95, Image Quality 70.50, Object Class 92.93, Multiple Objects 77.29, Human Action 95.60, Color 87.94, Spatial Relationship 83.02, Scene 52.40, Appearance Style 23.03, Temporal Style 25.24, and Overall Consistency 26.78. The paper emphasizes gains on the semantic side, especially Spatial Relationship (83.02), Object Class (92.93), and Multiple Objects (77.29), treating this as consistent with the model’s specialization for prompt grounding and compositional control (Lim et al., 14 Apr 2026).

Relative to Wan2.1-T2V-1.3B, Motif-Video 2B leads on total score 83.76 vs. 83.31 and semantic score 80.44 vs. 75.65, although Wan2.1-1.3B is stronger on Subject Consistency 97.56 vs. 95.38 and Temporal Flickering 99.55 vs. 98.16. Relative to SANA-Video 2B, the paper reports a slight lead on total score 83.76 vs. 83.71, but notes that SANA’s full per-dimension numbers are not publicly reported.

The report explicitly cautions against over-interpreting the benchmark win. In a blind pairwise human study over 40 prompts, converted to ELO, Motif-Video 2B obtains Total ELO 1026.2, Prompt-following 1015.1, and Video-fidelity 1047.3. This is below Wan2.1-14B, which scores 1114.8, 1054.4, and 1101.6, and also below HunyuanVideo-1.5 and Wan2.2 in total ELO, though ahead of Wan2.1-1.3B, SANA-Video, and CogVideoX. The authors attribute the discrepancy partly to VBench’s uniform weighting across 16 dimensions and partly to the fact that semantic correctness can remain high even when humans notice anatomical or temporal flaws (Lim et al., 14 Apr 2026).

This distinction matters for interpreting the contribution. The strongest claim supported by the paper is not perceptual parity with the best larger models, but unusually strong prompt adherence and compositional semantics at low budget. The report also lists clear failure modes: bad hand anatomy, broken body structure under high motion, attribute leakage between similar co-present animals, physically implausible dynamics for liquids or collisions, coherence loss in dense multi-agent scenes, and unintended mid-clip scene transitions. Some of these are described as primarily data coverage limitations, especially anatomy, high-displacement motion, and multi-subject disambiguation; others are presented as potentially capacity-bound, especially complex-scene temporal consistency. The authors suggest decoder-side scaling as a natural next experiment because their attention analysis indicates temporal structure concentrates in the decoder (Lim et al., 14 Apr 2026).

A further caveat is methodological. The report does not provide full controlled ablation tables for Shared Cross-Attention, the three-part backbone, DDT, TREAD, or REPA. It explicitly asks that the results be interpreted as evidence that the composed recipe works, not as isolated marginal gains for each component.

7. Relation to similarly named “MoTIF” work

Despite the lexical overlap, Motif-Video 2B is distinct from several arXiv papers with similar names. “MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution” is a continuous space-time video super-resolution method based on local implicit neural functions, explicit forward optical flow, and reliability-aware splatting; it is not a text-to-video generative model, and it is approximately a 12.55 million parameter restoration network rather than a 2B-parameter diffusion transformer (Chen et al., 2023).

It is also distinct from “Concepts in Motion: Temporal Bottlenecks for Interpretable Video Classification,” which introduces MoTIF (Moving Temporal Interpretable Framework) as a concept bottleneck model for video classification. That work is centered on concept-isolated temporal self-attention, LSE pooling, and intrinsic interpretability in action recognition, not text-conditioned video synthesis or latent video diffusion (Knab et al., 25 Sep 2025).

A third nearby term appears in “Action Motifs: Self-Supervised Hierarchical Representation of Human Body Movements,” where Action Motifs denote reusable temporal compositions of Action Atoms in a self-supervised pose-based human behavior model. That work is conceptually relevant to motif-based temporal representation learning, but its core input modality is 3D pose sequences, not raw RGB video generation, and its method is a nested latent Transformer for human motion analysis rather than an open-domain text-to-video generator (Kinoshita et al., 30 Apr 2026).

The naming overlap can therefore be misleading. In the context of generative video modeling, Motif-Video 2B specifically denotes the role-specialized text-to-video flow-matching transformer described in the 2026 technical report, whose key innovations are Shared Cross-Attention, a three-stage backbone separating early fusion, joint multimodal reasoning, and detail/temporal refinement, and an efficiency-oriented training stack combining REPA, TREAD, progressive scaling, curated data filtering, and distributed sampling under a micro-budget regime (Lim et al., 14 Apr 2026).