Motif-Video 2B: Technical Report

Abstract: Training strong video generation models usually requires massive datasets, large parameter counts, and substantial compute. In this work, we ask whether strong text-to-video quality is possible at a much smaller budget: fewer than 10M clips and less than 100,000 H200 GPU hours. Our core claim is that part of the answer lies in how model capacity is organized, not only in how much of it is used. In video generation, prompt alignment, temporal consistency, and fine-detail recovery can interfere with one another when they are handled through the same pathway. Motif-Video 2B addresses this by separating these roles architecturally, rather than relying on scale alone. The model combines two key ideas. First, Shared Cross-Attention strengthens text control when video token sequences become long. Second, a three-part backbone separates early fusion, joint representation learning, and detail refinement. To make this design effective under a limited compute budget, we pair it with an efficient training recipe based on dynamic token routing and early-phase feature alignment to a frozen pretrained video encoder. Our analysis shows that later blocks develop clearer cross-frame attention structure than standard single-stream baselines. On VBench, Motif-Video~2B reaches 83.76\%, surpassing Wan2.1 14B while using 7$\times$ fewer parameters and substantially less training data. These results suggest that careful architectural specialization, combined with an efficiency-oriented training recipe, can narrow or exceed the quality gap typically associated with much larger video models.

• The paper presents a micro-budget text-to-video model that leverages a 2B parameter architecture with targeted efficiency measures to match state-of-the-art compositional metrics.
• Its unique three-stage backbone and Shared Cross-Attention mechanism overcome challenges in prompt fidelity and temporal coherence while optimizing GPU usage.
• The efficiency-oriented training regime, incorporating dynamic token routing and bucket-balanced sampling, enables high-quality video generation with substantially reduced compute.

Motif-Video 2B: A Micro-Budget Architecture and Training Recipe for Text-to-Video Generation

Introduction and Motivation

Motif-Video 2B addresses the question of whether state-of-the-art text-to-video generation performance can be achieved without resorting to massive datasets and extreme compute budgets. Existing models such as Wan2.1, HunyuanVideo, and Seedance are typically trained with hundreds of millions of clips and parameter counts upward of 14B, restricting participation to organizations with exceptional resources. Motif-Video 2B demonstrates that, through careful architectural role separation and efficient training practices, a model with 2B parameters trained on fewer than 10M clips and ~100,000 H200 GPU hours can match or surpass prior open models in compositional video generation metrics.

The central hypothesis of this work is that, in video diffusion transformers, scaling parameter count and data does not directly resolve the interference between prompt following, temporal consistency, and fine-detail recovery. Instead, architectural specialization and targeted efficiency strategies are necessary for strong multimodal alignment.

Motif-Video 2B introduces two core ideas:

1. Shared Cross-Attention: A residual cross-attention structure that stabilizes text conditioning in long video token sequences by reusing self-attention key and value projections, solved the observed degradation of text grounding due to token imbalance.
2. Three-Part Backbone: Architectural decomposition into dual-stream, single-stream, and decoupled decoder layers allows explicit allocation of capacity to modality fusion, joint representation learning, and visual detail synthesis.

This is supported by a training pipeline leveraging dynamic token routing (TREAD) and phase-constrained representation alignment (REPA with a V-JEPA teacher). The approach results in a model achieving 83.76% on VBench, outscoring Wan2.1-14B while using 7× fewer parameters.

Figure 1: Representative generations from Motif-Video 2B, illustrating prompt fidelity, temporal coherence, and fine visual detail in multi-domain prompts.

Architecture

Three-Stage Model Backbone

Motif-Video 2B employs an explicit functional decomposition for transformer-based diffusion:

• Dual-Stream Fusion (Early Layers): 12 layers maintain separate text and video streams before cross-modal mixing, adapted from FLUX's principle of delayed feature entanglement. This is crucial to avoid degradation from early statistical mismatches.
• Single-Stream Joint Representation: 16 layers process concatenated token sequences from both modalities, supporting higher-order text-video interactions. However, attention analysis reveals text token sparsity becomes problematic as video sequence length grows.
• Decoupled Decoder (DDT) Layers: 8 decoder layers (see [wang2025ddt]) isolate high-frequency detail synthesis from semantic encoding, thereby resolving the optimization conflict between capturing spatial detail and global video semantics.

  Figure 2: Motif-Video 2B model overview with three distinct backbone stages, enabling role separation for modality fusion, representation, and fine-detail reconstruction.

Attention visualizations confirm the DDT decoder develops strong inter-frame dependencies that are far less pronounced in the single- or dual-stream encoder blocks.

Figure 3: Inter-layer attention structure comparison—DDT exhibits strong, localized inter-frame attention, especially for temporal coherence.

Shared Cross-Attention

Standard joint self-attention results in severe dilution of text influence due to quadratic growth in video tokens versus a fixed number of text tokens. As frame or resolution increases, prompt alignment and scene semantics degrade.

Motif-Video 2B corrects this with Shared Cross-Attention modules appended to single-stream blocks. Key and value weights are shared with self-attention (for geometric compatibility), but queries are learned from the post-attention video state, enabling sequential refinement—i.e., the model can rediscover text relevance after local spatiotemporal context has been integrated. The design ensures stability even when inserted into pretrained checkpoints, unlike approaches that use raw text features for cross-attention, which can corrupt downstream representations (see Figure 4 in the paper).

Figure 5: Example of text-attention mass in dual vs. single-stream blocks. The single-stream block shows text influence degradation that Shared Cross-Attention addresses.

Figure 4: Empirical comparison—cross-attention with ungrounded key-value geometry (top) collapses model generations, while Shared Cross-Attention (bottom) preserves stability and coherence.

Efficiency-Oriented Training Regime

Curriculum and Representation Alignment

Training proceeds through a staged curriculum that alternates between text-to-image and text-to-video learning at incrementally higher resolutions and frame counts. Representation alignment with REPA is used early (with a V-JEPA teacher), boosting convergence speed substantially, and is then disabled to avoid bottlenecking model capacity as spatial detail and temporal complexity become the focus.

Token Routing (TREAD)

TREAD token routing randomly bypasses half of the tokens from early to deep layers during training, reducing per-step FLOPs by 27.5% and achieving a 1.3× throughput increase without noticeable loss in sample quality. Depth and stage selection for routing are tuned to avoid harming high-frequency detail reconstruction. Token routing is strictly a training efficiency measure and is not used at inference.

Bucket-Balanced Data Sampling

A WebDataset-compatible offline sampler assigns shards to training ranks to maintain balanced utilization across all resolution and frame count buckets, addressing synchronization bottlenecks in distributed training and recovering 90% data utilization from a baseline of 20%.

Figure 6: Overview of the offline bucket-balanced sampler, showing metadata-driven planning for efficient multi-bucket distributed training.

Dataset and Processing

Motif-Video 2B is trained on a mix of proprietary and public sources filtered through stringent multi-stage quality and domain controls:

• Sanitation removes non-decodable, NSFW, and watermarked clips.
• Aesthetic, technical, and motion quality filters use automated VLMs and optical flow statistics.
• Captioning and Annotation leverage Qwen3-VL, producing both structured tags and three caption variants per clip, allowing flexible conditioning and curriculum design.

This progressive data refinement, culminating in a high-quality SFT corpus, focuses model capacity on aesthetically and semantically reliable content.

Figure 7: Sankey diagram of the training data pipeline, visualizing progressive contraction toward high-quality training and SFT corpora.

Figure 8: Subject distribution of the SFT corpus, evidencing targeted curation for compositional and motion-diverse challenges.

Evaluation

Benchmark Results

On VBench [huang2023vbench], Motif-Video 2B achieves a Total Score of 83.76%, outperforming Wan2.1-14B (83.69%) and HunyuanVideo (83.24%), particularly on spatial relationship and semantic alignment metrics. Quality dimensions such as temporal flickering and background consistency remain behind the largest models, highlighting decodable capacity limits.

Attention Analysis

Cross-attention module contributions account for an average of 7.6% of the self-attention residual norm, peaking in early denoising steps where semantic grounding is established. The injected directions are nearly orthogonal to the standard self-attention outputs, confirming that these modules function as information injectors grounded in the backbone’s learned representation geometry.

Figure 9: Shared Cross-Attention contributes consistently across single-stream encoder blocks and throughout the denoising trajectory, with peak activity early in denoising.

Qualitative and Human Evaluation

Sample generations exhibit strong prompt fidelity, visual detail, and temporal consistency across diverse subjects and compositional styles.

Figure 10: Representative samples from Motif-Video 2B across subjects and visual styles.

Text-to-image and image-to-video modes can be handled within a unified model, with appearance preservation and motion generation controlled through dual-path conditioning and timestep-aware latent blurring.

Figure 11: Example image-to-video results, maintaining identity while extrapolating plausible motion.

Human preference studies reveal a consistent quality gap with 14B-scale models (notably in perceptual quality), yet Motif-Video 2B is preferred over all 2B- and 1.3B-class open alternatives.

Failure Analysis and Limitations

Micro-scale semantic distortions (anatomical artifacts, attribute leakage) and temporal failures (physically implausible motion, scene drift) persist. VBench aggregate scores do not capture these perceptual failures, which are more apparent to human raters. These limitations are hypothesized to reside in data coverage and scaling—fine anatomical fidelity and complex motion coherence require greater corpus diversity and potentially increased decoder capacity.

Figure 12: Semantic distortion examples at the sub-object level.

Figure 13: Typical temporal failure modes—unstable dynamics, identity drift, and scene transition errors.

Implications and Future Directions

This work formally demonstrates that careful architecture and micro-budget recipes can close significant performance gaps with much larger video generation models for text-grounded compositional correctness. However, scaling data diversity and model capacity, especially in decoders, will likely be required to address temporal stability and nuanced perceptual realism. The methodology provides a template for efficient, accessible advances in video generation models, with clear axes for further research in data curation, representation alignment phases, and capacity allocation for dynamic scene understanding.

Conclusion

Motif-Video 2B combines Shared Cross-Attention, three-stage role specialization, and an efficiency-focused training strategy to match or exceed the semantic and compositional metrics of significantly larger models with drastically reduced resources. The design and recipe are parameter-agnostic, positioning this approach for further scaling along data and decoder-capacity axes. Temporal coherence and anatomical fidelity remain main challenges, suggesting future research priorities in curated data acquisition, larger decoder architectures, and refined representation alignment.