Waver: Wave Your Way to Lifelike Video Generation (2508.15761v2)

Abstract: We present Waver, a high-performance foundation model for unified image and video generation. Waver can directly generate videos with durations ranging from 5 to 10 seconds at a native resolution of 720p, which are subsequently upscaled to 1080p. The model simultaneously supports text-to-video (T2V), image-to-video (I2V), and text-to-image (T2I) generation within a single, integrated framework. We introduce a Hybrid Stream DiT architecture to enhance modality alignment and accelerate training convergence. To ensure training data quality, we establish a comprehensive data curation pipeline and manually annotate and train an MLLM-based video quality model to filter for the highest-quality samples. Furthermore, we provide detailed training and inference recipes to facilitate the generation of high-quality videos. Building on these contributions, Waver excels at capturing complex motion, achieving superior motion amplitude and temporal consistency in video synthesis. Notably, it ranks among the Top 3 on both the T2V and I2V leaderboards at Artificial Analysis (data as of 2025-07-30 10:00 GMT+8), consistently outperforming existing open-source models and matching or surpassing state-of-the-art commercial solutions. We hope this technical report will help the community more efficiently train high-quality video generation models and accelerate progress in video generation technologies. Official page: https://github.com/FoundationVision/Waver.

• The paper introduces Waver, a unified foundation model that leverages a Hybrid Stream DiT and Cascade Refiner to enable efficient high-resolution video generation.
• It integrates a comprehensive data curation pipeline and advanced representation alignment to enhance motion synthesis and visual quality across text-to-image, text-to-video, and image-to-video tasks.
• Empirical evaluations demonstrate that Waver ranks highly on public leaderboards with superior human evaluation metrics, particularly in complex motion scenarios.

Waver: Unified High-Fidelity Video Generation via Hybrid Stream DiT and Cascade Refinement

Introduction

The paper presents Waver, a unified foundation model for high-resolution image and video generation, supporting text-to-video (T2V), image-to-video (I2V), and text-to-image (T2I) tasks within a single framework. Waver leverages a Hybrid Stream DiT architecture and a Cascade Refiner to achieve efficient training and inference for 1080p video synthesis, with strong performance in both general and complex motion scenarios. The model is distinguished by its comprehensive data curation pipeline, advanced representation alignment, and infrastructure optimizations, resulting in top-tier rankings on public leaderboards and superior human evaluation metrics.

Model Architecture

Waver's architecture is composed of two principal modules: Task-Unified DiT and Cascade Refiner. The Task-Unified DiT is built on rectified flow Transformers and is designed to jointly model T2I, T2V, and I2V tasks via a flexible input conditioning mechanism. This mechanism concatenates noisy latents, conditional frame latents, and binary masks, enabling seamless task unification and extensibility to video interpolation and other conditional tasks.

Figure 1: Architecture of Task-Unified DiT, illustrating the integration of dual and single stream blocks for modality alignment and efficiency.

The Hybrid Stream structure combines Dual Stream blocks (separate parameters for video and text, merged at self-attention) in the early layers for strong modality alignment, and Single Stream blocks (shared parameters) in later layers for parameter efficiency. Empirical results demonstrate that this hybridization accelerates convergence compared to pure approaches.

Figure 2: Loss comparison between Hybrid Stream, Dual Stream, and Single Stream structures. Hybrid Stream's loss converges faster.

For positional encoding, Waver employs a hybrid scheme: 3D RoPE for relative spatio-temporal positions and factorized learnable embeddings for absolute positions, enhancing extrapolation and convergence.

The Cascade Refiner, also based on rectified flow Transformers, upscales 720p videos to 1080p using window attention and a two-stage degradation process (pixel and latent). This approach achieves a 40% acceleration over single-stage 1080p generation and corrects generative artifacts.

Figure 3: Pipeline of Cascade Refiner, depicting the hierarchical upscaling and artifact correction process.

Figure 4: Refiner output: (a) upscaling and artifact correction; (b) video editing via latent manipulation.

Data Curation and Quality Modeling

Waver's data pipeline integrates multi-source acquisition, advanced segmentation, hierarchical filtering, and semantic balancing. Scene detection is refined using DINOv2 features, and motion magnitude is quantified via RAFT-based optical flow, with foreground segmentation to isolate subject dynamics.

Figure 5: Data processing pipeline: acquisition, preprocessing, filtering, captioning, and balancing.

A manually annotated MLLM-based quality model further filters over 1M clips, labeling 13 dimensions of low quality. The model achieves 78% accuracy in high-quality sample prediction.

Figure 6: Distribution of video quality issues in annotated training clips.

Hierarchical filtering stages progressively increase quality criteria, culminating in a dataset optimized for each training phase.

Figure 7: Hierarchical data filtering funnel for progressive dataset refinement.

Synthetic data generation and manual review are employed to address aesthetic gaps, especially in high-motion scenes.

Figure 8: Aesthetic comparison: synthetic data exhibits superior lighting and composition over real-world video.

Figure 9: Manual review statistics for synthetic video samples; distortion is the primary failure reason.

Training and Optimization Strategies

Waver's multi-stage training schedule covers T2I, T2V, I2V, and refinement tasks, with progressive resolution scaling and joint training to enhance motion and instruction following. Representation alignment is enforced via cosine similarity between Qwen2.5-VL semantic features and DiT intermediate features, yielding improved semantic quality.

Figure 10: Qualitative comparison: representation alignment produces more organized and meaningful video semantics.

Motion optimization is achieved through low-resolution pretraining, mode-based timestep sampling, and joint T2V/I2V training. Mode sampling yields larger motion amplitudes than logit-normal, as evidenced in ablation studies.

Figure 11: 480p T2V results: 192p pretraining enables larger motion.

Figure 12: 720p T2V: mode sampling produces more intense motion than logit-normal.

Figure 13: Probability density functions for timestep sampling: mode distribution is sharply peaked for T2V/I2V.

I2V motion is further improved by introducing initial frame conditioning with a 20% probability during joint training.

Figure 14: 720p I2V: joint T2V/I2V training yields larger motion amplitudes.

Foreground motion scoring and quality model filtering remove static and slow-motion clips, ensuring training data supports robust motion synthesis.

Figure 15: Foreground optical flow: high vs. low motion scores in training clips.

Aesthetics optimization leverages curated synthetic data and balanced data diets, with high-aesthetic finetuning yielding a 7% improvement in visual quality without degrading motion.

Figure 16: Visual quality before and after high-aesthetic finetuning.

Model balancing is achieved via prompt tagging, video APG (CFG decomposition and normalization), and model averaging, with empirical improvements in realism and artifact reduction.

Figure 17: Six distinct video styles generated via prompt tagging.

Figure 18: APG hyperparameters yield greater realism and fewer artifacts than standard CFG.

Figure 19: Model averaging improves motion, visual quality, and prompt following.

Prompt rewriting aligns user inputs with training captions, enhancing generation consistency and richness.

Figure 20: Prompt rewriting improves visual richness and aesthetic quality.

Infrastructure and Scaling

Waver employs hybrid sharded FSDP, torch.compile, Ulysses sequence parallelism, bucket dataloaders, selective activation checkpointing, and activation offloading to enable efficient training of extreme long-sequence, high-resolution video models.

Figure 21: Infrastructure optimizations: sequence parallelism, checkpointing, and offloading.

MFU (Model FLOPs Utilization) increases from 0.32 at 192p to 0.40 at 1080p, indicating effective hardware utilization.

Benchmarking and Evaluation

Waver ranks third on both T2V and I2V tracks of the Artificial Analysis Arena, matching or surpassing state-of-the-art commercial solutions.

Figure 22: Official T2V and I2V leaderboard rankings: Waver is third in both.

Human evaluation on Waver-bench 1.0 and Hermes Motion Testset demonstrates superior motion quality, visual quality, and prompt following compared to Veo3, Kling2.0, and Wan2.1, with pronounced advantages in complex motion scenarios.

Figure 23: Human evaluation win rates: Waver outperforms competitors, especially in complex motion scenarios.

Empirical Analysis and Future Directions

Sparse attention analysis reveals heterogeneous, layer-wise, and timestep-wise sparsity patterns, motivating adaptive sparse attention mechanisms over fixed-window approaches. NSA-based methods and spatial-temporal sliding windows are promising for future efficiency gains.

VAE optimization is critical: KL loss weight must be balanced to avoid grainy textures and background distortion, while LPIPS loss should be minimized to prevent grid-like artifacts. Future VAEs should target higher compression ratios and multimodal fusion.

Detailed captioning and explicit spatial relation QA pairs are shown to improve instruction following and semantic alignment in T2V models.

Conclusion

Waver establishes a unified, scalable framework for high-fidelity video generation, integrating architectural innovations, data curation, and infrastructure optimizations. The model achieves strong empirical results in both general and complex motion scenarios, with robust motion, visual quality, and prompt adherence. Limitations remain in high-motion detail preservation and expressiveness, suggesting future work in RL-based distortion mitigation and multimodal VAE development. The practical recipes and insights provided are directly applicable to advancing video generation research and deployment.