Wan: Open-Source Video Foundation Models

• Wan is a family of large-scale, open-source video generative models built using the Diffusion Transformer paradigm with 1.3B and 14B parameter variants.
• It features a 3D causal VAE and spatio-temporal self-attention that enable efficient latent video diffusion and robust multimodal conditioning.
• Wan supports a broad range of applications including text-to-video, image-to-video, editing, personalization, streaming, and audio-driven cinematic adaptations.

Wan is a family of large-scale, open-source video foundation models built on the Diffusion Transformer paradigm, trained on billions of images and videos and released in 1.3B- and 14B-parameter variants. In the video-generation literature considered here, the term refers both to the general model suite described in the technical report and, in the specific case of Wan-S2V, to Wan-14B as a pre-trained text-to-video backbone that serves as a cinematic video prior for later audio-driven adaptation [2503.20314].

1. Definition, scope, and model family

Wan is presented as a comprehensive suite of video generative models rather than a single narrowly defined text-to-video system. The technical report emphasizes four defining properties: leading performance, comprehensiveness across multiple downstream applications, consumer-grade efficiency for the 1.3B variant, and full openness of code and weights. The released family centers on Wan 1.3B and Wan 14B, where the smaller model targets efficiency and the larger model targets state-of-the-art generation quality [2503.20314].

The suite is explicitly organized as a video foundation stack. At its core is a DiT-based latent video diffusion model coupled to a dedicated spatio-temporal variational autoencoder, multilingual text conditioning, large-scale pre-training, and a downstream adaptation framework spanning text-to-video, image-to-video, unified editing, personalization, camera control, real-time streaming generation, text-to-image, and video-to-audio generation. This breadth is important because Wan is framed not merely as a prompt-to-video generator, but as a reusable generative prior for a broader multimodal ecosystem [2503.20314].

Within later work, especially "Wan-S2V: Audio-Driven Cinematic Video Generation," Wan refers specifically to Wan-14B as a large diffusion transformer video generative model whose pre-trained knowledge of motion, camera movement, and multi-person scenes is reused and extended for speech-conditioned cinematic character generation [2508.18621].

2. Generative formulation and core architecture

Wan is a latent video diffusion architecture in which video frames are first compressed by Wan-VAE and then denoised in latent-token space by a Diffusion Transformer. For a video (V \in \mathbb{R}^{(1+T)\times H \times W \times 3}), Wan-VAE produces a latent tensor

[
x \in \mathbb{R}^{(1 + T/4) \times H/8 \times W/8 \times C}, \quad C = 16.
]

Temporal compression is (4\times), spatial compression is (8\times 8), and the first frame is only spatially compressed. A subsequent 3D convolution with kernel size ((1,2,2)) patchifies the latent into a token sequence of length

[
L = (1+T/4)\cdot H/16 \cdot W/16.
]

The DiT then performs full spatio-temporal self-attention over these tokens, with text injected through cross-attention and timestep information injected through adaptive layer normalization [2503.20314].

The underlying diffusion formalism is Rectified Flow within the Flow Matching framework. Let (x_1) denote a clean latent and (x_0 \sim \mathcal{N}(0,I)) denote Gaussian noise. Wan defines

[
x_t = t x_1 + (1-t)x_0,
]

with ground-truth velocity

[
v_t = \frac{d x_t}{dt} = x_1 - x_0.
]

The network (u(x_t, c_{\text{txt}}, t; \theta)) is trained by minimizing

[
\mathcal{L}{\text{RF}} =
\mathbb{E}{x_0, x_1, c_{\text{txt}}, t}
\left|u(x_t, c_{\text{txt}}, t; \theta) - v_t\right|^2.
]

This replaces a classical DDPM-style discrete noise schedule with a continuous velocity-field objective [2503.20314].

Wan-VAE is itself a substantial architectural contribution. It is a 3D causal VAE with RMSNorm rather than GroupNorm, a compact 127M-parameter design, and a feature-cache mechanism for chunk-wise causal convolution over arbitrarily long videos. The cache retains the needed previous frames for causal kernels while avoiding full-sequence recomputation. The reported design goal is simultaneous reconstruction quality, temporal causality, and efficiency, and the report states that Wan-VAE achieves high PSNR with (2.5\times) faster reconstruction than HunyuanVideo’s VAE on 200 test videos of 25 frames at (720\times720) [2503.20314].

Text conditioning is provided by umT5 rather than a causal LLM encoder. The report attributes this choice to strong multilingual ability and bidirectional attention, both of which were found more suitable for the diffusion setting. A further architectural choice is to share the timestep-conditioned adaLN MLP across transformer blocks while giving each block its own biases; this reduces parameters by about (25\%) relative to an unshared design and, in the reported ablations, favors deeper DiTs over duplicated modulation networks [2503.20314].

3. Data curation, captioning, and scaling strategy

Wan is trained on a very large multimodal corpus assembled through aggressive filtering and caption enrichment. The pipeline begins from a very large candidate pool of images and videos and removes roughly half of it through fundamental filtering stages that include OCR-based text coverage, aesthetic filtering, NSFW detection, watermark and logo detection, black-border cropping, overexposure detection, synthetic-image detection, blur detection, and minimum duration and resolution constraints. Subsequent stages apply feature clustering, expert-model quality scoring, and motion-based stratification to favor visually clean samples with meaningful and diverse dynamics [2503.20314].

A dedicated dense captioning model is trained to provide richer supervision than generic web captions. This caption model follows a LLaVA-style design with a ViT vision encoder and a Qwen 2.5 language model, and is trained on caption, QA, and instruction-style data, including specialized supervision for camera angle, camera motion, object counting, OCR, spatial relations, and fine-grained visual categories. The resulting captions are used not only for semantic alignment but also for explicit supervision of motion-centric and camera-centric prompt dimensions [2503.20314].

The text-rendering capability is supported by a specialized visual-text data mixture. One branch consists of hundreds of millions of synthetic images with rendered Chinese characters on white backgrounds; another uses real web images containing text, from which OCR is extracted and passed to Qwen2-VL to produce dense captions that preserve exact text content. The report directly attributes Wan’s Chinese- and English-text generation ability to this synthetic-plus-real training mixture [2503.20314].

Pre-training follows a curriculum. Wan first undergoes text-to-image pre-training at (256) px, then image-video joint training in three stages: images at (256) px with videos at (192) px and 5 seconds, then both images and videos at (480) px, and finally both at (720) px. A post-training stage further refines the model using a few million high-quality images and millions of carefully selected videos with both simple and complex motion. This scaling strategy is explicitly linked to the report’s claim that Wan demonstrates scaling laws for video generation with respect to both data and model size [2503.20314].

4. Task coverage and controllability

Wan’s task surface is unusually broad. As a text-to-video model, it is described as supporting large motion, complex camera movements, strong physical plausibility, multilingual prompt following, and in-scene visual text rendering. Because image and video training are unified, the same backbone also serves as a high-quality text-to-image generator [2503.20314].

Wan-I2V extends the model to image-to-video by concatenating a condition latent derived from the reference image, a binary mask specifying fixed versus generated regions, and global image features from a CLIP image encoder. The same mask-based framework is used for related tasks such as continuation, first-last-frame transformation, and random-frame interpolation. This is not an isolated auxiliary model; it is presented as a unified conditioning interface over the same latent-video backbone [2503.20314].

Editing is systematized through VACE, a unified video editing framework based on a Video Condition Unit (V=[T;F;M]), where (T) is text, (F) is a set of context frames, and (M) is a set of aligned binary masks. The design decouples editable and preserved content and can be realized either by full fine-tuning or by a Context Adapter, also called Res-Tuning, that leaves the base backbone frozen while injecting context features into the original DiT blocks. VACE is described as supporting text-guided editing, inpainting, outpainting, style transfer, reference-to-video, and analogous image-editing formulations [2503.20314].

Personalization is handled through raw face-image conditioning in latent space rather than via an external identity embedding model. During training, segmented face frames are prepended to the video latent sequence with aligned masks so that the DiT learns to reconstruct these identity cues and propagate them into later generated frames. The report states that this design achieves an ArcFace similarity of (0.5526), competitive with leading commercial personalization systems [2503.20314].

Wan also includes explicit camera-motion control. Camera pose is encoded using Plücker-coordinate representations derived from intrinsics and extrinsics, then injected into the DiT as affine modulation terms of the form

[
f_i = (\gamma_i + 1)\cdot f_{i-1} + \beta_i.
]

This allows prompts and camera trajectories to jointly determine generation. Beyond offline synthesis, Wan is extended to streaming generation through Streamer and Latent Consistency Model distillation, enabling infinite-horizon generation with bounded attention windows and large sampling-speed reductions. A separate video-to-audio branch adds synchronized ambient sound and background music using a 1D audio VAE and a DiT conditioned on video and text features [2503.20314].

5. Wan as the backbone of Wan-S2V

In "Wan-S2V: Audio-Driven Cinematic Video Generation," Wan is used specifically as the backbone of an audio-driven cinematic character video generator. The paper characterizes Wan-14B as a DiT-style latent video diffusion model with a 3D VAE encoder-decoder, a transformer operating on patchified latent tokens, and cross-attention to text tokens for prompt control. In that setting, Wan functions as a pre-trained video prior from which Wan-S2V starts and then fully fine-tunes all parameters end-to-end after adding audio-conditioning modules and modified attention blocks called Audio Blocks [2508.18621].

The significance of Wan in this context is not just architectural convenience. The paper states that Wan already encodes rich motion, camera movement, and multi-person scene structure, and that Wan-S2V preserves Wan’s text conditioning while extending it to joint text-plus-audio control. Text remains responsible for global semantics such as camera motion, scene layout, and interaction structure, while audio modulates local motion trajectories, lip synchronization, expressions, and gestures [2508.18621].

This usage clarifies an important property of Wan as a foundation model: it can be retrofitted into downstream multimodal systems without discarding its cinematic prior. A plausible implication is that Wan’s role is analogous to a reusable generative manifold over which new condition channels can be learned, rather than a backbone that must be narrowly retrained for every task. That interpretation is strongly suggested by the end-to-end Wan-S2V design, though the paper formulates it concretely as preserving Wan’s global text-controlled cinematic plan while introducing per-frame audio-visual attention for local synchronization [2508.18621].

6. Evaluation, efficiency, openness, and limitations

Wan is evaluated through both public and in-house benchmarks. On VBench, Wan 14B reports a total score of (86.22\%), with (86.67\%) visual quality and (84.44\%) semantic quality, while Wan 1.3B reports (83.96\%). On Wan-Bench, the 14B model achieves the highest weighted score, (0.724), and the 1.3B model scores (0.689). Human evaluations reported for text-to-video and image-to-video show strong win rates against both open and commercial systems, including I2V overall win rates up to (81.6\%) against leading competitors [2503.20314].

Efficiency is a central design target. The report states that Wan 1.3B requires only (8.19) GB VRAM, making it compatible with consumer-grade GPUs. At the systems level, Wan combines context parallelism, FSDP, activation offloading, diffusion caching, FP8 GEMM, and 8-bit attention. Diffusion cache alone is reported to provide (1.62\times) speedup on Wan 14B, while FP8 GEMM and 8-bit attention provide additional acceleration. In streaming settings, the report describes 10–20(\times) speedups through consistency distillation and gives examples of real-time generation regimes on A100 and RTX 4090 hardware [2503.20314].

Openness is treated as a defining characteristic of the project. The report states that the entire series of Wan models, source code, training and inference pipelines, evaluation tools, and associated components are released under the Wan 2.1 open-source package at the public repository specified in the report [2503.20314].

The main reported limitations concern fine-grained detail retention under large motion, the substantial computational cost of the 14B model, incomplete specialization for narrow professional domains such as medical or educational video, and the absence of speech generation in the current video-to-audio branch. Wan-S2V adds a related downstream limitation: even with Wan as a cinematic prior, truly nuanced multi-person film interaction and precise camera control driven solely by audio remain difficult [2508.18621].

Taken together, these papers present Wan as an open, large-scale video foundation architecture whose importance lies in both its internal technical design and its role as a reusable prior for downstream multimodal generation. The technical report frames it as a comprehensive model family spanning generation, editing, personalization, streaming, and audio, while Wan-S2V demonstrates that the same pretrained prior can support film-style audio-driven character animation without abandoning its original text-to-video structure [2503.20314].