HunyuanVideo: Scalable Multimodal Video Generation

• HunyuanVideo is a family of large-scale, multimodal video models that integrate 3D latent video representation with transformer-based denoising for advanced video generation.
• It employs a unified two-stage pipeline that combines a Diffusion Transformer generator with video super-resolution to produce high-resolution, semantically coherent videos.
• The ecosystem has spurred diverse descendant models tailored for animation, human-object interaction, audio-driven avatars, and inference optimization, driving practical deployment advances.

HunyuanVideo is a family of large-scale video models in the Tencent Hunyuan ecosystem spanning text-to-video and image-to-video generation, domain-adapted animation and human animation, video-to-audio synthesis, and structured video comprehension. In the literature surveyed here, the name refers both to an earlier 13B-class generative backbone reused by later adaptation and acceleration studies, and to HunyuanVideo 1.5, an 8.3B-parameter open-source successor implemented as a two-stage system consisting of a unified Diffusion Transformer video generator and a video super-resolution model (Wu et al., 24 Nov 2025). Across these works, HunyuanVideo is characterized by a 3D latent video representation, transformer-based denoising, strong multimodal conditioning, and unusually broad reuse as a substrate for specialization and inference-time optimization (Chen et al., 21 Dec 2025).

1. Architectural foundations

In PTTA, HunyuanVideo is described as a large latent diffusion model trained in a VAE latent space, with a causal 3D VAE, a multimodal LLM text encoder, and a DiT-based denoising backbone at 13B-parameter-scale. For a video $\mathbf{v}$ with latent $\mathbf{z}_0 = \text{VAE.encode}(\mathbf{v})$, the forward process is

$$\mathbf{z}_t = \sqrt{\bar{\alpha}_t}\mathbf{z}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon},$$

and the conditioning-aware denoiser is trained with the latent-space MSE objective

$$\mathcal{L}_{\text{LDM}} = \mathbb{E}_{\mathbf{z}_0, \mathbf{c}, \boldsymbol{\epsilon}, t} \left[ \left\| \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_{\theta}(\mathbf{z}_t, t, \mathbf{c}) \right\|^2 \right].$$

PTTA keeps this generative skeleton unchanged and adapts it only through HydraLoRA inserted into the linear layers of the transformer backbone (Chen et al., 21 Dec 2025).

Other descendants describe the inherited base somewhat differently. HunyuanVideo-HOMA states that its generator is modified from HunyuanVideo-T2V, “including an MMDiT-based video diffusion model and a 3D VAE,” and characterizes the inherited training paradigm as flow matching rather than DDPM-style denoising. HunyuanVideo-Avatar similarly adopts HunyuanVideo-I2V with an MM-DiT backbone and 3D VAE, and FastLightGen treats HunyuanVideo as a large flow-matching DiT video model whose generic objective is written as

$$\mathop{\arg\min}_{\Theta} \int_0^1 \mathbb{E}\left[\alpha_t\Vert \partial p_t(\mathbf{x}_t)/\partial t - \mathbf{v}_\Theta(\mathbf{x}_t,t)\Vert_2^2\right]dt.$$

These descriptions indicate that the HunyuanVideo lineage is consistently organized around latent video transformers and multimodal conditioning, even though different papers foreground different training parameterizations and interfaces (Huang et al., 10 Jun 2025, Chen et al., 26 May 2025, Shitong et al., 2 Mar 2026).

2. HunyuanVideo 1.5 as a unified open-source system

HunyuanVideo 1.5 is presented as a compact but high-capacity successor with a single 8.3B-parameter backbone and a two-stage pipeline: a unified DiT video generator for T2V, I2V, and T2I, followed by a video super-resolution model that upscales outputs to 1080p. The generator operates in a 3D latent space produced by a causal 3D VAE with $16\times$ spatial compression, $4\times$ temporal compression, and 32 latent channels. The DiT backbone comprises 54 dual-stream blocks, model dimension 2048, FFN dimension 8192, 16 attention heads, and head dimension 128 (Wu et al., 24 Nov 2025).

A defining systems contribution of HunyuanVideo 1.5 is Selective and Sliding Tile Attention (SSTA), a parameter-free block-sparse attention mechanism designed for long, high-resolution video latents. SSTA computes block-level similarity and redundancy scores,

$$\mathrm{Score}_i = \lambda \cdot \mathrm{Score}_s - \beta \cdot \mathrm{Score}_r,$$

selects top-$k$ important blocks, combines these with a local sliding-window mask, and executes the resulting pattern with a custom block-attention kernel. On 720p, 241-frame T2V, SSTA reduces per-step time from 5.5070 s to 2.9475 s without engineering-level tricks; with SageAttention, torch.compile, and feature caching, total 50-step runtime falls from 96.78 s to 58.39 s (Wu et al., 24 Nov 2025).

The conditioning stack is also distinctive. HunyuanVideo 1.5 uses a dual text encoder combining Qwen2.5-VL for semantic understanding and Glyph-ByT5 for glyph-aware multilingual text processing, particularly for Chinese. I2V conditioning combines VAE latent concatenation of the reference image with the noisy video latent and SigLIP semantic embedding concatenation with text tokens. A learnable type embedding indicates whether the current task is T2I, T2V, or I2V (Wu et al., 24 Nov 2025).

Training is progressive in both modality and scale. The report describes 5B images at 256p, 1B images at 512p, then video stages from 256p, 16 fps, 2–10 s through 720p, 24 fps, 2–10 s, followed by continuing training for T2V and I2V, supervised fine-tuning, and RLHF. The filtered video corpus contains approximately 800M high-quality video segments after multi-stage curation from a raw pool of more than 10 million hours. This design supports native generation at 480p–720p, 16–24 fps, and 2–10 seconds, while still permitting 720p, 121-frame generation on a single consumer GPU with 13.6 GB peak memory via offloading and VAE tiling (Wu et al., 24 Nov 2025).

3. Task-specific descendants and ecosystem expansion

HunyuanVideo has functioned less as a single model than as a reusable substrate for domain-specific systems. In PTTA, the base HunyuanVideo backbone is frozen and adapted to animation with HydraLoRA inserted into all linear layers of the DiT. Training uses over 12,000 text–animation pairs, rank = 32, alpha = 32, and a default $N=4$ HydraLoRA heads corresponding to male, female, object, background. On VideoScore metrics, PTTA improves over HunyuanVideo(Base) from 2.823 to 2.895 in VSVQ, from 2.858 to 2.933 in VSDD, and from 2.713 to 3.078 in VSTVA, while maintaining competitive temporal coherence (Chen et al., 21 Dec 2025).

HunyuanVideo-HOMA extends the backbone to generic human-object interaction generation under weak supervision. Its inputs include a reference human image, reference object image, sparse arm pose, sparse object-center trajectory dots, optional text, and optional audio. It introduces dual-space appearance and motion fusion, a parameter-space HOI adapter inserted into even-numbered MMDiT layers, and a facial audio cross-attention adapter. On the self-collected HOI benchmark, HOMA reports FID 51.60, FVD 502.69, Object CLIP 90.05, and Sync-C 4.33, and outperforms baselines on both the complex HOI test set and the AnchorCrafter benchmark without per-object finetuning (Huang et al., 10 Jun 2025).

HunyuanVideo-Avatar further specializes the family for audio-driven human animation, including multi-character dialogue videos. It retains the HunyuanVideo-I2V MM-DiT core and adds three modules: a Character Image Injection Module (CIM), an Audio Emotion Module (AEM), and a Face-Aware Audio Adapter (FAA). Training uses approximately 500,000 filtered clips, about 1,250 hours of video, with a two-stage regimen over 160 GPUs. On a wild full-body test set, the model attains IQA 4.66, ASE 3.03, Sync-C 5.56, and FID 49.38, while a user study reports the best identity preservation (4.84) and lip synchronization (4.65) among compared systems (Chen et al., 26 May 2025).

HunyuanVideo-Foley occupies the audio side of the ecosystem. It is a text–video-to-audio model that pairs video features from SigLIP2, text features from CLAP, audio latents from a DAC-VAE, and a hybrid MMDiT + unimodal audio DiT generator. Its dataset pipeline curates approximately 100k hours of text–video–audio triplets, and its REPA strategy aligns internal audio features to ATST-Frame representations via a cosine-similarity loss. On Kling-Audio-Eval it achieves FD$\mathbf{z}_0 = \text{VAE.encode}(\mathbf{v})$0 6.07, KL 1.89, PQ 6.12, IB 0.38, and DeSync 0.54, all reported as best among compared baselines in that table (Shan et al., 23 Aug 2025).

The name also now covers a comprehension branch. ARC-Hunyuan-Video-7B is presented as the structured video understanding counterpart within the broader HunyuanVideo ecosystem rather than a generator. Built on Hunyuan-7B VLM with a ViT visual encoder, Whisper audio encoder, and timestamp-overlaid frame sampling, it supports timestamped captioning, summarization, QA, grounding, and reasoning. It reaches 74.3 accuracy on ShortVid-Bench, 54.8 mIoU on Charades-STA, and processes a one-minute video in about 10 seconds on an H20 GPU (Ge et al., 28 Jul 2025).

4. Inference acceleration and systems research

A large portion of recent efficient video-generation research uses HunyuanVideo as a principal benchmark. This suggests that its long contexts, transformer-heavy compute profile, and open implementations have made it a canonical stress test for inference-time optimization (Yang et al., 24 May 2025, Zheng et al., 20 May 2026).

Method	Mechanism	Reported HunyuanVideo result
AdaSpa (Xia et al., 28 Feb 2025)	Dynamic block-sparse attention with online precise search	1.78× speedup on 8 s, 720p generation; VBench 80.13 vs 80.10 dense
SVG2 (Yang et al., 24 May 2025)	Semantic-aware permutation sparse attention	2.30× FP16 / 2.55× FP8 speedup on 720p, 33-frame T2V; PSNR 30.452, VBench 0.852
Fresco (Zheng et al., 12 Jan 2026)	Progressive spatial resolution with unified noise field	4.68× latency and 4.92× FLOPs reduction on 720×1280, 125 frames; Total VBench 80.76 vs 80.12
DPCache (Cui et al., 26 Feb 2026)	Path-aware key-timestep selection with cached feature prediction	4.75× speedup at 640×480×65; VBench 80.23 vs 80.93, memory overhead +0.36 GB
HyCa (Zheng et al., 5 Oct 2025)	Dimension-wise hybrid ODE-solver feature caching	5.56× FLOPs speedup; VBench 80.25 vs 80.66
Chipmunk (Silveria et al., 3 Jun 2025)	Dynamic column-sparse deltas over cached activations	2.16× alone, 3.72× with step caching; VBench total 82.94 alone
LVSA (Glorian et al., 29 May 2026)	Windowed sparse attention with rotating global anchors	3.33× speedup at 193 frames on HunyuanVideo 1.5; enables 257-frame generation where dense is OOM
DVG (Zheng et al., 20 May 2026)	Content-aware dynamic allocation across time and space	Up to 7.35× on HunyuanVideo T2V and 18.09× on distilled HunyuanVideo-1.5 I2V
FastLightGen (Shitong et al., 2 Mar 2026)	Joint step-and-parameter distillation	4-step, 70%-parameter HunyuanVideo-ATI2V with roughly 35.71× theoretical speedup over 50-step CFG

These methods target different bottlenecks. Sparse-attention systems such as AdaSpa, SVG2, LVSA, and Chipmunk exploit the fact that HunyuanVideo is strongly attention-bound: AdaSpa estimates roughly 500 PFLOPs of attention out of 600 PFLOPs for an 8-second 720p sample, while SVG2 reports that attention can take more than 80% of end-to-end time for a 33-frame HunyuanVideo run with about 118,800 tokens per denoising step (Xia et al., 28 Feb 2025, Yang et al., 24 May 2025). Dynamic-resolution systems such as Fresco and DVG instead reshape the latent grid over denoising stages. Caching methods such as DPCache and HyCa exploit the smoothness of cross-step hidden trajectories, while FastLightGen changes the model itself through pruning and few-step distillation rather than remaining purely training-free (Zheng et al., 12 Jan 2026, Cui et al., 26 Feb 2026, Zheng et al., 5 Oct 2025, Shitong et al., 2 Mar 2026).

5. Evaluation, deployment characteristics, and practical usage

HunyuanVideo 1.5 is evaluated mainly with human-centric rating and GSB comparisons. In T2V ratings, it reports 61.57 for instruction following, 63.30 for aesthetic quality, 57.35 for visual quality, 79.75 for structural stability, and 57.67 for motion effects. In GSB, HunyuanVideo 1.5 at 720p records positive win rates over Wan2.2 (+17.12%), Kling2.1 Master (+12.60%), and Seedance Pro (+11.02%), while remaining behind Veo3 (-10.32%). In I2V GSB, it is positive against Wan2.2 (+12.65%) and Kling2.1 Master (+9.72%), and slightly negative against Seedance Pro (-5.77%) and Veo3 (-3.61%) (Wu et al., 24 Nov 2025).

A complementary deployment-oriented picture appears in LLMPopcorn, which treats HunyuanVideo as one of three open-source video generators in an LLM-assisted micro-video pipeline. There, HunyuanVideo uses 12,016 MB of VRAM and 54 seconds per video on a single NVIDIA H100 80G GPU, outperforming CogVideoX-5B in most pairwise popularity comparisons and remaining close to LTX-Video. Under the basic prompt setting, HunyuanVideo wins 56% of comparisons against CogVideoX-5B on concrete prompts and 51% against LTX-Video, but under prompt enhancement it falls to 40% against LTX-Video on concrete prompts, suggesting that it is competitive yet not uniformly the most responsive generator to popularity-oriented prompt engineering (Fu et al., 18 Feb 2025).

Beyond generation, the HunyuanVideo-named ecosystem is already used in production comprehension settings. ARC-Hunyuan-Video-7B has been deployed in Tencent products for brief summaries, detailed summaries, and recommendation-oriented “extended browsing words,” with reported retrieval CTR +5.88%, landing-page consumption time +5.11%, and video floating-layer click CTR +7.26% after downstream finetuning (Ge et al., 28 Jul 2025). This does not make ARC-Hunyuan-Video a generator, but it shows that the HunyuanVideo label now denotes a broader multimodal video stack rather than only a denoising backbone.

6. Limitations, robustness, and research directions

Several limitations recur across the literature. In animation transfer, base HunyuanVideo is trained primarily on natural videos and exhibits geometric inconsistencies, style mismatch, and less suitable temporal behavior on anime prompts; PTTA explicitly reports issues such as “misaligned books,” realistic shading where line art is desired, and motion that is not tuned to stylized animation physics (Chen et al., 21 Dec 2025). In weakly conditioned HOI generation, HOMA remains sensitive to the distance between the hand skeleton and object trajectory dot, uses no explicit physics or collision losses, and depends on a weakly labeled dataset of about 140 hours (Huang et al., 10 Jun 2025). In audio-driven avatar animation, HunyuanVideo-Avatar still relies on emotion reference images rather than direct emotion extraction from audio, and a 10-second video at 720×1216 and 50 diffusion steps is reported to take about 60 minutes on the authors’ hardware (Chen et al., 26 May 2025). HunyuanVideo 1.5 itself, despite strong open-source performance, still trails Veo3 on some T2V human-evaluation axes, especially instruction following and motion effects (Wu et al., 24 Nov 2025).

Prompt robustness has also become an explicit research theme. T2VAttack evaluates HunyuanVideo as a black-box victim model and finds it to be the most robust among the four tested systems, partly attributing this to an integrated prompt rewrite mechanism. Even so, adversarial prompt perturbations remain substantial: on T2VAttackBench, HunyuanVideo’s semantic score falls from 83.0 to 58.8 under substitution-based semantic attack, and its temporal score falls from 72.1 to 36.6 under substitution-based temporal attack; insertion attacks are weaker but still significant (Li et al., 30 Dec 2025).

Future directions are converging around longer horizons, tighter multimodal integration, and more explicit control. PTTA points toward longer narratives and finer style controls for animation; HOMA motivates improved weak controls and stronger human–object disentanglement; HunyuanVideo-Avatar identifies direct audio emotion modeling and real-time avatar generation as open problems; HunyuanVideo-Foley suggests eventual end-to-end text-to-video-plus-audio generation; and long-video acceleration work such as LVSA and DVG points toward content-aware spatiotemporal scheduling for horizons beyond current training lengths (Chen et al., 21 Dec 2025, Huang et al., 10 Jun 2025, Chen et al., 26 May 2025, Shan et al., 23 Aug 2025, Glorian et al., 29 May 2026, Zheng et al., 20 May 2026). Taken together, these works indicate that HunyuanVideo has evolved from a single generative backbone into a broader research program around scalable, controllable, and increasingly multimodal video systems.