UltraGen: High-Resolution Video Generation with Hierarchical Attention (2510.18775v1)

Abstract: Recent advances in video generation have made it possible to produce visually compelling videos, with wide-ranging applications in content creation, entertainment, and virtual reality. However, most existing diffusion transformer based video generation models are limited to low-resolution outputs (<=720P) due to the quadratic computational complexity of the attention mechanism with respect to the output width and height. This computational bottleneck makes native high-resolution video generation (1080P/2K/4K) impractical for both training and inference. To address this challenge, we present UltraGen, a novel video generation framework that enables i) efficient and ii) end-to-end native high-resolution video synthesis. Specifically, UltraGen features a hierarchical dual-branch attention architecture based on global-local attention decomposition, which decouples full attention into a local attention branch for high-fidelity regional content and a global attention branch for overall semantic consistency. We further propose a spatially compressed global modeling strategy to efficiently learn global dependencies, and a hierarchical cross-window local attention mechanism to reduce computational costs while enhancing information flow across different local windows. Extensive experiments demonstrate that UltraGen can effectively scale pre-trained low-resolution video models to 1080P and even 4K resolution for the first time, outperforming existing state-of-the-art methods and super-resolution based two-stage pipelines in both qualitative and quantitative evaluations.

• The paper introduces a hierarchical dual-branch attention architecture that decomposes full attention into local and global branches for efficient high-resolution video generation.
• The approach achieves state-of-the-art 1080P and 4K outputs with strong semantic consistency and a 4.78× speedup compared to baseline models.
• Ablation studies confirm that components like spatial compression, cross-window, and domain-aware LoRA are essential for ensuring high fidelity and reduced computational costs.

UltraGen: High-Resolution Video Generation with Hierarchical Attention

Introduction

UltraGen addresses the persistent challenge of native high-resolution video generation in diffusion transformer-based models, which are fundamentally constrained by the quadratic complexity of full spatiotemporal attention. Existing approaches either limit output to low resolutions (≤720P) or rely on two-stage pipelines (low-res generation followed by super-resolution), both of which fail to deliver authentic, detail-rich high-definition (HD) content. UltraGen introduces a hierarchical dual-branch attention architecture that decomposes full attention into local and global branches, enabling efficient, end-to-end generation of 1080P and 4K videos with strong semantic consistency and fine-grained detail.

Figure 1: Typical video generation models exhibit significant quality degradation and increased processing time with higher resolutions, whereas UltraGen delivers superior video quality at resolutions beyond 2K while achieving a 4.78× speedup compared to the Wan-T2V-1.3B baseline.

Hierarchical Dual-Branch Attention Architecture

UltraGen's core innovation is the decomposition of full attention into two complementary branches:

• Local Attention Branch: Focuses on high-fidelity regional content by partitioning the video latent into non-overlapping spatial windows and applying self-attention within each window. This reduces computational complexity from $\mathcal{O}((TWH)^2 D)$ to $\mathcal{O}((TWH)^2 D / K^2)$, where $K$ is the number of windows per spatial dimension.
• Global Attention Branch: Ensures overall semantic consistency by compressing the spatial dimensions of the video latent via frame-wise depthwise convolution, applying self-attention at the reduced resolution, and then decompressing via 3D convolution to restore spatial and temporal fidelity.

The outputs of both branches are fused using a time-aware, learnable factor $\alpha(t)$, which dynamically balances global and local information throughout the diffusion process. Early denoising steps emphasize global structure, while later steps refine local details.

Figure 2: Overview of UltraGen, which decomposes full attention into a global attention branch for semantic consistency and a local attention branch for high-fidelity regional content.

Spatially Compressed Global Attention

The global attention branch leverages spatial compression to make global context modeling tractable at high resolutions. A $k \times k$ depthwise convolution with stride $k$ downsamples the spatial dimensions, followed by global self-attention. To avoid parameter redundancy, UltraGen employs domain-aware LoRA, adapting local attention weights for global modeling via low-rank residuals. After attention, bilinear upsampling and 3D convolution restore the original resolution and temporal coherence.

This design ensures that global dependencies are captured efficiently, with the computational cost of global attention matching that of local attention, and avoids the semantic fragmentation observed in purely local attention schemes.

Cross-Window and Hierarchical Local Attention

While local attention reduces complexity, it introduces boundary artifacts due to limited inter-window communication. UltraGen addresses this with two mechanisms:

• Cross-Window Attention: Alternates window partitioning between adjacent transformer layers (standard and shifted windows), enabling overlapping regions and direct information flow across window boundaries.

  Figure 3: Cross-window Attention enables hierarchical interaction across neighboring windows between adjacent transformer layers.

• Hierarchical Local Attention (HLA): Introduces an intermediate scale by partitioning the spatial domain into larger, coarse windows and applying local attention within them. This is combined with domain-aware LoRA adaptation and alternating window shifts, further enhancing the model's ability to capture fine-grained motion and maintain consistency for fast-moving small objects.

The outputs of cross-window and hierarchical local attention are fused using a time-aware factor, analogous to the global-local fusion.

Experimental Results

UltraGen is evaluated by extending the Wan-1.3B model to 1080P and 4K resolutions, trained on the UltraVideo dataset. The evaluation employs both standard and novel high-resolution metrics: HD-FVD, HD-MSE, HD-LPIPS, CLIP score, and temporal consistency.

Qualitative Results: UltraGen produces videos with significantly sharper details and better prompt alignment than Wan, HunyuanVideo, and super-resolution pipelines. Competing models exhibit blurring, semantic drift, or over-smoothing, especially at 4K.

Figure 4: Comparison results of existing state-of-the-art video generation methods on 1080P video generation. The red boxes highlight zoomed-in regions, where UltraGen produces the clearest high-resolution videos with the most fine-grained details.

Figure 5: More generated HD videos (1080P and 4K) by UltraGen, demonstrating consistent high-quality outputs across diverse prompts.

Quantitative Results: UltraGen achieves the lowest HD-FVD (214.12 for 1080P, 424.61 for 4K), highest HD-MSE and HD-LPIPS, and best temporal consistency among all native HD generation methods. Notably, UltraGen delivers a 4.78× speedup in 4K inference over Wan, making high-resolution video generation practical for both training and deployment.

Ablation Studies

Ablation experiments confirm the necessity of each architectural component:

• Without global attention: Severe semantic fragmentation and disjoint content.
• Without cross-window or hierarchical attention: Boundary inconsistencies and reduced detail.
• Without domain-aware LoRA: Blurry outputs due to insufficient parameter specialization.
• Swin-Attention replacement: Smooth boundaries but poor semantic coherence across windows.

  Figure 6: Ablation paper on the proposed modules, highlighting the impact of each component on boundary consistency and semantic fidelity.

Additional Qualitative Results

UltraGen consistently generates high-quality 1080P and 4K videos across a wide range of prompts, maintaining both local detail and global coherence.

Figure 7: More qualitative comparisons between UltraGen and existing HD video generation methods.

Figure 8: More 1080P visualization results generated by UltraGen.

Figure 9: More 4K visualization results generated by UltraGen.

Implications and Future Directions

UltraGen demonstrates that hierarchical attention decomposition, spatial compression, and cross-window mechanisms can overcome the quadratic bottleneck of full attention in high-resolution video generation. The architecture is modular and can be integrated with other transformer-based diffusion models, enabling scalable HD and UHD video synthesis. The introduction of high-resolution-specific evaluation metrics (HD-FVD, HD-MSE, HD-LPIPS) provides a more accurate assessment of generative quality at scale.

Potential future directions include:

• Extending hierarchical attention to even longer temporal horizons for minute-scale video generation.
• Integrating UltraGen with retrieval-augmented or multimodal conditioning for controllable video synthesis.
• Further optimizing memory and compute efficiency for deployment on resource-constrained hardware.

Conclusion

UltraGen establishes a new state-of-the-art for native high-resolution video generation, achieving both superior visual fidelity and computational efficiency. Its hierarchical dual-branch attention architecture, spatially compressed global modeling, and cross-window local attention collectively enable end-to-end 1080P and 4K video synthesis, outperforming prior methods in both qualitative and quantitative evaluations. The approach provides a scalable foundation for future advances in high-resolution, long-form video generation and editing.