BlockVid: Block Diffusion for High-Quality and Consistent Minute-Long Video Generation (2511.22973v1)

Abstract: Generating minute-long videos is a critical step toward developing world models, providing a foundation for realistic extended scenes and advanced AI simulators. The emerging semi-autoregressive (block diffusion) paradigm integrates the strengths of diffusion and autoregressive models, enabling arbitrary-length video generation and improving inference efficiency through KV caching and parallel sampling. However, it yet faces two enduring challenges: (i) KV-cache-induced long-horizon error accumulation, and (ii) the lack of fine-grained long-video benchmarks and coherence-aware metrics. To overcome these limitations, we propose BlockVid, a novel block diffusion framework equipped with semantic-aware sparse KV cache, an effective training strategy called Block Forcing, and dedicated chunk-wise noise scheduling and shuffling to reduce error propagation and enhance temporal consistency. We further introduce LV-Bench, a fine-grained benchmark for minute-long videos, complete with new metrics evaluating long-range coherence. Extensive experiments on VBench and LV-Bench demonstrate that BlockVid consistently outperforms existing methods in generating high-quality, coherent minute-long videos. In particular, it achieves a 22.2% improvement on VDE Subject and a 19.4% improvement on VDE Clarity in LV-Bench over the state of the art approaches. Project website: https://ziplab.co/BlockVid. Inferix (Code): https://github.com/alibaba-damo-academy/Inferix.

• The paper introduces a semi-autoregressive block diffusion framework that segments videos into chunks to mitigate error propagation over extended sequences.
• It combines semantic sparse KV caching, block forcing training, and progressive cosine noise scheduling to enhance temporal coherence and perceptual quality.
• Experimental results on LV-Bench show significant improvements in Video Drift Error metrics, establishing new state-of-the-art benchmarks in video synthesis.

BlockVid: A Semi-Autoregressive Block Diffusion Framework for Minute-Long Video Generation

Introduction and Motivation

The BlockVid framework addresses two persistent bottlenecks in long-form video generation: accumulation error induced by key-value (KV) cache over extended sequences, and the absence of evaluation benchmarks that measure long-range coherence and perceptual stability. Long video synthesis models have been historically constrained by either the inefficiency and length rigidity of pure diffusion models or the weak fidelity and lack of parallelism inherent to autoregressive (AR) approaches. BlockVid introduces a semi-autoregressive block diffusion mechanism that merges these paradigms to generate minute-long, temporally coherent videos chunk by chunk, ameliorating error propagation while enabling variable-length decoding and parallel inference. This is accomplished by three core contributions: semantic-aware sparse KV caching, chunk-level noise scheduling and shuffling, and an effective Block Forcing training regime—all evaluated on a dedicated benchmark, LV-Bench, with new Video Drift Error (VDE) metrics.

Block Diffusion Architecture

BlockVid's blockwise generation decouples global sequence modeling from local chunk synthesis. Input videos are segmented into chunks (each with temporal and spatial dimensions compressed via a causal VAE). Within each chunk, a latent representation is denoised via a block diffusion transformer, while generation of the next chunk is conditioned on both a local KV cache and a globally assembled context. The global context is retrieved from prior chunks based on prompt embedding similarity, utilizing a semantic sparse KV caching mechanism.

Figure 1: Overview of BlockVid's semi-autoregressive framework: chunk generation is conditioned on both local KV cache and a globally retrieved semantic context, updated with salient tokens upon each chunk's completion.

Semantic sparse KV caching restricts the memory footprint by storing only the most salient historical tokens (top-k by probe-aggregated attention scores), and retrieving those most semantically aligned with the current generation prompt. This enables efficient long-range conditioning and largely mitigates accumulation error, since spurious or redundant context is systematically pruned.

Block Forcing Training and Error Mitigation

Exposure bias and drift are endemic in long video generation, due to the gap between ground-truth conditioning (teacher forcing) during training and self-rolled predictions at inference. To address this, BlockVid leverages Self Forcing—a sequence-level adversarial loss that matches model-generated distributions to real videos. This ensures exposure to its own errors and enforces realism throughout the sequence.

However, Self Forcing alone does not guarantee cross-chunk semantic and temporal alignment. Block Forcing augments this by supervising the model under a stochastic interpolant flow-matching regime, penalizing deviations both in denoising velocity and semantic similarity with top-l historical references. The final training loss thus combines sequence-level realism (Self Forcing) with chunk-level fidelity and semantic anchoring (Block Forcing), reducing both drift and exposure bias accumulation.

Chunk-Level Noise Scheduling and Shuffling

Temporal drift and abrupt transitions arise when error propagation between chunks is not adequately controlled. To counteract this, BlockVid adopts a progressive cosine noise schedule at both training and inference time, with chunk noise levels increasing smoothly from early to late segments. This ensures that initial chunks establish reliable context, while later ones avoid deterministic collapse. At inference, local chunk-boundary noise shuffling is applied; suffix and prefix frames of adjacent chunks are locally permuted to introduce shared stochasticity and prevent overfitting at boundaries.

Benchmarking: LV-Bench and Video Drift Error Metrics

BlockVid's capabilities are rigorously quantified on LV-Bench—a novel dataset comprising 1,000 minute-long videos, annotated at 2–5 second intervals via a human-in-the-loop pipeline employing GPT-4o for fine-grained captioning. LV-Bench provides object class diversity and reliable per-chunk semantic spans.

Assessment employs five VDE metrics: Clarity, Motion, Aesthetic, Background, and Subject drift, each implemented as a weighted sum of relative deviations from the initial chunk's score (using domain-specific predictors for sharpness, optical flow, aesthetics, background staticness, and subject identity). These directly measure error accumulation, drift, and perceptual collapse over minute-long horizons.

Experimental Results and Ablation Studies

BlockVid achieves strong numerical improvements over prior block diffusion and AR baselines. On LV-Bench, it yields a 22.2% improvement in VDE Subject and a 19.4% improvement in VDE Clarity over the strongest baseline (SkyReels-V2-DF-1.3B). It sets new state-of-the-art across nearly all metrics on both LV-Bench and VBench, with highest subject consistency, background stability, motion smoothness, and overall perceptual quality. Ablation studies confirm the critical impact of:

• Semantic sparse KV caching (particularly at high attention thresholds, $\tau=0.98$)
• Block Forcing in addition to Self Forcing
• Progressive cosine noise scheduling and chunk-boundary shuffling
• Multi-stage post-training using long videos ($\geq 50$ s) for extrapolation.

  Figure 2: Comparison between BlockVid and baseline methods showing robust suppression of accumulation error across minute-long sequences.

  

  Figure 3: Qualitative results depicting high-fidelity, temporally consistent video synthesis with BlockVid (DJ scenario, minute-long).

  

  Figure 4: Architectural and compositional stability achieved over extended temporal spans (Victorian mansion scenario).

  

  Figure 5: Successful long-term identity and scene maintenance (auto shop social scene).

  

  Figure 6: Preservation of aesthetic dynamics and lighting across a minute (nighttime bar scene).

  

  Figure 7: Scene, background, and reflection stability in a challenging automotive rain-lit scenario.

  

  Figure 8: Equipment and environment stability in a modern gym, showing minimal drift over a minute.

Implications and Future Directions

BlockVid demonstrates that minute-long, highly coherent video synthesis is possible with semi-AR block diffusion, provided that accumulation error is explicitly controlled. The implications for world model construction, agentic simulation, and next-generation multimodal simulators are considerable: the framework enables scalable, variable-length, realistic sequence generation, opening doors for AI agents to operate in visually complex environments over extended horizons.

Practically, semantic sparse KV caching and chunk-based noise scheduling could generalize to other autoregressive modeling domains, including language, audio, and multimodal fusion. Future avenues include handling scene composition and multi-shot transitions, extending LV-Bench, and incorporating 3D-aware or physics-grounded generative priors to further enhance scenario realism and consistency.

Conclusion

BlockVid introduces a robust block diffusion approach for minute-long video generation, resolving KV-cache accumulation error and enabling consistent, high-fidelity synthesis through semantic sparse context selection, advanced training strategies, and noise scheduling innovations. Its superior performance across coherence-aware and perceptual quality metrics is demonstrated on LV-Bench and VBench. These findings suggest that the semi-autoregressive block diffusion paradigm, properly regularized and context-managed, is a strong foundation for event-long video modeling and world simulation.