Motion-Aware Caching for Efficient Autoregressive Video Generation

Abstract: Autoregressive video generation paradigms offer theoretical promise for long video synthesis, yet their practical deployment is hindered by the computational burden of sequential iterative denoising. While cache reuse strategies can accelerate generation by skipping redundant denoising steps, existing methods rely on coarse-grained chunk-level skipping that fails to capture fine-grained pixel dynamics. This oversight is critical: pixels with high motion require more denoising steps to prevent error accumulation, while static pixels tolerate aggressive skipping. We formalize this insight theoretically by linking cache errors to residual instability, and propose MotionCache, a motion-aware cache framework that exploits inter-frame differences as a lightweight proxy for pixel-level motion characteristics. MotionCache employs a coarse-to-fine strategy: an initial warm-up phase establishes semantic coherence, followed by motion-weighted cache reuse that dynamically adjusts update frequencies per token. Extensive experiments on state-of-the-art models like SkyReels-V2 and MAGI-1 demonstrate that MotionCache achieves significant speedups of $\textbf{6.28}\times$ and $\textbf{1.64}\times$ respectively, while effectively preserving generation quality (VBench: $1\%\downarrow$ and $0.01\%\downarrow$ respectively). The code is available at https://github.com/ywlq/MotionCache.

• The paper introduces a motion-aware caching technique that leverages intra-chunk frame differences as a proxy for residual instability to selectively update dynamic regions.
• The methodology employs a coarse-to-fine inference schedule with token-wise adaptive caching, ensuring critical semantic structure is maintained while reducing redundant computation.
• Experimental results show that MotionCache outperforms prior methods, achieving up to 6.28× acceleration on SkyReels-V2 with minimal perceptual degradation.

Motion-Aware Caching for Efficient Autoregressive Video Generation

Motivation and Overview

Autoregressive diffusion-based video generation models are constrained by sequential iterative denoising, which incurs significant computational overhead in practical deployment, especially for high-resolution and long-duration videos. Classical caching-based acceleration schemes, such as TeaCache and FlowCache, employ coarse-grained strategies for skipping redundant denoising steps. However, these approaches neglect the spatial heterogeneity inherent in frame-level motion dynamics: dynamic regions require frequent updates to avoid error accumulation, while stable regions allow aggressive step reduction. The paper "Motion-Aware Caching for Efficient Autoregressive Video Generation" (2605.01725) addresses these limitations by introducing MotionCache, a caching framework with token-wise motion awareness, theoretically grounded in residual instability principles.

Theoretical Foundations

The paper establishes that caching error during feature reuse is strictly determined by residual instability. Specifically, the approximation error at each denoising step for a chunk is proportional to the Euclidean norm difference between true and cached residuals. Since direct computation of residual instability is infeasible during inference, the authors demonstrate that intra-chunk frame difference acts as a mathematically grounded upper bound for residual instability, via Lipschitz continuity assumptions. This proxy enables accurate estimation of token importance, pinpointing dynamic regions that require computation, while deferentially caching static backgrounds. The ranking fidelity between frame difference and true residual change is empirically validated with NDCG scores consistently above 0.94, confirming robustness throughout the diffusion process.

Figure 1: Heterogeneous token update demand and intra-chunk frame discrepancy distributions demonstrate long-tailed, highly non-uniform temporal redundancy, motivating a token-wise caching mechanism.

Methodology: MotionCache Framework

MotionCache employs a coarse-to-fine inference schedule, beginning with an initial warm-up phase that builds global structure via chunk-wise full computation. Once semantic integrity stabilizes, the framework transitions to token-wise adaptive caching. Token importance is calculated as spatial frame difference, normalized within each frame to produce motion weights $\mathcal{W}$. An importance-weighted accumulator tracks the estimated residual change for each token, with computation triggered only when a threshold is exceeded.

Figure 2: MotionCache's token-wise policy dynamically determines computational updates per spatial location, contrasting with coarse-grained chunk-level strategies of TeaCache and FlowCache.

This hierarchical scheduling ensures resource allocation is prioritized to fast-evolving regions, while static tokens exploit cached residuals, minimizing redundant computation without sacrificing visual fidelity. The dual-phase mechanism reflects empirical observation that early denoising is critical for semantic structure, whereas later steps are predominantly concerned with high-frequency details.

Figure 3: Visualization of ground-truth frames and computed importance maps indicates strong spatial correspondence, identifying motion-intensive regions for targeted computation.

Figure 4: Temporal evolution of importance maps $\mathcal{W}$; diffuse distribution in early steps transitions to sharp delineation of dynamic subjects as denoising proceeds.

Experimental Results and Analysis

Comprehensive experiments on state-of-the-art SkyReels-V2 and MAGI-1 autoregressive models demonstrate MotionCache's superiority over existing methods. Quantitatively, on SkyReels-V2, MotionCache achieves a 6.28$\times$ acceleration with negligible perceptual degradation (VBench: $1\%\downarrow$), outperforming both FlowCache (6.26$\times$) and TeaCache (1.89$\times$) in speed and alignment metrics (PSNR, SSIM, LPIPS). On MAGI-1, the fast configuration achieves a 2.07$\times$ speedup with robust quality preservation.

Figure 5: MotionCache sustains high visual fidelity and structural coherence during accelerated video generation, outperforming TeaCache and FlowCache.

Qualitative evaluation reveals that, unlike TeaCache (which generates texture artifacts) and FlowCache (which causes semantic drift and hallucinations), MotionCache maintains critical semantic details and temporal consistency, such as preserving anatomical integrity and avoiding color flicker.

Figure 6: MotionCache preserves structural and semantic details in text-to-video generation for SkyReels-V2 samples, outperforming baselines at comparable speedups.

Figure 7: MotionCache achieves superior fidelity and semantic preservation in MAGI-1 text-to-video generation, especially for complex subjects.

Ablation studies further confirm the necessity of the soft-mapping floor parameter $\alpha$ and Phase 1 duration $K$: $\alpha = 0.6$ achieves the best balance between quality and efficiency, while $\mathcal{W}$0 optimally consolidates semantic structure before transition to fine-grained scheduling.

Implications and Future Directions

The fine-grained caching paradigm established by MotionCache shifts the acceleration regime in autoregressive video synthesis from rigid, coarse-grained policies to a dynamic, spatially adaptive framework. Practically, this enables real-time inference for models that would otherwise be prohibitive, significantly broadening their applicability in high-resolution, long-duration workflows. Theoretically, the residual instability principle and motion-proxy validation offer a foundation for further adaptation to other generative temporal domains (e.g., audio synthesis or multimodal sequence generation).

Future developments might extend the surrogate importance estimation via learned motion predictors or integrate content-aware scheduling for even greater efficiency. Additionally, merging token-wise caching with hardware-friendly pruning or quantization strategies could yield further latency and memory improvements, bridging the gap to interactive, user-facing video generation.

Conclusion

MotionCache introduces a theoretically principled, token-wise motion-aware caching strategy for autoregressive diffusion models, breaking the limitations of prior coarse-grained acceleration methods. The framework achieves substantial speedups without compromising perceptual and structural quality, leveraging intra-chunk frame differences as a robust proxy for computational prioritization. This approach materially advances the efficiency of video generation pipelines and lays the groundwork for scalable, real-time deployment of autoregressive synthesis models (2605.01725).