Error-Aware Dynamic Cache Windows

• The paper introduces an adaptive caching algorithm that dynamically adjusts window sizes in each attention block using real-time ℓ₂ error metrics.
• It leverages both output-space and attention-map error measurements to monitor the denoising trajectory and maintain quality during video generation.
• Empirical results show reduced multiply–accumulate operations and lower latency, achieving computational efficiency without sacrificing visual fidelity.

Error-Aware Dynamic Cache Windows (EDCW) constitute an adaptive algorithmic approach for runtime memory reuse in attention modules of diffusion transformers, specifically tailored for video generation tasks. EDCW, introduced as part of the UniCP framework, dynamically adjusts cache window sizes at the granularity of attention blocks and timesteps by explicitly monitoring the instantaneous error between cached and current values. This mechanism addresses the challenge of error non-uniformity in the denoising trajectory, notably the presence of U-shaped error curves and unpredictable error spikes, thereby enabling significant computational efficiency while maintaining rigorous fidelity guarantees (Sun et al., 6 Feb 2025).

1. Background and Motivation

Diffusion transformers for video generation are computationally intensive, largely due to the quadratic complexity of attention computations across dense temporal and spatial domains. Prior work, including PAB and related fixed-window caching schemes, observed that the per-block difference between attention outputs at consecutive denoising steps (measured via a suitable distance metric) exhibits a characteristic U-shaped curve: errors are elevated at the initial and terminal intervals of the process and minimized near the midpoint. Fixed caching intervals fail to adapt to these dynamics and often lead to pronounced quality degradation during error spikes and at the curve's extremities. EDCW resolves this by adaptively adjusting the cache interval, thereby ensuring the error induced by caching remains within a user-specified threshold at all times (Sun et al., 6 Feb 2025).

2. Error Metric Formulation

At the core of EDCW are two distinct ℓ₂-based error metrics for quantifying the discrepancy between the current computation at timestep $t$ and a previously cached instance at timestep $t-k$ for block $b$:

• Output-space error: $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$, where $o_{b,t} \in \mathbb{R}^{H \times W \times C}$ denotes the post-attention output tensor.
• Attention-map error: $$e_{b,t}^{(a,k)} = \lVert A_{b,t} - A_{b,t-k} \rVert_2$$, where $A_{b,t} \in \mathbb{R}^{N \times N}$ is the raw attention map.

These metrics are efficiently computed and directly reflect the inferential error introduced by cache reuse at varying window sizes, thereby enabling fine-grained control over model output fidelity (Sun et al., 6 Feb 2025).

3. Dynamic Caching Window Mechanism

For each attention block and timestep, EDCW iteratively identifies the maximal cache window $W_{b,t}$ (i.e., how many preceding steps can be reused) such that the corresponding error does not exceed a pre-set threshold $\delta_{b,t}$. The search proceeds as follows:

1. For $k = 1$ to maximum window size $t-k$0, compute $t-k$1. If $t-k$2, set $t-k$3 and cache the full output ($t-k$4).
2. If above is not satisfied, repeat for $t-k$5 and cache just the attention map if acceptable ($t-k$6).
3. If neither condition holds within $t-k$7, no caching is performed for this step/block ($t-k$8), and pruning is triggered.

The pseudocode is concretely specified in the original work. The window size $t-k$9 is typically set to cover the flat section of the error curve (often 10–15 steps), and $b$0 may be set as constant or annealed; five settings $b$1–$b$2 for $b$3 were empirically evaluated. Initial steps default to full computation due to the absence of historical cache (Sun et al., 6 Feb 2025).

4. Integration with UniCP Pipeline

EDCW operates within the broader UniCP pipeline by populating a binary cache map $b$4 with one of three actions per block and timestep: cache full output, cache attention map, or prune (for cases of excessive error). For cache actions, expensive Q-K-V matrix multiplications are fully or partially skipped, yielding direct computational savings. When pruning is triggered, subsequent mechanisms such as PCA-based Slicing (PCAS) and Dynamic Weight Shift (DWS) dynamically reduce and manage model weights, further optimizing performance with minimal loss to detail fidelity (Sun et al., 6 Feb 2025).

5. Complexity and Performance Analysis

The introduction of EDCW substantially reduces multiply–accumulate operations (MACs) and latency relative to both unoptimized and fixed-window caching baselines. In Open-Sora (with $b$5), total MACs reduced from $b$6 to $b$7 (a $b$8 decrease), speedup was $b$9, and latency dropped from $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$0 to $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$1. For moderate $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$2 settings (e.g., $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$3), visual quality (measured by LPIPS, SSIM, PSNR) quantitatively matches or outperforms existing methods such as PAB and FasterCache. Empirical ablations reveal that "dynamic select" (EDCW) offers the lowest latency without significant loss in VBench or image metrics compared to strict output-level or attention-map caching (Sun et al., 6 Feb 2025).

Method	Latency (s)	VBench (%)	LPIPS
EDCW (dynamic)	49.07	78.17	0.0857
Output cache	49.19	78.23	–
Map cache	49.27	78.26	–

All values are from Table II and associated ablations in (Sun et al., 6 Feb 2025).

6. Comparative Evaluation

EDCW was compared against both PAB and FasterCache across three diffusion transformer models. Under $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$4 (E2 regime), Open-Sora realized a $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$5 speedup, LPIPS of $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$6, SSIM of $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$7, and PSNR of $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$8, consistently outperforming prior caching methods in both efficiency and video generation fidelity. As $$e_{b,t}^{(o,k)} = \lVert o_{b,t} - o_{b,t-k} \rVert_2$$9 increases, MACs and latency monotonically decrease, while VBench, SSIM, and PSNR remain within $o_{b,t} \in \mathbb{R}^{H \times W \times C}$0–$o_{b,t} \in \mathbb{R}^{H \times W \times C}$1 of baseline, confirming robust error–efficiency trade-off characteristics (Sun et al., 6 Feb 2025).

7. Illustrative Examples and Empirical Observations

Figure 2a in (Sun et al., 6 Feb 2025) demonstrates the canonical U-shaped error curve with clear spikes. Figure 2 shows the variable cache routine where window $o_{b,t} \in \mathbb{R}^{H \times W \times C}$2 is dynamically selected per block and timestep. Video samples presented in Figure 3 exhibit maintained color and detail even for the most aggressive error settings ($o_{b,t} \in \mathbb{R}^{H \times W \times C}$3). The dynamic window mechanism robustly adapts to local error fluctuations, validating the EDCW approach across a range of real-world sequences and diffusion models.