Transformation Caching

• Transformation caching is a method that exploits redundant computations in iterative models by storing and reusing intermediate outputs to reduce recomputation.
• Hybrid strategies, including rule-based, token-wise, and ODE-adaptive methods, combine direct reuse and low-rank approximations to ensure computational efficiency with minimal quality loss.
• Empirical benchmarks across image, video, and coded caching applications demonstrate speedups from 1.3× to 6× with negligible degradation in performance metrics.

Transformation caching refers to a class of methods that exploit redundancy across sequential computations in iterative, layered models—especially transformers and diffusion architectures—by storing intermediate results ("caches") and strategically reusing or adapting them to increase computational efficiency. These techniques are now central to accelerating diffusion transformers for image, video, and audio generation, as well as being relevant for coded caching in multiaccess networks and recurrent transformer architectures. Recent advances comprehensively analyze not only the temporal but also the spatial and global dynamics of features to optimize where and how transformation caches are deployed.

1. Core Principles and Formal Definitions

Transformation caching targets scenarios where models apply deep transformations iteratively (over timesteps, layers, or tokens), producing activations that evolve smoothly or are highly redundant between steps. Rather than recomputing each transformation for every iteration, the methods cache intermediate outputs—be it entire block outputs, per-token representations, or feature vectors—and, at later steps, substitute computation with (a) direct reuse, (b) low-rank or linear approximation, or (c) hybrid strategies combining both reuse and recalibration.

Key formalism in diffusion transformers: let $x_t$ denote the model state at step $t$. Transformation caching seeks a function $\mathcal{C}$ such that $\tilde{x}_{t} = \mathcal{C}(x_{t-1},\dots)$ closely approximates the result of a fresh forward computation, minimizing induced error: $e_{t} := \| \tilde{x}_t - x_t \|_2$ More generally, for models composed of an ordered set of transformations $\mathcal{G} = g_L\circ\cdots\circ g_1$, one caches at varying granularity: layers, blocks, feature coordinates, or even entire transformation trajectories (Zou et al., 25 Dec 2024, Zou et al., 5 Oct 2024, Zheng et al., 5 Oct 2025, Chu et al., 22 Aug 2025).

2. Caching Strategies: Temporal, Spatial, Hybrid

A rich taxonomy of transformation caching has emerged:

• Rule-based temporal caching: Features from prior steps substituted directly, often by a fixed or dynamic schedule (Ma et al., 3 Jun 2024, Cui et al., 17 Sep 2025).
• Token-wise and spatial-aware caching: Exploits heterogeneity among tokens; only the "least sensitive" or "most redundant" tokens are cached (Zou et al., 5 Oct 2024, Liu et al., 26 May 2025).
• Cluster-driven caching: Clusters tokens spatially at full-compute steps; in partial steps, recomputes only representatives and propagates their features to the rest, yielding up to nearly order-of-magnitude reductions in create-to-token-compute (Zheng et al., 12 Sep 2025).
• Block-wise and multi-granularity schemes: Caching is dynamically determined per transformer block or across different granularities (step, block, CFG) by context-sensitive policies (Wei et al., 18 Aug 2025, Cui et al., 17 Sep 2025).
• Hybrid/dimension-wise ODE caching: Hidden features are modeled as a mixture of ODEs; each coordinate (or cluster thereof) is updated by a solver with locally optimal forecasting/caching (Zheng et al., 5 Oct 2025).
• Aggressive–conservative dual cycles: Alternates high-skipping ("aggressive") steps with correcting ("conservative") steps to bound error accumulation (Zou et al., 25 Dec 2024).

Theoretical analyses explain that conservative caching alone limits acceleration; aggressive-only caching swiftly degrades output due to error accumulation; hybrid schedules (e.g., dual-mode, inflection-aware, or ODE-adaptive) offer near-optimal speed–accuracy trade-offs (Zou et al., 25 Dec 2024, Qiu et al., 7 Mar 2025, Zheng et al., 5 Oct 2025).

3. Error Control, Correction, and Calibration

Unchecked transformation caching can lead to severe error accumulation and drift, necessitating error-minimization mechanisms:

• Low-rank and increment-calibrated correction: Stored activations are corrected by a learned or analytical low-rank increment; channel-aware SVD is used to robustly handle outlier channels (Chen et al., 9 May 2025).
• Gradient and trend-based optimizations: Gradient-optimized cache (GOC) computes and propagates finite-difference corrections, applying them unless feature trajectories enter "inverse gradient" regimes (Qiu et al., 7 Mar 2025). Error-optimized cache (EOC) precomputes per-block "trends" and perturbs cached features accordingly, targeting blocks or steps with large expected errors (Qiu et al., 31 Jan 2025).
• Exposure bias alignment: Feature caching modulates the effective denoising schedule, amplifying exposure bias. EB-Cache compensates with noise scaling and step-adaptive error thresholds to retain the clean generative trajectory (Zou et al., 10 Mar 2025).
• Noise filtering and trajectory analysis: Trajectory-oriented methods (OmniCache) consider the entire denoising path, globally distributing cache reuse at low-curvature, high-similarity segments and applying dynamic filtering to suppress cache-induced noise (Chu et al., 22 Aug 2025).

All advanced methods empirically tune thresholds on error metrics (L1, L2, SNR), often via offline calibration phases, and exploit known statistical regularities in the evolution of model features.

4. Implementation Methodologies and Algorithms

Transformation caching is realized by integrating cache decision policies into inference loops. Typical implementation elements:

• Calibration / offline profiling: Quantifies per-layer, per-token, or per-step redundancy and error statistics; thresholds and schedules (e.g., cache table, clustering, per-step solvers) are extracted (Ma et al., 3 Jun 2024, Chu et al., 22 Aug 2025, Wei et al., 18 Aug 2025).
• Real-time scheduling: At inference, for each step/layer/token, caching or recomputation is decided via precomputed schedules, runtime similarity metrics, or ODE prediction strategies. Representative pseudocode is available in most recent works (Zou et al., 25 Dec 2024, Liu et al., 26 May 2025, Cui et al., 17 Sep 2025, Qiu et al., 7 Mar 2025).
• Hybrid and plug-in design: Most methods require no architectural modification or retraining; cache modules, recalibration transforms, and clustering operate as external wrappers.

Memory overhead is generally modest, requiring storage of cached activations for only the most recent steps or selected tokens/blocks (Chu et al., 22 Aug 2025, Cui et al., 17 Sep 2025). Computation is minimized by both skipping and (when needed) correcting or forecasting features.

5. Empirical Results, Benchmarks, and Trade-Offs

Comprehensive evaluations on ImageNet (DiT-XL/2), FLUX, OpenSora, PixArt-α, and HunyuanVideo demonstrate significant acceleration:

Method	Domain	Speedup (×)	Quality Drop (FID/VBench/etc.)
ToCa (Zou et al., 5 Oct 2024)	img/video	1.93–2.36	<1.0
ClusCa (Zheng et al., 12 Sep 2025)	img/video	4–6	<1% reward loss
ICC+CA-SVD (Chen et al., 9 May 2025)	image	1.45	IS +12, FID <0.06
HyCa (Zheng et al., 5 Oct 2025)	img/video	5.5–6.2	Near-lossless
DuCa (Zou et al., 25 Dec 2024)	img/video	2.48–2.7	<0.3 FID
OmniCache (Chu et al., 22 Aug 2025)	img/video	2–2.5	<0.1
MixCache (Wei et al., 18 Aug 2025)	video	~1.94	LPIPS +0.01–0.03
FastCache (Liu et al., 26 May 2025)	image	1.32–1.37	t-FID −0.07
EOC (Qiu et al., 31 Jan 2025)	image	<1% extra time	2–29% FID improvement (over cache baseline)
GOC (Qiu et al., 7 Mar 2025)	image	0.8–0.82 (rel)	IS +26%, FID −43% (over cache baseline)

On text-to-speech (F5-TTS), SmoothCache can cache up to 50% of steps without loss at high NFE (steps), yielding 1.8× speedup (Sakpiboonchit, 10 Sep 2025). Video models, especially DiTs on Open-Sora and HunyuanVideo, demonstrate end-to-end accelerations exceeding 2× with negligible LPIPS/SSIM/PSNR/loss (Cui et al., 17 Sep 2025, Wei et al., 18 Aug 2025).

6. Applications: Diffusion, Multimodal, Coded Caching

• Diffusion transformers: The dominant application; all recent state-of-the-art speed-ups in text-to-image, text-to-video, and editing tasks leverage transformation caching in some form (Zou et al., 25 Dec 2024, Zou et al., 5 Oct 2024, Zou et al., 10 Mar 2025, Wei et al., 18 Aug 2025, Zheng et al., 5 Oct 2025).
• Transformer architectures for text/audio: TTS systems and LLMs profit from layer/block-level selective caching and compressive cache variants (Sakpiboonchit, 10 Sep 2025, Zhang et al., 2023).
• Coded caching in communications: Transformation methods are used to export shared-link caching schemes (e.g., Maddah-Ali–Niesen PDAs) into multiaccess network settings, preserving coded caching gain, optimizing subpacketization, and supporting privacy transformations (Cheng et al., 2020, Liang et al., 2021).

In coded caching, transformation caching enables the mapping of combinatorial placement/delivery arrays from shared-link to multiaccess/cyclic topologies, preserving optimality and maximum local gain (Cheng et al., 2020).

7. Limitations, Open Problems, and Outlook

Despite significant progress, current transformation caching methods exhibit limitations:

• Error Accumulation: Excessively aggressive reuse leads to rapidly accumulating error; even with corrective mechanisms, some nontrivial quality drop remains under extreme acceleration regimes (Zou et al., 25 Dec 2024, Qiu et al., 7 Mar 2025).
• Static Routers vs. Dynamic Difficulty: Static, input-invariant routers (e.g., in L2C) cannot adapt to per-sample difficulty, possibly underutilizing redundancy (Ma et al., 3 Jun 2024).
• Sparse and Memory-Efficient Attention: Certain token-selection and importance-score approaches (ToCa) require full attention maps, restricting compatibility with FlashAttention/memory-efficient attention. Recent methods (DuCa's V-Caching) alleviate this (Zou et al., 25 Dec 2024).
• Computational/Memory Overhead: Some methods may introduce moderate memory overhead from storing cached activations, though typically under 10% of model size (Chu et al., 22 Aug 2025).

Ongoing research explores adaptive thresholding, integration with fine-tuning/distillation, dynamic per-instance scheduling, and extension to non-diffusion iterative transformers and coded caching for privacy (Wei et al., 18 Aug 2025, Liang et al., 2021). The principle of transformation caching—exploiting structural and temporal redundancy via selective reuse and correction—has become foundational to efficient large-model inference across broad domains.