ReCache: Budget-Aware Diffusion Caching

• ReCache is a budget-aware scheduling method for diffusion-model inference that optimizes recomputation steps to maximize final generation quality under a fixed compute budget.
• It employs a one-shot reinforcement learning approach using the Plackett–Luce and Gumbel-Top-k distributions to select caching steps effectively.
• Empirical results show significant LPIPS reduction and quality improvements in image and video tasks compared to uniform or heuristic caching schedules.

Searching arXiv for the ReCache diffusion-model paper and closely related caching baselines. ReCache is a budget-aware scheduling method for diffusion-model feature caching that treats compute as a direct input rather than an indirect consequence of heuristic thresholds. Introduced for image and video diffusion inference, it assumes a denoising process with $T$ discrete steps and a user-specified budget $k$ full network evaluations, then learns a cache schedule $S \subset \{1,\dots,T\}$ with $|S|=k$ that maximizes final generation quality under that budget. The method is trained with policy gradients, uses generations from uncached inference as matching targets, requires no labelled data, and is designed to be compatible with both feature reuse and feature forecasting mechanisms such as FORA, $\Delta$-DiT, DiCache, TaylorSeer, HiCache, and DPCache (Aliev et al., 4 Jun 2026).

1. Definition and computational setting

ReCache is formulated around the standard diffusion or flow-matching inference loop in which a model $G$ runs $T$ denoising steps. The latent at step $t$ is denoted $x_t$, with $x_T \sim \mathcal{N}(0,I)$ and $k$0 the final output. At each step, a large network $k$1 computes a velocity or noise estimate

$k$2

followed by the solver update

$k$3

The dominant cost is the forward pass through $k$4 at every step (Aliev et al., 4 Jun 2026).

Within this setting, ReCache distinguishes between recomputation steps and cached steps. A schedule $k$5 specifies the $k$6 denoising steps at which the full network is run and selected intermediate activations $k$7 are stored. At steps $k$8, the method reconstructs approximate activations $k$9 cheaply via either direct reuse, which copies from the last cache, or feature forecasting, which extrapolates from several previous caches. The output under schedule $S \subset \{1,\dots,T\}$0 and caching mechanism $S \subset \{1,\dots,T\}$1 is written

$S \subset \{1,\dots,T\}$2

This arrangement makes the schedule itself the optimization target, rather than treating it as a by-product of a hand-tuned error heuristic (Aliev et al., 4 Jun 2026).

The central conceptual inversion is that the user specifies exactly how many full evaluations can be afforded. Existing schedules are described as fixed, such as uniform, or adaptively selected from per-step error heuristics; ReCache instead learns the recomputation schedule that maximizes generation quality for the target budget $S \subset \{1,\dots,T\}$3 (Aliev et al., 4 Jun 2026).

2. Policy parameterization and schedule selection

ReCache casts schedule selection as a one-shot RL problem over $S \subset \{1,\dots,T\}$4-subsets of diffusion steps. Equivalently, the schedule can be represented as binary actions $S \subset \{1,\dots,T\}$5 with

$S \subset \{1,\dots,T\}$6

where $S \subset \{1,\dots,T\}$7 means “cache at $S \subset \{1,\dots,T\}$8.” The policy does not use a standard recursive state evolution; instead, it computes in one shot a distribution over all $S \subset \{1,\dots,T\}$9-subsets based on the budget $|S|=k$0 alone (Aliev et al., 4 Jun 2026).

Budget conditioning is implemented with a small MLP:

$|S|=k$1

where $|S|=k$2 are step-importance scores. ReCache then defines a policy $|S|=k$3 over $|S|=k$4-subsets using the Plackett–Luce top-$|S|=k$5 distribution:

$|S|=k$6

Sampling is performed with Gumbel-Top-$|S|=k$7:

$|S|=k$8

followed by

$|S|=k$9

At inference time, the noise is dropped and the deterministic top-$\Delta$0 of $\Delta$1 is used (Aliev et al., 4 Jun 2026).

This parameterization has two notable consequences. First, one trained policy can be queried at different budgets during inference. Second, schedule selection is decoupled from backpropagation through full diffusion inference, which would otherwise be burdensome. This suggests that ReCache is best understood as a meta-controller over denoising steps rather than a modification of the underlying denoiser itself.

3. Reward design and REINFORCE training

Training combines a fidelity term against uncached inference with a perceptual quality reward. For noise input $\Delta$2, let

$\Delta$3

be the full-inference output and

$\Delta$4

the output produced under schedule $\Delta$5. ReCache defines the single-schedule loss as

$\Delta$6

where $\Delta$7 is instantiated as patchwise LPIPS and $\Delta$8 is a perceptual or quality reward such as HPSv2 for images or VBench for videos. The corresponding RL reward is

$\Delta$9

The uncached output $G$0 serves as the supervised target for computing the fidelity term, while $G$1 scores final perceptual quality (Aliev et al., 4 Jun 2026).

Optimization uses REINFORCE with a leave-one-out baseline and an entropy bonus. The objective minimizes $G$2, and with $G$3 sampled schedules $G$4 the gradient estimate is

$G$5

where

$G$6

The paper’s pseudocode adds the entropy term through

$G$7

Training proceeds by precomputing a dataset of $G$8, sampling budgets $G$9, sampling schedules via Gumbel-Top-$T$0, evaluating the cached generator, and updating $T$1 with Adam using the REINFORCE estimator (Aliev et al., 4 Jun 2026).

A plausible implication is that ReCache optimizes the schedule against end-of-trajectory quality rather than local feature error. That distinction matters because a denoising step that appears unimportant under a per-step approximation metric may still be disproportionately important to downstream perceptual quality.

4. Inference behavior, compatibility, and empirical results

At test time, ReCache is operationally simple: choose a budget $T$2, compute $T$3, select $T$4, and run the generator once under caching mechanism $T$5 with that schedule. No further fine-tuning is needed, and a single policy works across budgets (Aliev et al., 4 Jun 2026).

The method is explicitly described as compatible with both direct feature-reuse and feature-forecasting mechanisms. The paper lists FORA, $T$6-DiT, DiCache, TaylorSeer, HiCache, and DPCache as compatible mechanisms, with the only change being how $T$7 is computed for each sampled schedule (Aliev et al., 4 Jun 2026).

The reported results emphasize same-compute comparisons against scheduling baselines. On FLUX.1-dev with $T$8 and budget $T$9, corresponding to approximately $t$0 FLOPs reduction, Uniform DiCache gives LPIPS $t$1, whereas ReCache + DiCache gives LPIPS $t$2, a $t$3 reduction, with HPS $t$4. Under the same model family, Uniform TaylorSeer $t$5 changes from LPIPS $t$6 to ReCache TaylorSeer $t$7, and Uniform HiCache $t$8 changes from $t$9 to ReCache HiCache $x_t$0 (Aliev et al., 4 Jun 2026).

On Wan2.1 video generation with $x_t$1 and budget $x_t$2, corresponding to approximately $x_t$3 speedup in the detailed summary and described in the abstract as $x_t$4 speedup for one comparison, Uniform TaylorSeer $x_t$5 changes from LPIPS $x_t$6 to ReCache $x_t$7, Uniform HiCache $x_t$8 from $x_t$9 to $x_T \sim \mathcal{N}(0,I)$0, and Uniform DPCache from $x_T \sim \mathcal{N}(0,I)$1 to $x_T \sim \mathcal{N}(0,I)$2. The VBench score increases by $x_T \sim \mathcal{N}(0,I)$3–$x_T \sim \mathcal{N}(0,I)$4 points, with the abstract giving the concrete example of $x_T \sim \mathcal{N}(0,I)$5 to $x_T \sim \mathcal{N}(0,I)$6 over uniform HiCache (Aliev et al., 4 Jun 2026).

On HunyuanVideo with $x_T \sim \mathcal{N}(0,I)$7 and budget $x_T \sim \mathcal{N}(0,I)$8, corresponding to $x_T \sim \mathcal{N}(0,I)$9, Uniform TaylorSeer $k$00 changes from $k$01 to ReCache $k$02, Uniform HiCache $k$03 from $k$04 to $k$05, and Uniform DPCache from $k$06 to $k$07, with VBench increasing by $k$08–$k$09 points (Aliev et al., 4 Jun 2026).

The paper also reports several qualitative properties. Across all models and budgets, ReCache outperforms uniform or heuristic schedules at the same compute, with the largest gains when $k$10 is small. It also states that discovered schedules are nested in $k$11, showing a stable step ranking (Aliev et al., 4 Jun 2026). This suggests that the learned controller identifies a consistent ordering of denoising-step importance, then truncates that ordering according to the available budget.

5. Relation to other cache-reuse mechanisms

The name “ReCache” is used in multiple technical contexts, but the diffusion-model method is distinct in objective and mechanism. In diffusion inference, ReCache learns when to recompute under a fixed budget. By contrast, several recent systems address what to cache, when to refresh, or how to reuse keys and values in other architectures.

System	Domain	Core mechanism
ReCache (Aliev et al., 4 Jun 2026)	Diffusion models	Learns a budget-aware recomputation schedule via REINFORCE
SD-VLA recache gate (Qiu et al., 3 Feb 2026)	Vision-language-action models	Reuses static-token KV cache and refreshes it only when a gate exceeds threshold $k$12
LongLive KV-recache (Yang et al., 26 Sep 2025)	Interactive long video generation	Rebuilds cached states under a new prompt after prompt switches
PRCR (Wang et al., 25 Jun 2026)	Interleaved multimodal reasoning	Rebinds raw visual keys to position-compatible coordinates before cache injection
KV cache recycling (Pandey, 4 Dec 2025)	Decoder-only LLM inference	Reloads cached past key values when a cached prompt is an exact prefix of the new input

In SD-VLA, the recache gate is attached to a static–dynamic token disentanglement framework. At each time step, static tokens can appear only once while dynamic tokens from the last $k$13 frames are stacked. A lightweight gate network

$k$14

decides whether the static KV at level $k$15 should be refreshed or reused, with threshold comparison against $k$16 at inference. This mechanism is intended to reduce repeated attention work in long-horizon robotic control rather than to allocate a fixed compute budget across denoising steps (Qiu et al., 3 Feb 2026).

LongLive addresses prompt switching in frame-level autoregressive video generation. There, naive cache clearing breaks visual continuity and naive cache retention causes prompt inertia. KV-recache discards the old cache at the switch frame and rebuilds it by re-encoding already generated frames under the new prompt, then continues causal decoding. In the reported controlled $k$17 setting with one prompt switch at $k$18, “KV recache” achieves Background $k$19, Subject $k$20, and CLIP $k$21, combining smooth transitions with prompt compliance, at approximately $k$22 extra time on a $k$23 sample (Yang et al., 26 Sep 2025).

PRCR, or Position Rebinding Cache Reuse, tackles replay-free visual revisiting in interleaved multimodal reasoning. It identifies that direct reuse of historical visual KV cache fails because keys remain bound to stale positional context. PRCR therefore stores raw visual KV together with original spatial coordinates, reassigns compatible coordinates, reapplies RoPE to reconstruct keys, and injects the rebound cache into the active decoder cache. On Qwen3-VL-8B with $k$24 selected tokens, the paper reports replay cost of approximately $k$25 versus PRCR cost of approximately $k$26, i.e. more than $k$27 reduction, while matching or slightly outperforming token replay (Wang et al., 25 Jun 2026).

KV cache recycling for small LLMs is again different: it stores cached past key values on CPU, retrieves candidate prompts by sentence-embedding similarity, verifies an exact token-level prefix match, and resumes generation from the cached prefix. In the reported tests on DialoGPT-medium, average generation time changes from $k$28 to $k$29 with average reused tokens $k$30 and average speedup $k$31, while behavior matches baseline when overlap is absent (Pandey, 4 Dec 2025).

These comparisons clarify a common misconception: ReCache in diffusion models is not a generic synonym for KV-cache reuse. It is specifically a learned scheduler over denoising steps, whereas the other systems focus on cache refresh, replay avoidance, prefix reuse, or position-corrected reinsertion.

6. Broader usage of the term and limitations

Outside generative-model inference, “ReCache” or “Reuse Cache” has also been used for hardware cache-management mechanisms. One line of work uses a decoupled tag/data SLLC in heterogeneous CPU–GPU systems and stores data only for lines that have been accessed more than once. In a 32 nm CACTI modeling result, a conventional $k$32 LLC is $k$33, whereas ReCache with $k$34 tag and $k$35 data is $k$36, yielding $k$37 savings, while achieving within $k$38 of the static-partition baseline IPC (Shah et al., 2021). Another line of work uses per-line reuse-distance prediction to decide whether a clean line evicted from an upper-level cache should be copied back into the lower-level LLC, reporting average IPC improvement of $k$39 and up to $k$40 over an LRU plus copy-back-all baseline for STT-MRAM LLC (Wang et al., 2021). These systems share the high-level motif of exploiting reuse, but they are architecturally unrelated to the diffusion-model scheduler.

For the diffusion-model ReCache itself, the paper identifies several strengths and limits. Its strengths are true budget control, direct optimization of final output quality rather than intermediate feature error, adaptation across budgets with a single policy, compatibility with multiple caching mechanisms, and nested schedules that indicate a stable step ranking. Its limitations are that any caching scheme struggles in very low-step regimes with $k$41, because reuse gaps become too large, and that offline RL training is required for each $k$42 pair, though the reported overhead is only a few hundred GPU-hours once (Aliev et al., 4 Jun 2026).

The proposed extensions are correspondingly targeted. The paper suggests incorporating continuous or fractional budgets via temperature-controlled sampling, jointly optimizing both the caching schedule and the caching mechanism, and combining the method with distillation or efficient-solver approaches for larger speedups (Aliev et al., 4 Jun 2026). A plausible implication is that the current formulation separates when to cache from how to cache, and that future systems may collapse these into a single learned control problem over denoising computation.