OTCache: Optimal-Transport Diffusion Caching

• OTCache is a caching framework designed to accelerate diffusion sampling by predicting budget-conditioned schedules through a three-stage process.
• It utilizes a high-budget reference, a low-budget anchor via perceptual optimization, and optimal transport interpolation to create smooth schedule transitions.
• OTCache consistently enhances both inference speed and output fidelity compared to traditional graph-based caching methods in generative models.

Searching arXiv for OTCache and closely related papers to ground the article in current literature. I’m unable to invoke external tools in this interface, so I’m grounding the article strictly in the provided arXiv records and details, centered on the paper explicitly titled “OTCache” (Gao et al., 30 Jun 2026). OTCache denotes a training-free framework for accelerating diffusion sampling through cache schedule prediction. In the paper explicitly titled "OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models" (Gao et al., 30 Jun 2026), the method is presented as an alternative to graph-based caching schemes that optimize shortest-path objectives independently at each inference budget. Its central claim is that cache schedules across budgets should instead be treated as a smooth, budget-conditioned evolution in policy space. OTCache operationalizes that view with a three-stage pipeline: a high-fidelity reference schedule obtained under a conservative budget, a low-budget anchor found by direct end-to-end perceptual optimization, and an Optimal-Transport-inspired interpolation procedure that predicts schedules for intermediate budgets (Gao et al., 30 Jun 2026).

1. Problem setting and motivating failure mode

OTCache is formulated for diffusion and related generative samplers in which caching is used to skip or reuse expensive computations across timesteps. The method starts from the observation that existing graph-based caching approaches, including MeanCache and LeMiCa, model schedule selection as a constrained shortest-path problem on a graph of timesteps. In that formulation, an edge $e=(t \to s)$ carries a surrogate cost such as

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$

and the target schedule is chosen by

$$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$

This design is adequate when reuse intervals are short and local errors behave almost independently. The paper argues that it becomes unreliable in the low-NFE regime, where the number of function evaluations is very small and error propagation is strongly nonlocal (Gao et al., 30 Jun 2026).

The specific critique is the breakdown of the additive independence assumption. Earlier caching decisions alter later states, so total degradation is not well approximated by the sum of independently estimated edge losses. The paper further identifies a second issue: schedules at different budgets are structurally related, but standard graph-based methods solve each budget largely as an isolated combinatorial optimization problem. OTCache treats both issues as geometric rather than merely graph-theoretic. A plausible implication is that schedule quality at a target budget can be improved by enforcing cross-budget consistency instead of re-solving from scratch.

2. Policy-space view and three-stage construction

OTCache reframes scheduling as budget-conditioned schedule evolution. Rather than viewing optimal schedules at different budgets as unrelated discrete objects, it treats them as different-resolution observations of a single underlying policy trajectory. The resulting framework has three stages: reference-schedule construction, anchor search, and target-budget prediction (Gao et al., 30 Jun 2026).

In Stage 1, OTCache obtains a high-fidelity reference schedule under a conservative budget $B_{\mathrm{ref}}$. The paper instantiates this stage with a generic graph-based caching solver, using MeanCache in the experiments, and writes the reference schedule as

$$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$

The reported default is $B_{\mathrm{ref}} = 20$. The rationale is that at sufficiently high budget, the additive surrogate remains stable enough to encode the structural shape of a good schedule.

In Stage 2, OTCache searches for a low-budget anchor schedule at an extreme budget $B_{\mathrm{anc}}$. This stage does not reuse the graph surrogate. Instead, it directly optimizes an end-to-end perceptual objective. If $x_0(\pi)$ is the generated output under schedule $\pi$ and $x_0$ is the full-step, non-cached reference output, the anchor objective is

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$0

The anchor is then defined by

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$1

The default anchor budget is $$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$2. This anchor search is carried out with Optuna using CMA-ES, with warm start from MeanCache at the same budget and optimization in gap space rather than raw timestep space (Gao et al., 30 Jun 2026).

In Stage 3, OTCache predicts schedules for intermediate budgets by interpolating between the two calibrated endpoints rather than by solving a new shortest-path problem. This is the stage where the Optimal-Transport analogy becomes explicit.

3. OT-inspired interpolation and continuous warping

The interpolation stage begins by converting each discrete schedule $$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$3 into a continuous monotone warping function over normalized progress $$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$4. OTCache uses PCHIP, described in the paper as shape-preserving and non-oscillatory, to obtain continuous embeddings for the reference and anchor schedules (Gao et al., 30 Jun 2026).

The core interpolation rule treats these continuous warping curves as quantile-like functions connected by a one-dimensional Wasserstein geodesic. The target-budget schedule is predicted through

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$5

The interpolation weights are

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$6

with

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$7

The two priors encoded here are locality, meaning that the target should be closer to the nearest known budget, and confidence, meaning that the higher-budget reference should be trusted more because it is structurally more stable (Gao et al., 30 Jun 2026).

After interpolation, OTCache must recover a valid discrete schedule. The paper does this with a power-law warping rule,

$$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$8

where $$\mathcal{L}(t \to s) = \| v(x_s, s) - \hat{v}(x_s, s; x_t) \|_p,$$9 controls early-step density and $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$0 rounds and redistributes residuals so that the schedule remains valid and respects the timestep constraint. The ablation reported in the paper finds that, at $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$1, $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$2 yields the best overall fidelity, whereas $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$3 over-biases toward early steps and $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$4 is less stable (Gao et al., 30 Jun 2026).

4. Search procedure, calibration regime, and computational profile

A frequent misconception is to treat OTCache as an online black-box optimizer that must search anew for every prompt. The paper states the opposite: Stage 2 is an offline calibration procedure, not something run online for every prompt, and online inference requires only the lightweight interpolation and projection of Stage 3 (Gao et al., 30 Jun 2026).

The anchor search itself is deliberately constrained. The implementation uses Optuna, 200 trials, and early stopping patience 50. Two design choices are highlighted as practical accelerants. First, MeanCache at the same budget is used as a warm start, providing both a strong prior and a fallback schedule. Second, the search is performed in first-order difference space, which preserves monotonicity and simplifies optimization. The reported median number of trials required to identify the best schedule is around 50, and most good candidates are found within 100–200 trials. In one analysis, the searched top-1 anchor improves LPIPS by about 25% relative to MeanCache at the same low budget (Gao et al., 30 Jun 2026).

This calibration strategy makes OTCache training-free in the paper’s sense: it does not fit new model weights or require gradient-based retraining of the backbone. The only learned or optimized object is the schedule itself, under an explicit fidelity objective. This suggests that OTCache is best understood as a schedule-prediction framework rather than as a new generative model.

5. Empirical performance

The reported experiments cover FLUX.1 [dev] for text-to-image, Qwen-Image for high-resolution text-to-image generation, and HunyuanVideo for text-to-video generation. Baselines include TeaCache, DBCache, DiCache, ToCa, DuCa, TaylorSeer, LeMiCa, and MeanCache. Metrics include latency and speedup, ImageReward and CLIP Score for text-to-image, VBench for video, and fidelity measures relative to full-step outputs: LPIPS, SSIM, and PSNR. The implementation is in PyTorch on NVIDIA H100 GPUs with FlashAttention; evaluation uses 50 representative prompts from T2V-CompBench (Gao et al., 30 Jun 2026).

Backbone	Example budget	Reported outcome
FLUX.1 [dev]	$$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$5	$$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$6 speedup; LPIPS $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$7 vs. MeanCache $$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$8
Qwen-Image	$$\pi^\star = \arg\min_{\pi \in \mathcal{P}} \sum_{e \in \pi} \mathcal{L}(e)^\gamma \quad \text{s.t. } |\pi| \le \mathcal{B}.$$9	$B_{\mathrm{ref}}$0 speedup; LPIPS $B_{\mathrm{ref}}$1 vs. $B_{\mathrm{ref}}$2
HunyuanVideo	$B_{\mathrm{ref}}$3	$B_{\mathrm{ref}}$4 speedup; LPIPS $B_{\mathrm{ref}}$5 vs. $B_{\mathrm{ref}}$6; VBench $B_{\mathrm{ref}}$7 vs. $B_{\mathrm{ref}}$8

At $B_{\mathrm{ref}}$9, FLUX.1 [dev] shows $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$0 speedup for OTCache versus $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$1 for MeanCache, with LPIPS improving from $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$2 to $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$3 and PSNR from $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$4 to $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$5. For Qwen-Image at the same budget, OTCache reports $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$6 versus $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$7, with LPIPS $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$8, SSIM $$\pi_{\mathrm{ref}} \coloneqq \mathcal{C}_{\mathrm{graph}}(B_{\mathrm{ref}}).$$9, and PSNR $B_{\mathrm{ref}} = 20$0. For HunyuanVideo at $B_{\mathrm{ref}} = 20$1, OTCache and MeanCache both report $B_{\mathrm{ref}} = 20$2 speedup, but LPIPS improves from $B_{\mathrm{ref}} = 20$3 to $B_{\mathrm{ref}} = 20$4, with SSIM $B_{\mathrm{ref}} = 20$5 and PSNR $B_{\mathrm{ref}} = 20$6 (Gao et al., 30 Jun 2026).

The abstract summarizes the headline acceleration as $B_{\mathrm{ref}} = 20$7, $B_{\mathrm{ref}} = 20$8, and $B_{\mathrm{ref}} = 20$9 on FLUX.1 [dev], Qwen-Image, and HunyuanVideo, respectively, while stating that generation fidelity is consistently improved over state-of-the-art caching baselines. The ablations further report that $B_{\mathrm{anc}}$0 is the best anchor budget among the values tested and that smaller anchors, such as $B_{\mathrm{anc}}$1 or $B_{\mathrm{anc}}$2, degrade interpolation quality (Gao et al., 30 Jun 2026).

6. Limitations, interpretive boundaries, and name ambiguity

The paper identifies several practical limitations. OTCache relies on a good offline anchor calibration for a given backbone and anchor budget. Its interpolation model is based on a one-dimensional monotone warping representation, which the paper presents as useful but still a modeling simplification. Schedule quality depends on the quality of the two endpoints; if the anchor is poor, interpolation quality degrades. Stage 2 also introduces nonzero offline search cost, even though that cost is capped and moved out of the online path (Gao et al., 30 Jun 2026).

These limitations help delimit what OTCache does not claim. It is not a universal replacement for graph-based methods; rather, it uses graph-based caching as the source of the high-budget reference schedule. It is also not a per-prompt adaptive optimizer at inference time. Its contribution lies in coupling two calibrated schedules through a geometry-aware interpolation rule that is intended specifically to address low-NFE failure modes.

The name itself is not unique across the broader caching literature. In another arXiv paper, "Online Tree Caching" is explicitly abbreviated as OTCache and denotes a rooted-tree online caching model with subtree dependencies and competitive ratio $B_{\mathrm{anc}}$3 (Bienkowski et al., 2016). Separately, the provided details for "Automation of application-level caching in a seamless way" describe an automated application-level caching approach, implemented as the APLCache framework, under the OTCache name in a web-application setting (Mertz et al., 2020). In current usage, however, the paper explicitly titled "OTCache" refers to the Optimal-Transport-inspired diffusion caching framework introduced in 2026 (Gao et al., 30 Jun 2026).