---
title: 'Approximate Caching: Methods & Analysis'
url: https://www.emergentmind.com/topics/approximate-caching
type: topic
---

# Approximate Caching: Methods & Analysis

Approximate caching, as the surveyed literature presents it, is a broad family of caching methods in which exactness is deliberately relaxed at one or more layers of the caching stack: request observation, lookup semantics, cache-state optimization, or performance analysis. The approximation may take the form of noisy request estimates in online learning [2309.02055], approximate kNN indexes in similarity caching [2107.00957], transformed keys with exact lookup in a reduced space [2112.06671], similarity-threshold reuse in retrieval-augmented generation [2503.05530], reuse of intermediate diffusion states for similar prompts [2312.04429], or analytical surrogates such as Che’s approximation for LRU [1202.3974]. This suggests that approximate caching is best treated as a unifying design principle for obtaining tractability, scalability, or latency reduction while keeping error, regret, or performance loss controlled.

## 1. Scope of the approximation

The literature suggests several recurring approximation targets. Some works approximate what the cache knows about demand, some approximate what counts as a cache hit, some approximate the optimization problem used to populate caches, and others approximate the dynamics of the cache itself for analysis.

| Approximation target | Representative mechanism | Representative papers |
|---|---|---|
| Request observations | Noisy request estimates, sampled requests, approximate counters | [2309.02055] |
| Lookup semantics | Approximate kNN indexes, approximate keys, similarity thresholds | [2107.00957], [2112.06671], [2503.05530] |
| Optimization | SDP relaxation, log-sum-exp smoothing, local optimal assignment | [1910.08291], [1905.01633], [2012.03005] |
| Performance analysis | Che approximation, TTL approximation, RCW, Poisson approximation | [1202.3974], [2309.12149], [1803.03914], [1712.07307] |

Two distinctions recur. First, some systems keep lookup exact but compress the key space. Approximate-key caching does this by transforming the original input $x$ into $x'=\text{approx}(x)$ and then performing exact-match lookup on $x'$ [2112.06671]. Second, some systems retain the original key space but relax the matching rule itself, as in similarity caching or embedding-based query reuse [2107.00957], [2503.05530].

A further distinction is between approximate service semantics and approximate analysis. The former changes runtime behavior, for example by returning previously retrieved documents for a similar query or by reusing a cached intermediate diffusion state. The latter leaves the implemented cache unchanged but replaces exact stochastic analysis with a tractable surrogate, as in Che’s approximation for LRU or RND-TTL for similarity caching [1202.3974], [2309.12149].

## 2. Online control with partial or noisy information

A central line of work studies approximation at the observation layer. “No-Regret Caching with Noisy Request Estimates” develops Noisy-Follow-the-Perturbed-Leader (NFPL), a variant of FPL for settings in which the cache no longer observes exact requests but only noisy estimates [2309.02055]. At each batch of $B$ requests, NFPL replaces the true request vector $\mathbf{r}_t$ with an estimated vector $\hat{\mathbf{r}}_t$, requires unbiasedness $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$, and chooses the next cache allocation according to
\[
\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t),
\qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).
\]
Under bounds on noisy vectors and state diameter, Theorem 1 gives
\[
\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},
\]
with $\eta=\sqrt{\hat{R}\hat{A}T/D}$, hence sublinear regret [2309.02055].

The paper specializes this framework to two sampling regimes. In NFPL-Fix, a fixed number $b$ of requests is sampled from each batch and $\hat{\mathbf{r}}_t=\frac{B}{b}\hat{\mathbf{d}}_t$. In NFPL-Var, each request is sampled independently with probability $f$ and $\hat{\mathbf{r}}_t=\frac{1}{f}\hat{\mathbf{s}}_t$. The corresponding regret bounds are
\[
\mathcal{R}_T(\mathrm{NFPL\textrm{-}Fix}) \le 2\sqrt{2}\,B\sqrt{CT},
\qquad
\mathcal{R}_T(\mathrm{NFPL\textrm{-}Var}) \le 2\sqrt{2}\,\frac{B}{f}\sqrt{CT}.
\]
A notable point is that the bound does not depend on the catalog size $N$; the penalty for noise appears instead as multiplicative factors such as $1/f$ [2309.02055].

The empirical picture is also instructive. On Zipf requests, Akamai CDN traces, and round-robin adversarial traces, the NFPL variants approach OPT as the sampling rate increases. Lower sampling rates produce higher average miss ratios, but on round-robin adversarial workloads noisier NFPL variants can outperform exact FPL because the noise prevents overfitting to misleading past histories [2309.02055]. This is one of the clearest cases in which approximation is not merely tolerated but can be functionally beneficial.

Approximation also enters online control through simplified policies that replace history-dependent optimal decisions with history-independent surrogates. “Keep-Alive Caching for the Hawkes process” studies settings without a fixed-size cache, where the trade-off is between holding cost and miss cost. For Hawkes arrivals, the optimal keep-alive window is history-dependent, but the paper develops history-independent TTL-like policies, including a fixed policy $\tau_{\text{fixed}}=c_{cs}/c_p$ with a $2$-approximation guarantee and an optimized-TTL policy obtained by averaging the optimal history-dependent windows over sample histories [2309.03521]. On Azure Functions traces, optimized-TTL retains approximately $95\%$ of the improvement of the optimal policy and achieves about one fewer cold start per application per day on the treated applications [2309.03521].

## 3. Approximate lookup semantics and semantic reuse

Approximate caching often changes the notion of a hit. In similarity caching, a request for one object may be served by a sufficiently similar object. “Ascent Similarity Caching with Approximate Indexes” introduces AÇAI, which combines approximate kNN indexes over the local cache and the remote catalog with an online mirror ascent algorithm [2107.00957]. For each request, AÇAI uses both indexes to find top-$k$ candidates, assigns local objects cost $c_d(r,o)$ and remote objects cost $c_d(r,o)+c_f$, and selects the combination of $k$ objects with minimum total cost. Cache contents are maintained through a fractional state $y_t$, mirror-ascent updates using subgradients of the caching gain, and randomized rounding such as DepRound or CoupledRounding [2107.00957].

AÇAI provides a $(1-1/e)$-approximation to the optimal static cache allocation in hindsight and sublinear regret under adversarial request processes; with CoupledRounding the regret is $O(\sqrt{T})$, while freezing yields $O(T^{(1+\beta)/2})$ [2107.00957]. On synthetic, SIFT1M, and Amazon workloads, it reports 10–35% higher normalized caching gain than SIM-LRU, CLS-LRU, and QCache, while approximate indexing maintains sub-millisecond query times for million-object catalogs and more than 97% recall [2107.00957]. The approximate index is therefore not a secondary implementation detail but part of the caching policy itself.

Approximate-key caching takes a different route. “Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching” does not search for similar items in the original input space. Instead, it maps many original inputs to the same reduced key space and then uses exact matching on the transformed key [2112.06671]. The cache stores $(x',y)$ pairs with $x'=\text{approx}(x)$, so the cost of lookup stays that of a standard exact-match cache. The resulting approximation error comes from collisions: different inputs may map to the same $x'$ while having different labels. The paper defines the per-key error without control as
\[
e_i = 1-\sum_j p_{ij}^2,
\]
and introduces auto-refresh, which periodically verifies cached outputs by rerunning the classifier and then exponentially backs off the verification interval when the cached class remains correct [2112.06671]. In the best reported setting, prefix$_{10}$ with $\beta=1.5$, the miss rate is cut by half, the error rate is at most $1.4\%$, and verification is needed for only about $3\%$ of queries [2112.06671].

Embedding-level approximate reuse has become prominent in LLM and generative-model systems. “Leveraging Approximate Caching for Faster Retrieval-Augmented Generation” introduces Proximity, an approximate key-value cache for RAG pipelines in which the key is a previous query embedding and the value is the set of retrieved document indices [2503.05530]. A hit occurs if the nearest cached key lies within threshold $\tau$:
\[
d(q,k)\le \tau.
\]
At $\tau=0$ the cache degenerates to a standard exact cache; larger $\tau$ increases hit rate at the cost of possible recall degradation. On MMLU and MedRAG, Proximity reduces retrieval latency by up to $59\%$ while maintaining accuracy, and the detailed evaluation reports up to $70.8\%$ faster retrieval on MedRAG; for $\tau\ge 5$, hit rate reaches $93\%$ on MMLU and $98.4\%$ on MedRAG [2503.05530].

“Approximate Caching for Efficiently Serving Diffusion Models” applies the same semantic-reuse principle to intermediate computation rather than final outputs [2312.04429]. Nirvana stores intermediate noise states from prior generations and, for a similar prompt identified via CLIP text embeddings, resumes denoising from step $K$ instead of starting from pure Gaussian noise. The latency model for a hit is
\[
l_s + \mathcal{C}\frac{N-K}{N} + l_r,
\]
and the compute savings are
\[
f_C = h(K)\frac{K}{N}.
\]
The system couples this with the Least Computationally Beneficial and Frequently Used policy, whose eviction score is $f_i\times K_i$, so that entries yielding larger denoising skips are treated as more valuable [2312.04429]. On two real production workloads, Nirvana reports 21% GPU compute savings, 19.8% end-to-end latency reduction, 19% dollar savings, and 28% higher throughput, with 88% hit rate and quality close to vanilla diffusion according to FID, CLIPScore, PickScore, and a user study [2312.04429].

## 4. Approximate analytical models for cache behavior

Another major meaning of approximate caching is analytical rather than operational: the cache policy is standard, but its performance is approximated by a simpler model. Che, Tung, and Wang proposed a particularly influential approximation for LRU, and Fricker, Robert, and Roberts gave a mathematical explanation of its accuracy in “A versatile and accurate approximation for LRU cache performance” [1202.3974]. Under the Independent Reference Model with popularity law $q(n)$, the hit probability is approximated by
\[
h(n)\approx 1-e^{-q(n)t_C},
\]
where the characteristic time $t_C$ solves
\[
C=\sum_{i=1}^{N}\left(1-e^{-q(i)t_C}\right).
\]
The approximation runs in $O(N)$ time, extends to variable-sized objects through
\[
C=\sum_{i=1}^{N}(1-e^{-q(i)t})\theta(i),
\]
and becomes exact asymptotically for Zipf popularity when $N$ is large and $C/N$ is held fixed [1202.3974].

“A unified approach to the performance analysis of caching systems” generalizes Che’s decoupling principle along three axes: broader eviction and insertion policies, renewal traffic with temporal locality, and interconnected caches [1307.6702]. For LRU under renewal traffic, the paper distinguishes occupancy and hit probability:
\[
p_{\text{in}}(m)=\hat{F}_R(m,T_C), \qquad p_{\text{hit}}(m)=F_R(m,T_C),
\]
and derives analogous approximations for q-LRU, k-LRU, FIFO, RANDOM, and networked caches using leave-copy strategies [1307.6702]. One of its main claims is that the generalized Che-based approach improves over the Poisson approximation in cache networks by accounting for correlations among caches [1307.6702].

The same analytical strategy has been adapted to similarity caches. “Performance Model for Similarity Caching” extends TTL approximations to RND-LRU through the RND-TTL model, where each item has a timer, insertion probability $p_n^i$, and refresh rate $\lambda_n^r$ capturing similarity-induced refreshes [2309.12149]. Its occupancy is written as
\[
o_n=\frac{\mathbb{E}[T_{\mathrm{on}}]}{\mathbb{E}[T_{\mathrm{on}}]+\mathbb{E}[T_{\mathrm{off}}]},
\]
with
\[
\mathbb{E}[T_{\mathrm{off}}]=\frac{1}{\lambda_n^i}, \qquad \mathbb{E}[T_{\mathrm{on}}]=\frac{e^{\lambda_n^r T}-1}{\lambda_n^r},
\]
and the characteristic timer $T$ is chosen through
\[
\sum_{n\in\mathcal{I}} g(\lambda_n^r,\lambda_n^i,T)=C,
\quad
g(x_1,x_2,x_3)=\frac{e^{x_1x_3}-1}{e^{x_1x_3}-1+x_1/x_2}.
\]
The paper provides a fixed-point iteration with damping and sufficient conditions for convergence [2309.12149].

Two further analytical approximations target dynamic and decentralized settings. “Optimized Dynamic Cache Instantiation and Accurate LRU Approximations under Time-varying Request Volume” introduces the Request Count Window model, which keeps objects requested within the last $L$ requests and yields exact RCW expressions such as
\[
A = N-\sum_{i=1}^{N}(1-p_i)^L, \qquad
H = 1-\sum_{i=1}^{N} p_i(1-p_i)^L
\]
for cache-on-first-request under IRM, together with constant-time approximations for Zipf workloads and transient formulas for newly instantiated caches [1803.03914]. “Network Cache Design under Stationary Requests: Exact Analysis and Poisson Approximation” reformulates TTL utility maximization as a convex optimization under non-increasing hazard rate IRTs and then advocates lightweight Poisson-approximate decentralized algorithms that use the exponential-arrival solution structure even when the true arrival process is more complex [1712.07307].

## 5. Approximate optimization in distributed, coded, and strategic caching

Approximation is also used to make otherwise intractable cache-population problems solvable. In geo-distributed erasure-coded storage, “Optimal Caching for Low Latency in Distributed Coded Storage Systems” distinguishes an offline optimal chunk-caching problem from an online approximate scheme driven by measured data popularity and network latency [2012.03005]. The online policy estimates popularity with the Discounting Rate Estimator and chunk latency with EWMA, updates per-item valuation arrays in real time, and restricts re-optimization to a local window of at most $K+1$ items. Its worst-case approximation ratio is
\[
1-\frac{2K-1}{C},
\]
and the per-request complexity is at most $O((K+1)^3K!)$ [2012.03005]. On Amazon S3 experiments with six AWS regions and $K=6$, the online scheme stays within 1.3–2.4% of the optimal scheme for cache sizes 60, 80, 100, and 120 while reducing computation time from up to tens of minutes to a few milliseconds per request [2012.03005].

In D2D-enabled cellular caching, “D2D-Enabled Mobile User Edge Caching: A Multi-Winner Auction Approach” formulates placement as a maximal weighted independent set with cache-conflict constraints, then applies semidefinite programming relaxation to obtain an approximate optimal placement and introduces a heuristic repeated auction, MRAC, to reduce complexity [1910.08291]. The relaxed optimization is
\[
\max_{S\succeq 0}\ \mu^TS\mu
\quad\text{s.t.}\quad \mathrm{tr}(S)=1,\ S_{nn'}=0 \ \forall (E_{nn'}=1),
\]
while MRAC reduces complexity from $O(M^2N^2)$ to $O(N^2)$ and reaches about 98–99% of MOAC’s social welfare in simulations [1910.08291].

Coded caching under heterogeneous demands exposes a different role for approximation. Maddah-Ali and Niesen’s setting with nonuniform demands shows that Highest-Popularity-First, although optimal for a single cache, can be highly suboptimal with multiple caches [1308.0178]. The proposed remedy is grouped coded caching: partition files into groups of comparable popularity, run decentralized coded caching within each group, and allocate memory across groups. The achievable expected rate is upper-bounded by
\[
R(M)\le \min_{\{M_\ell\}:\sum M_\ell=M}\sum_{\ell=1}^{L}\mathbb{E}\!\left[r(M_\ell,N_\ell,|\mathcal{K}_\ell|)\right],
\]
and the resulting memory-rate tradeoff is within a bicriteria $(cL,L)$ factor of optimal under the specified grouping condition [1308.0178].

“Optimization-based Decentralized Coded Caching for Files and Caches with Arbitrary Size” treats heterogeneous file sizes and cache sizes via an exact complementary geometric programming approach and then introduces a lower-complexity approximation by replacing nondifferentiable maxima with log-sum-exp upper bounds [1905.01633]. The generic bound
\[
\max\{x_1,\ldots,x_n\}\le \frac{1}{c}\ln\!\left(\sum_{i=1}^{n} e^{cx_i}\right)
\]
yields smooth objectives for average-load and worst-case-load optimization, and the performance loss can be bounded explicitly by terms such as
\[
L_{\mathrm{avg}}(\mathbf{K},N,\mathbf{V}) \le \frac{1}{c}\sum_{i=1}^{K}\binom{K}{i}\ln i.
\]
This makes the approximation quality tunable through the smoothing parameter $c$ [1905.01633].

Strategic settings use approximate equilibrium concepts rather than approximation of loads or hits. “Selfish Caching Games on Directed Graphs” defines a $\beta$-approximate pure strategy Nash equilibrium by the condition
\[
g_s(\mathbf{x}_s',\mathbf{x}_{-s}^{\beta\text{-NE}})
\le
\beta\cdot g_s(\mathbf{x}_s^{\beta\text{-NE}},\mathbf{x}_{-s}^{\beta\text{-NE}}),
\]
when exact best responses become NP-hard because unequal-sized items induce a knapsack problem [2012.14148]. Under the cloud property and path overlap property, the paper gives a polynomial-time algorithm of complexity $\mathcal{O}(|V||I|)$ and a price-of-anarchy bound
\[
\mathrm{PoA}^{\beta}\ge \frac{1}{1+\alpha\beta}.
\]
Approximate equilibria are therefore used not because selfish caching is intrinsically approximate, but because exact equilibrium computation is computationally prohibitive in realistic heterogeneous instances [2012.14148].

## 6. Guarantees, limits, and recurrent misconceptions

A common misconception is that approximation in caching is synonymous with unguided heuristic behavior. The literature contradicts that view. NFPL carries sublinear regret guarantees under unbiased request estimators [2309.02055]. AÇAI provides both a $(1-1/e)$-approximation to the best static allocation and sublinear regret under adversarial requests [2107.00957]. The online chunk-caching scheme for coded storage has an explicit worst-case approximation ratio [2012.03005]. Approximate PSNE admits bounded inefficiency under stated structural assumptions [2012.14148]. Analytical approximations such as Che’s approximation and RND-TTL come with fixed-point formulations, convergence conditions, and asymptotic exactness or strong empirical validation [1202.3974], [2309.12149].

A second misconception is that approximation must always lower effective performance. The empirical record is more nuanced. In adversarial round-robin traces, noisier NFPL variants can outperform exact FPL because noise reduces overfitting to past request histories [2309.02055]. In Proximity, approximate reuse can sometimes slightly outperform strictly closest-neighbor retrieval, plausibly because of beneficial document overlap [2503.05530]. In the diffusion setting, approximate reuse of intermediate states reduces cost and latency without degrading image quality relative to the reported baselines [2312.04429]. Conversely, the same papers emphasize that overly aggressive approximation can be damaging: high similarity tolerance $\tau$ can sharply reduce MedRAG accuracy, weak approximation functions in AKC produce large initial error before auto-refresh correction, and dissimilar prompts in diffusion serving force fallback to full generation [2503.05530], [2112.06671], [2312.04429].

A third misconception is that approximate caching is only relevant to semantic or ML-intensive workloads. The analytical literature shows otherwise. Approximation is equally central in classical cache analysis, where exact Markov or trace-based evaluation can be infeasible at scale. Fricker, Robert, and Roberts explain why Che’s approximation remains accurate even when the original intuition does not apply, including large heterogeneous catalogs relevant to information-centric networking [1202.3974]. The unified Che-based framework extends this idea to renewal traffic, temporal locality, and cache networks, while RCW and Poisson approximations support time-varying or decentralized control [1307.6702], [1803.03914], [1712.07307].

The cumulative picture is that approximate caching is less a single technique than a recurrent systems strategy: replace exact computation or exact semantics with a controlled surrogate, expose the new trade-off variable—sampling rate, similarity threshold, smoothing parameter, TTL, or approximation function—and then recover guarantees strong enough to make the surrogate operationally meaningful. The surveyed works show that these surrogates can target regret, latency, throughput, cache gain, average load, social welfare, equilibrium efficiency, or analytical tractability, depending on which part of the caching problem is being approximated.

Source: https://www.emergentmind.com/topics/approximate-caching