Approximate Caching: Methods & Analysis

• Approximate caching is a family of methods that relaxes exact cache operations at various layers, such as request observation and lookup, to enhance scalability and efficiency.
• Techniques like noisy estimators, approximate kNN, transformed keys, and similarity thresholds enable controlled trade-offs between error and performance.
• Analytical surrogates like Che’s approximation and RCW models provide tractable performance guarantees, mitigating overfitting and reducing latency.

Approximate caching, as the surveyed literature presents it, is a broad family of caching methods in which exactness is deliberately relaxed at one or more layers of the caching stack: request observation, lookup semantics, cache-state optimization, or performance analysis. The approximation may take the form of noisy request estimates in online learning (Mazziane et al., 2023), approximate kNN indexes in similarity caching (Si-Salem et al., 2021), transformed keys with exact lookup in a reduced space (Finamore et al., 2021), similarity-threshold reuse in retrieval-augmented generation (Bergman et al., 7 Mar 2025), reuse of intermediate diffusion states for similar prompts (Agarwal et al., 2023), or analytical surrogates such as Che’s approximation for LRU (Fricker et al., 2012). This suggests that approximate caching is best treated as a unifying design principle for obtaining tractability, scalability, or latency reduction while keeping error, regret, or performance loss controlled.

1. Scope of the approximation

The literature suggests several recurring approximation targets. Some works approximate what the cache knows about demand, some approximate what counts as a cache hit, some approximate the optimization problem used to populate caches, and others approximate the dynamics of the cache itself for analysis.

Approximation target	Representative mechanism	Representative papers
Request observations	Noisy request estimates, sampled requests, approximate counters	(Mazziane et al., 2023)
Lookup semantics	Approximate kNN indexes, approximate keys, similarity thresholds	(Si-Salem et al., 2021, Finamore et al., 2021, Bergman et al., 7 Mar 2025)
Optimization	SDP relaxation, log-sum-exp smoothing, local optimal assignment	(Zhang et al., 2019, Wang et al., 2019, Liu et al., 2020)
Performance analysis	Che approximation, TTL approximation, RCW, Poisson approximation	(Fricker et al., 2012, Mazziane et al., 2023, Carlsson et al., 2018, Panigrahy et al., 2017)

Two distinctions recur. First, some systems keep lookup exact but compress the key space. Approximate-key caching does this by transforming the original input $x$ into $x'=\text{approx}(x)$ and then performing exact-match lookup on $x'$ (Finamore et al., 2021). Second, some systems retain the original key space but relax the matching rule itself, as in similarity caching or embedding-based query reuse (Si-Salem et al., 2021, Bergman et al., 7 Mar 2025).

A further distinction is between approximate service semantics and approximate analysis. The former changes runtime behavior, for example by returning previously retrieved documents for a similar query or by reusing a cached intermediate diffusion state. The latter leaves the implemented cache unchanged but replaces exact stochastic analysis with a tractable surrogate, as in Che’s approximation for LRU or RND-TTL for similarity caching (Fricker et al., 2012, Mazziane et al., 2023).

2. Online control with partial or noisy information

A central line of work studies approximation at the observation layer. “No-Regret Caching with Noisy Request Estimates” develops Noisy-Follow-the-Perturbed-Leader (NFPL), a variant of FPL for settings in which the cache no longer observes exact requests but only noisy estimates (Mazziane et al., 2023). At each batch of $B$ requests, NFPL replaces the true request vector $\mathbf{r}_t$ with an estimated vector $\hat{\mathbf{r}}_t$, requires unbiasedness $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$, and chooses the next cache allocation according to

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$

Under bounds on noisy vectors and state diameter, Theorem 1 gives

$$\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},$$

with $\eta=\sqrt{\hat{R}\hat{A}T/D}$, hence sublinear regret (Mazziane et al., 2023).

The paper specializes this framework to two sampling regimes. In NFPL-Fix, a fixed number $x'=\text{approx}(x)$0 of requests is sampled from each batch and $x'=\text{approx}(x)$1. In NFPL-Var, each request is sampled independently with probability $x'=\text{approx}(x)$2 and $x'=\text{approx}(x)$3. The corresponding regret bounds are

$x'=\text{approx}(x)$4

A notable point is that the bound does not depend on the catalog size $x'=\text{approx}(x)$5; the penalty for noise appears instead as multiplicative factors such as $x'=\text{approx}(x)$6 (Mazziane et al., 2023).

The empirical picture is also instructive. On Zipf requests, Akamai CDN traces, and round-robin adversarial traces, the NFPL variants approach OPT as the sampling rate increases. Lower sampling rates produce higher average miss ratios, but on round-robin adversarial workloads noisier NFPL variants can outperform exact FPL because the noise prevents overfitting to misleading past histories (Mazziane et al., 2023). This is one of the clearest cases in which approximation is not merely tolerated but can be functionally beneficial.

Approximation also enters online control through simplified policies that replace history-dependent optimal decisions with history-independent surrogates. “Keep-Alive Caching for the Hawkes process” studies settings without a fixed-size cache, where the trade-off is between holding cost and miss cost. For Hawkes arrivals, the optimal keep-alive window is history-dependent, but the paper develops history-independent TTL-like policies, including a fixed policy $x'=\text{approx}(x)$7 with a $x'=\text{approx}(x)$8-approximation guarantee and an optimized-TTL policy obtained by averaging the optimal history-dependent windows over sample histories (Narayana et al., 2023). On Azure Functions traces, optimized-TTL retains approximately $x'=\text{approx}(x)$9 of the improvement of the optimal policy and achieves about one fewer cold start per application per day on the treated applications (Narayana et al., 2023).

3. Approximate lookup semantics and semantic reuse

Approximate caching often changes the notion of a hit. In similarity caching, a request for one object may be served by a sufficiently similar object. “Ascent Similarity Caching with Approximate Indexes” introduces AÇAI, which combines approximate kNN indexes over the local cache and the remote catalog with an online mirror ascent algorithm (Si-Salem et al., 2021). For each request, AÇAI uses both indexes to find top-$x'$0 candidates, assigns local objects cost $x'$1 and remote objects cost $x'$2, and selects the combination of $x'$3 objects with minimum total cost. Cache contents are maintained through a fractional state $x'$4, mirror-ascent updates using subgradients of the caching gain, and randomized rounding such as DepRound or CoupledRounding (Si-Salem et al., 2021).

AÇAI provides a $x'$5-approximation to the optimal static cache allocation in hindsight and sublinear regret under adversarial request processes; with CoupledRounding the regret is $x'$6, while freezing yields $x'$7 (Si-Salem et al., 2021). On synthetic, SIFT1M, and Amazon workloads, it reports 10–35% higher normalized caching gain than SIM-LRU, CLS-LRU, and QCache, while approximate indexing maintains sub-millisecond query times for million-object catalogs and more than 97% recall (Si-Salem et al., 2021). The approximate index is therefore not a secondary implementation detail but part of the caching policy itself.

Approximate-key caching takes a different route. “Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching” does not search for similar items in the original input space. Instead, it maps many original inputs to the same reduced key space and then uses exact matching on the transformed key (Finamore et al., 2021). The cache stores $x'$8 pairs with $x'$9, so the cost of lookup stays that of a standard exact-match cache. The resulting approximation error comes from collisions: different inputs may map to the same $B$0 while having different labels. The paper defines the per-key error without control as

$B$1

and introduces auto-refresh, which periodically verifies cached outputs by rerunning the classifier and then exponentially backs off the verification interval when the cached class remains correct (Finamore et al., 2021). In the best reported setting, prefix$B$2 with $B$3, the miss rate is cut by half, the error rate is at most $B$4, and verification is needed for only about $B$5 of queries (Finamore et al., 2021).

Embedding-level approximate reuse has become prominent in LLM and generative-model systems. “Leveraging Approximate Caching for Faster Retrieval-Augmented Generation” introduces Proximity, an approximate key-value cache for RAG pipelines in which the key is a previous query embedding and the value is the set of retrieved document indices (Bergman et al., 7 Mar 2025). A hit occurs if the nearest cached key lies within threshold $B$6: $B$7 At $B$8 the cache degenerates to a standard exact cache; larger $B$9 increases hit rate at the cost of possible recall degradation. On MMLU and MedRAG, Proximity reduces retrieval latency by up to $\mathbf{r}_t$0 while maintaining accuracy, and the detailed evaluation reports up to $\mathbf{r}_t$1 faster retrieval on MedRAG; for $\mathbf{r}_t$2, hit rate reaches $\mathbf{r}_t$3 on MMLU and $\mathbf{r}_t$4 on MedRAG (Bergman et al., 7 Mar 2025).

“Approximate Caching for Efficiently Serving Diffusion Models” applies the same semantic-reuse principle to intermediate computation rather than final outputs (Agarwal et al., 2023). Nirvana stores intermediate noise states from prior generations and, for a similar prompt identified via CLIP text embeddings, resumes denoising from step $\mathbf{r}_t$5 instead of starting from pure Gaussian noise. The latency model for a hit is

$\mathbf{r}_t$6

and the compute savings are

$\mathbf{r}_t$7

The system couples this with the Least Computationally Beneficial and Frequently Used policy, whose eviction score is $\mathbf{r}_t$8, so that entries yielding larger denoising skips are treated as more valuable (Agarwal et al., 2023). On two real production workloads, Nirvana reports 21% GPU compute savings, 19.8% end-to-end latency reduction, 19% dollar savings, and 28% higher throughput, with 88% hit rate and quality close to vanilla diffusion according to FID, CLIPScore, PickScore, and a user study (Agarwal et al., 2023).

4. Approximate analytical models for cache behavior

Another major meaning of approximate caching is analytical rather than operational: the cache policy is standard, but its performance is approximated by a simpler model. Che, Tung, and Wang proposed a particularly influential approximation for LRU, and Fricker, Robert, and Roberts gave a mathematical explanation of its accuracy in “A versatile and accurate approximation for LRU cache performance” (Fricker et al., 2012). Under the Independent Reference Model with popularity law $\mathbf{r}_t$9, the hit probability is approximated by

$\hat{\mathbf{r}}_t$0

where the characteristic time $\hat{\mathbf{r}}_t$1 solves

$\hat{\mathbf{r}}_t$2

The approximation runs in $\hat{\mathbf{r}}_t$3 time, extends to variable-sized objects through

$\hat{\mathbf{r}}_t$4

and becomes exact asymptotically for Zipf popularity when $\hat{\mathbf{r}}_t$5 is large and $\hat{\mathbf{r}}_t$6 is held fixed (Fricker et al., 2012).

“A unified approach to the performance analysis of caching systems” generalizes Che’s decoupling principle along three axes: broader eviction and insertion policies, renewal traffic with temporal locality, and interconnected caches (Martina et al., 2013). For LRU under renewal traffic, the paper distinguishes occupancy and hit probability: $\hat{\mathbf{r}}_t$7 and derives analogous approximations for q-LRU, k-LRU, FIFO, RANDOM, and networked caches using leave-copy strategies (Martina et al., 2013). One of its main claims is that the generalized Che-based approach improves over the Poisson approximation in cache networks by accounting for correlations among caches (Martina et al., 2013).

The same analytical strategy has been adapted to similarity caches. “Performance Model for Similarity Caching” extends TTL approximations to RND-LRU through the RND-TTL model, where each item has a timer, insertion probability $\hat{\mathbf{r}}_t$8, and refresh rate $\hat{\mathbf{r}}_t$9 capturing similarity-induced refreshes (Mazziane et al., 2023). Its occupancy is written as

$\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$0

with

$\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$1

and the characteristic timer $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$2 is chosen through

$\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$3

The paper provides a fixed-point iteration with damping and sufficient conditions for convergence (Mazziane et al., 2023).

Two further analytical approximations target dynamic and decentralized settings. “Optimized Dynamic Cache Instantiation and Accurate LRU Approximations under Time-varying Request Volume” introduces the Request Count Window model, which keeps objects requested within the last $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$4 requests and yields exact RCW expressions such as

$\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$5

for cache-on-first-request under IRM, together with constant-time approximations for Zipf workloads and transient formulas for newly instantiated caches (Carlsson et al., 2018). “Network Cache Design under Stationary Requests: Exact Analysis and Poisson Approximation” reformulates TTL utility maximization as a convex optimization under non-increasing hazard rate IRTs and then advocates lightweight Poisson-approximate decentralized algorithms that use the exponential-arrival solution structure even when the true arrival process is more complex (Panigrahy et al., 2017).

5. Approximate optimization in distributed, coded, and strategic caching

Approximation is also used to make otherwise intractable cache-population problems solvable. In geo-distributed erasure-coded storage, “Optimal Caching for Low Latency in Distributed Coded Storage Systems” distinguishes an offline optimal chunk-caching problem from an online approximate scheme driven by measured data popularity and network latency (Liu et al., 2020). The online policy estimates popularity with the Discounting Rate Estimator and chunk latency with EWMA, updates per-item valuation arrays in real time, and restricts re-optimization to a local window of at most $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$6 items. Its worst-case approximation ratio is

$\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$7

and the per-request complexity is at most $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$8 (Liu et al., 2020). On Amazon S3 experiments with six AWS regions and $\mathbb{E}[\hat{\mathbf{r}}_t]=\mathbf{r}_t$9, the online scheme stays within 1.3–2.4% of the optimal scheme for cache sizes 60, 80, 100, and 120 while reducing computation time from up to tens of minutes to a few milliseconds per request (Liu et al., 2020).

In D2D-enabled cellular caching, “D2D-Enabled Mobile User Edge Caching: A Multi-Winner Auction Approach” formulates placement as a maximal weighted independent set with cache-conflict constraints, then applies semidefinite programming relaxation to obtain an approximate optimal placement and introduces a heuristic repeated auction, MRAC, to reduce complexity (Zhang et al., 2019). The relaxed optimization is

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$0

while MRAC reduces complexity from $$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$1 to $$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$2 and reaches about 98–99% of MOAC’s social welfare in simulations (Zhang et al., 2019).

Coded caching under heterogeneous demands exposes a different role for approximation. Maddah-Ali and Niesen’s setting with nonuniform demands shows that Highest-Popularity-First, although optimal for a single cache, can be highly suboptimal with multiple caches (Niesen et al., 2013). The proposed remedy is grouped coded caching: partition files into groups of comparable popularity, run decentralized coded caching within each group, and allocate memory across groups. The achievable expected rate is upper-bounded by

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$3

and the resulting memory-rate tradeoff is within a bicriteria $$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$4 factor of optimal under the specified grouping condition (Niesen et al., 2013).

“Optimization-based Decentralized Coded Caching for Files and Caches with Arbitrary Size” treats heterogeneous file sizes and cache sizes via an exact complementary geometric programming approach and then introduces a lower-complexity approximation by replacing nondifferentiable maxima with log-sum-exp upper bounds (Wang et al., 2019). The generic bound

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$5

yields smooth objectives for average-load and worst-case-load optimization, and the performance loss can be bounded explicitly by terms such as

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$6

This makes the approximation quality tunable through the smoothing parameter $$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$7 (Wang et al., 2019).

Strategic settings use approximate equilibrium concepts rather than approximation of loads or hits. “Selfish Caching Games on Directed Graphs” defines a $$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$8-approximate pure strategy Nash equilibrium by the condition

$$\mathbf{x}_t = M(\hat{\mathbf{r}}_{1:t-1} + \boldsymbol{\gamma}_t), \qquad \boldsymbol{\gamma}_t \sim \mathrm{Uniform}([0,\eta]^N).$$9

when exact best responses become NP-hard because unequal-sized items induce a knapsack problem (Ma et al., 2020). Under the cloud property and path overlap property, the paper gives a polynomial-time algorithm of complexity $$\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},$$0 and a price-of-anarchy bound

$$\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},$$1

Approximate equilibria are therefore used not because selfish caching is intrinsically approximate, but because exact equilibrium computation is computationally prohibitive in realistic heterogeneous instances (Ma et al., 2020).

6. Guarantees, limits, and recurrent misconceptions

A common misconception is that approximation in caching is synonymous with unguided heuristic behavior. The literature contradicts that view. NFPL carries sublinear regret guarantees under unbiased request estimators (Mazziane et al., 2023). AÇAI provides both a $$\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},$$2-approximation to the best static allocation and sublinear regret under adversarial requests (Si-Salem et al., 2021). The online chunk-caching scheme for coded storage has an explicit worst-case approximation ratio (Liu et al., 2020). Approximate PSNE admits bounded inefficiency under stated structural assumptions (Ma et al., 2020). Analytical approximations such as Che’s approximation and RND-TTL come with fixed-point formulations, convergence conditions, and asymptotic exactness or strong empirical validation (Fricker et al., 2012, Mazziane et al., 2023).

A second misconception is that approximation must always lower effective performance. The empirical record is more nuanced. In adversarial round-robin traces, noisier NFPL variants can outperform exact FPL because noise reduces overfitting to past request histories (Mazziane et al., 2023). In Proximity, approximate reuse can sometimes slightly outperform strictly closest-neighbor retrieval, plausibly because of beneficial document overlap (Bergman et al., 7 Mar 2025). In the diffusion setting, approximate reuse of intermediate states reduces cost and latency without degrading image quality relative to the reported baselines (Agarwal et al., 2023). Conversely, the same papers emphasize that overly aggressive approximation can be damaging: high similarity tolerance $$\mathcal{R}_T(\mathrm{NFPL}) \le 2\sqrt{\hat{R}\hat{A}DT},$$3 can sharply reduce MedRAG accuracy, weak approximation functions in AKC produce large initial error before auto-refresh correction, and dissimilar prompts in diffusion serving force fallback to full generation (Bergman et al., 7 Mar 2025, Finamore et al., 2021, Agarwal et al., 2023).

A third misconception is that approximate caching is only relevant to semantic or ML-intensive workloads. The analytical literature shows otherwise. Approximation is equally central in classical cache analysis, where exact Markov or trace-based evaluation can be infeasible at scale. Fricker, Robert, and Roberts explain why Che’s approximation remains accurate even when the original intuition does not apply, including large heterogeneous catalogs relevant to information-centric networking (Fricker et al., 2012). The unified Che-based framework extends this idea to renewal traffic, temporal locality, and cache networks, while RCW and Poisson approximations support time-varying or decentralized control (Martina et al., 2013, Carlsson et al., 2018, Panigrahy et al., 2017).

The cumulative picture is that approximate caching is less a single technique than a recurrent systems strategy: replace exact computation or exact semantics with a controlled surrogate, expose the new trade-off variable—sampling rate, similarity threshold, smoothing parameter, TTL, or approximation function—and then recover guarantees strong enough to make the surrogate operationally meaningful. The surveyed works show that these surrogates can target regret, latency, throughput, cache gain, average load, social welfare, equilibrium efficiency, or analytical tractability, depending on which part of the caching problem is being approximated.