Caching for Dollars, Not Hits: An Exact Offline Reference for Cloud-Egress Caching and the Crossover That Decides When It Pays

Abstract: When a cache miss fetches from cloud object storage, the bill is per GET request and per byte of egress, not latency. Classic caching minimizes the miss rate, the wrong objective: a rarely but expensively fetched object can cost thousands of times more dollars than a frequently but cheaply fetched one. Generalized-caching theory bounds the miss-cost objective, but no reported benchmark measures how far deployed heuristics sit from the dollar-optimal offline policy on real cloud prices. We supply that reference. For uniform-size page caches with heterogeneous miss costs the offline dollar-optimum is exact in polynomial time via an integral interval linear program -- validated against brute force; variable sizes are NP-hard, so we extend the flow-based offline bound from the hit-ratio objective to dollars (cost-FOO), tight to about four percent. Against this reference we find: (i) a heterogeneity-regret law -- LRU's dollar-regret rises with miss-cost dispersion (Spearman 0.87) while cost-aware GreedyDual cuts it to roughly a tenth; (ii) a contention frontier -- GreedyDual's residual regret collapses to near zero exactly when the budget fits the expensive working set, and is the open slice otherwise; and (iii) a closed-form crossover s* = GET_fee/egress_rate (about 4 KB on S3, 330 B on GCS) that predicts which deployments need dollar-aware caching. On real memcache and CDN traces the price vector alone moves the workload across s*, shifting the regime as predicted. The artifact is a reproducible billing-faithful benchmark; heuristics and bounds it builds on are prior work, credited.

• The paper introduces a cent-exact offline benchmark that minimizes total dollar miss cost by balancing GET fees and egress costs.
• It establishes a closed-form crossover scale (s^ = GET_fee/egress_rate) to determine when cost-aware caching outperforms traditional hit-rate policies.
• Empirical validation shows that cost-aware policies like GDSF significantly reduce dollar regret in high heterogeneity regimes compared to conventional heuristics.

Exact Offline Reference and Characterization of Cloud-Egress Caching

Motivation and Problem Formulation

Public cloud object storage (e.g., S3, GCS, Azure) imposes billing per GET request and by byte egress, not by latency, fundamentally changing the cost calculus for distributed cache design. Traditional cache policies like LRU minimize miss rate, but this frequently disregards dollar cost optimization: infrequently fetched but large objects can accrue drastically higher bills than frequently requested but small objects. The paper establishes that minimizing miss count produces suboptimal results in terms of billing, especially as size and cost heterogeneity rise. Instead, the appropriate metric is the total dollar miss cost, specifically $$\mathrm{Cost} = \sum_i (\mathrm{GET\_fee} + s_i \cdot \mathrm{egress\_rate})$$ summed over misses.

Polynomial-Time Dollar-Optimality and Cost-Aware Benchmarking

The paper introduces an analytical benchmark for offline dollar-optimal caching under realistic cloud billing. For uniform-size caches with heterogeneous cloud miss costs, the exact offline optimum is constructed via an interval linear program with totally unimodular constraints; this produces a cent-exact optimum in polynomial time. For variable-size objects, the authors extend the flow-based fractional optimum (FOO) from hit-ratio to dollar cost, showing the resulting bound is typically within 4% of the true optimum, making regret calculations numerically meaningful.

Crucially, these dollar-optimal references are validated at scale (up to $10^5$ requests) and allow empirical measurement of the optimality gap ("regret") for deployed heuristics, such as LRU, LFU, GreedyDual-Size (GDS/GDSF), and hit-rate Belady, under real and synthetic workloads.

Heterogeneity-Regret Law and Contention Frontier

A key empirical result is the "heterogeneity-regret law," quantified by the access-weighted coefficient of variation $H$ in miss costs. The paper demonstrates that LRU's dollar regret increases monotonically with $H$ (Spearman 0.87), while cost-aware policies like GDSF reduce this penalty by nearly an order of magnitude. As $H$ grows, cost-blindness incurs increasingly higher dollar losses, whereas cost-aware ranking recovers most of this regret.

Figure 1: Heterogeneity-regret law: dollar-regret increases with miss-cost heterogeneity $H$; cost-aware caching mitigates regret.

The paper further identifies the "contention frontier": when cache budget $B$ suffices to fit all expensive objects, GDSF’s residual regret collapses to zero. If $B$ is insufficient, greedy cost-ranking leaves a quantifiable slice of money on the table, reflecting provably suboptimal savings. This reframes miss-cost heterogeneity as a contention problem governed by expensive-object budget pressure.

Closed-Form Crossover and Regime Prediction

The paper offers a precise closed-form crossover scale, $s^ = \mathrm{GET\_fee}/\mathrm{egress\_rate}$, defining the threshold at which dollar-aware caching becomes essential. Below $s^$, misses are GET-fee-dominated and cost heterogeneity is negligible, such that hit-rate caching becomes near-optimal. Above $10^5$0, egress cost dominates and cost-aware ranking is necessary. The crossover is determined solely by the price vector, not the workload. For industry list prices in June 2026, $10^5$1 varies from 330B to 20KB; deployments with a majority of objects above this threshold benefit decisively from dollar-aware caching.

Figure 2: Real Twitter memcache trace ($10^5$2): price vector, via $10^5$3, sets the optimal regime (cost-awareness is negligible when object sizes are below $10^5$4).

The evaluation on real traces (Twitter memcache, Wikipedia CDN) confirms the predictive power of $10^5$5: as price vectors change, workloads move across $10^5$6 and the regime shifts. On small-object traces ($10^5$7), cost awareness adds little; on large-object traces ($10^5$8), cost-aware policies materially improve dollar outcomes.

Experimental Validation

The benchmarking is rigorously validated on both synthetic (Zipf popularity, independent sizes) and real workloads (Twitter memcache, Wikipedia CDN). For real memcache traffic, most objects are below $10^5$9 and dollar-regret for LRU and GDSF are nearly identical; cost-aware enhancements do not buy meaningful savings. For Wikipedia CDN, in the deeply heterogeneous regime ($H$0--$H$1), cost-awareness recovers substantially more dollar-regret as predicted, though even here absolute regret is modest due to low reuse and a heavy-tailed distribution of one-hit-wonders.

The flow-based cost-FOO lower/upper bounds remain tight (median bracket 4%), supporting the robustness of regret calculations for variable-size workloads. Scalability is limited by computation: interval LP scales poorly for dense workloads; min-cost-flow formulation is viable to $H$2 requests but not yet to $H$3+ production scale.

Implications and Future Directions

The results have practical implications: cost-aware caching is only justified for workloads with substantial egress-dominated heterogeneity and where reuse is sufficient for expensive-object contention to matter. The crossover $H$4 provides a deployment rule for cloud cache strategy selection driven by price vectors rather than workload specifics.

From a theoretical standpoint, this paper closes the empirical gap by providing a reproducible dollar-optimal reference for benchmarking deployed heuristics, rather than optimizing competitive ratios. This augments prior work in cost-aware caching, flow-based bounding, and competitive analysis, and identifies real-world traces where suboptimality is negligible, as well as regimes where cost awareness is critical.

Open problems include scaling exact optimizations to larger traces, finding real workloads that hit high heterogeneity, high reuse, and tight budgets simultaneously, and handling adversarial price vector re-tiering.

Conclusion

This paper formally establishes a cent-exact offline dollar-optimum reference for cloud-egress caching and characterizes the regret landscape for common heuristics. By quantifying the crossover point and heterogeneity-regret law, it provides actionable policy selection criteria, demonstrating that dollar-aware caching is only warranted in regimes determined by price vector and object size distribution. The measurement artifact is reproducible, covering both uniform and variable-sized objects, and bridges the gap between theoretical bounds and practical cloud billing optimization (2606.20539).