---
title: Cost-Aware LFU for Cloud Caching
url: https://www.emergentmind.com/topics/cost-aware-least-frequently-used-lfu-policy
type: topic
---

# Cost-Aware LFU for Cloud Caching

A cost-aware least-frequently-used (LFU) policy is a caching approach for cloud-based systems that incorporates both access frequency and explicit cloud cost models—specifically storage and compute costs—rather than traditional fixed-capacity constraints. Unlike classical cache replacement algorithms that evict entries based purely on recency or historical frequency, the cost-aware LFU evaluates, for each individual item, whether retaining it in the cache minimizes overall operational cost under the cloud’s pay-per-usage paradigm. The policy is fully decomposable across items, provably optimal under stationary access patterns, and achieves near-optimal performance with practical, online frequency estimation.

## 1. Problem Setting and Cost Model

The central context is cloud-based caching where data can be either recomputed on-the-fly or served from cloud storage. For each item $i \in \{1,\ldots,N\}$, the following holds:
- **Request Arrival**: Each $i$ is requested according to a Poisson process at rate $\lambda_{i}$.
- **Compute Cost**: Recomputing (a miss) costs $C_{i}$ per access.
- **Storage Cost**: Storing item $i$ in the cloud cache costs $S_{i}$ per unit time.
- **Capacity Assumption**: There is no fixed storage limit. Operators pay proportional to cache occupancy and duration, distinct from classical caches dominated by up-front capacity investment.
- **Transfer Cost**: Assumed negligible or folded into $C_{i}$.

Empirical access frequency $f_{i}$ is measured as the count of requests in a sliding window $W$ divided by $W$; under steady-state Poisson arrivals, $f_{i} \rightarrow \lambda_{i}$ as $W \rightarrow \infty$ [1312.0499].

## 2. Mathematical Formulation

Let $x_{i} \in \{0,1\}$ indicate whether item $i$ is always cached ($x_{i}=1$) or never cached ($x_{i}=0$). The optimization objective is the long-run average cost per time unit:
$$
C_{\text{total}} = \sum_{i=1}^{N} \left[ \lambda_{i}(1-x_{i})C_{i} + x_{i} S_{i} \right]
$$
- If cached ($x_{i}=1$): Pay $S_{i}$ per unit time; all accesses are hits.
- If not cached ($x_{i}=0$): Pay recompute cost $C_{i}$ per access at rate $\lambda_{i}$.

The problem is separable across items; minimizing $C_{\text{total}}$ reduces to per-item decisions:
$$
\min_{x_{i} \in \{0,1\}} \lambda_{i}(1-x_{i})C_{i} + x_{i}S_{i}
$$

## 3. Cost-Aware LFU Rule and Online Algorithm

Define a per-item utility score:
$$
U(i) = \frac{\lambda_{i} C_{i}}{S_{i}}
$$
- $U(i) > 1$ implies it is cost-effective to cache $i$; otherwise, evict it.

Equivalently, use the threshold:
- If $\lambda_{i} > S_{i}/C_{i}$, set $x_{i}=1$ (cache indefinitely).
- If $\lambda_{i} \leq S_{i}/C_{i}$, set $x_{i}=0$ (never cache).

**Online implementation** uses a sliding window estimator of frequency:
```python
initialize empty count window of length W for each i
every arrival of request for item i at time t:
    record timestamp t in i’s window
    remove timestamps < t−W from i’s window
    estimate f_i = (count of timestamps in window)/W
    if f_i > S_i/C_i and item i not currently cached:
        PUT item i in cache  # pay storage from now on
    if f_i ≤ S_i/C_i and item i currently cached:
        DELETE item i from cache  # stop paying storage
```
This “cost-aware LFU” (Editor's term) policy compares observed frequency $f_{i}$ to $S_{i}/C_{i}$ for each item, diverging from classic LFU by making individual, threshold-based caching decisions absent any global capacity constraint [1312.0499].

## 4. Theoretical Properties

- **Optimality**: In steady-state and with exact $\lambda_{i}$, the per-item rule yields the minimum expected cost. Each item's decision independently selects the lower-cost strategy: recompute on demand, or always store.
- **Sliding Window Approximation**: Using $f_{i}$ as the plug-in estimate (MLE) for $\lambda_{i}$ results in estimation error proportional to $O(1/\sqrt{\lambda_{i}W})$. As $W$ increases, empirical performance converges to the optimal [1312.0499].
- **Full Decomposition**: Absence of cross-item interactions allows the global problem to decompose into $N$ independent, one-dimensional subproblems.

## 5. Empirical Evaluation

The policy was evaluated using both synthetic and real-world traces:
- **Workloads**: Synthetic Zipf-distributed items (e.g., 10,000 movies, 5,000 ads; various $\lambda$) and traces from Netflix (17,000 items, 6 years), YouTube Sci (~252,000 items), and Daum Travel (~9,000 items).
- **Costs**: Simulated with Amazon EC2 and S3 prices—compute $C \approx 7.2 \times 10^{-4}$ USD per chunk, storage $S \approx 4.86 \times 10^{-7}$ USD per chunk-hour.
- **Comparison Policies**: Evaluated against global TTL (one TTL shared across all items), a clairvoyant lower-bound (oracle), and LRU under fixed-size constraint.

**Measured Metrics**:
- Total cost per chunk served,
- Hit ratio (informative),
- Amortized cost per request.

**Principal Results**:
- Cost-aware LFU matches the clairvoyant lower bound closely.
- Delivers 10–20% lower cost than global TTL and up to 30% lower than LRU across all request rates.
- On real traces, individual TTL (per-item, cost-aware) saves 15% over global TTL and 25% over LRU [1312.0499].

| Policy           | Cost Improvement vs. LRU | Cost Improvement vs. Global TTL |
|------------------|-------------------------|---------------------------------|
| Cost-aware LFU   | Up to 30%               | 10–20%                          |
| Individual TTL   | 25% (real traces)       | 15% (real traces)               |

## 6. Parameter Sensitivity and Practical Guidance

- **Impact of Price Ratios**: As $S/C$ rises (storage more expensive than compute), the critical frequency threshold increases ($\lambda^* > S/C$), decreasing the cache population. As $S/C$ falls, more items are retained in cache. This allows adaptable cache sizing without explicit global limits.
- **Sliding Window Size ($W$)**: $W$ must be at least $C/S$ to resolve frequency estimates near the threshold. Empirical tests found $W \approx C/S$ is optimal; substantially larger $W$ increases estimation variance and adaptation lag under non-stationary access, while smaller $W$ leads to suboptimal thresholding.

A plausible implication is that this decomposable, adaptive approach can be directly tuned for cost minimization under dynamic pricing models and variable demand, with empirical validation demonstrating robust performance gains over traditional cache replacement schemes.

## 7. Summary and Significance

The cost-aware LFU policy reformulates cache management for cloud contexts by forgoing the fixed-capacity constraint and integrating explicit cost models with online, per-item access frequency tracking. It admits fully online execution, converges to the per-item optimum as estimator window size grows, adapts naturally to popularity skew and burstiness, and outperforms both size-based (LRU/LFU) and global TTL approaches in terms of realized cost, while nearly matching theoretical lower bounds [1312.0499]. This model is especially salient for cloud systems where elastic resources, item heterogeneity, and fine-grained economics are primary operational considerations.

Source: https://www.emergentmind.com/topics/cost-aware-least-frequently-used-lfu-policy