Distributed Page Cache (DPC) Overview

• Distributed Page Cache (DPC) is a family of architectures that disperses file- or block-based caching over multiple nodes to reduce central server load.
• Research focuses on strategic cache placement, replication methods like PPMM and KS+MLP, and adaptive request routing to improve load balancing.
• DPC studies tackle challenges in storage heterogeneity, coherence, and OS integration, while exploring hybrid memory and coded caching extensions.

Distributed Page Cache (DPC) denotes a family of distributed caching architectures in which file-backed or block-backed data is cached across multiple nodes rather than in a single host-local page cache. In one analytical formulation, a distributed cache system consists of one central server that stores the whole catalog, $m$ co-located caches, and a root node connecting caches and server to users; requests arrive in batches, each cache can serve at most one request per batch, and unmatched requests are served by the central server (Reddy et al., 2018). In a more recent OS-level formulation, DPC is a distributed page cache built on top of CXL 3.0 memory semantics that treats the entire cluster’s main memory as a single cache budget while preserving standard file-system interfaces and semantics (Bergman et al., 21 Apr 2026). Across these formulations, the central questions are how to model the cache as a distributed system, how to place and route data, how to handle skew and heterogeneity, and how to preserve coherence and strong semantics when cached state is physically distributed.

1. Formal models and problem statements

A common analytical starting point models DPC as a single-layer distributed cache in front of a backing store. The system has $m$ caches, catalog size $n = m^{\gamma}$ with $\gamma \ge 1$, cumulative cache storage $M = m^{\mu}$ with $\mu \ge 1$, and per-slot request batch size $\widetilde m = \rho m$, $0 < \rho < 1$. Files are equal sized and normalized to size 1. Requests are i.i.d. within a slot and follow a Zipf distribution $p_i \propto i^{-\beta}$. The key performance metric is the central server transmission rate per slot, denoted $R$, with objective $m$0; the ideal regime is $m$1 as $m$2 (Reddy et al., 2018).

This model makes explicit that DPC is constrained not only by aggregate storage but also by service capacity: each cache can serve at most one request per batch. That detail is not cosmetic. It implies that replica placement, request matching, and storage distribution are coupled, and that a configuration with high cumulative memory can still exhibit non-vanishing origin traffic if per-node capacity is poorly distributed.

A related formulation is the Multiple Cache Paging problem. There are $m$3 front-end caches, each of capacity $m$4, a backend holding the full catalog, and exactly $m$5 requests per slot that must be matched one-to-one to the $m$6 caches. A miss causes a fetch from the backend and an eviction at the selected cache. The main negative result is that for MCP with more than one cache, any online algorithm has unbounded competitive ratio for arbitrary input; the problem is therefore fundamentally different from classical single-cache paging (Vaze et al., 2016). A standard misconception is that a distributed page cache can be treated as a single cache of size $m$7. The MCP result rules out that reduction in the adversarial setting.

2. Placement, replication, and request routing

Because service capacity is distributed, placement and routing policies are central to DPC performance. In the heterogeneous-storage model, two policies are especially prominent. For $m$8, the homogeneous baseline uses PPMM, or Proportional Placement and Maximum Matching. For $m$9, the paper introduces KS+MLP, or Knapsack Storage plus Match Least Popular. KS+MLP uses a fractional knapsack objective to maximize the total probability mass of distinct requests covered by cached files, then places chosen files round-robin across caches and matches requests starting from least popular items (Reddy et al., 2018).

A complementary line of work studies routing under locality constraints. In a network of $n = m^{\gamma}$0 caching servers on a $n = m^{\gamma}$1 grid, Strategy I assigns each request to the nearest replica, minimizing communication cost but yielding poor maximum-load behavior. Strategy II, the proximity-aware two choices scheme, restricts each request to servers within a radius $n = m^{\gamma}$2 that cache the requested file, samples two such servers, and sends the request to the less loaded one. For $n = m^{\gamma}$3, per-node cache size $n = m^{\gamma}$4, and radius $n = m^{\gamma}$5, if

$n = m^{\gamma}$6

then the maximum load is $n = m^{\gamma}$7 with high probability and the communication cost is $n = m^{\gamma}$8 (Siavoshani et al., 2017). The nearest-replica strategy, by contrast, yields maximum load $n = m^{\gamma}$9 or at least $\gamma \ge 1$0 in the regimes analyzed. The practical implication is that modestly relaxing “always use the nearest cache” can produce exponentially better load balancing.

DistCache generalizes that principle to multi-layer caching. It co-designs cache allocation, cache topology, and query routing. Hot objects are partitioned between cache nodes in different layers by independent hash functions, and queries are routed adaptively with the power of two choices. The theoretical result is that cache throughput increases linearly with the number of cache nodes, derived by combining expander graphs, network flows, and queuing theory (Liu et al., 2019). In DPC terms, this replaces nearest-node heuristics with a small candidate set per object and a load-aware selection rule.

3. Popularity skew, storage heterogeneity, and cache granularity

The effect of storage heterogeneity depends strongly on the popularity profile. Under Zipf($\gamma \ge 1$1), the literature distinguishes comparatively spread popularity, $\gamma \ge 1$2, from lopsided popularity, $\gamma \ge 1$3. The central conclusion is that distributed caches are much more tolerant to storage heterogeneity when popularity is lopsided than when popularity is relatively flat (Reddy et al., 2018).

For $\gamma \ge 1$4, a homogeneous system with PPMM and cumulative memory $\gamma \ge 1$5 achieves $\gamma \ge 1$6. In contrast, if a constant fraction of caches in a heterogeneous system have memory

$\gamma \ge 1$7

then the best possible policy still has $\gamma \ge 1$8. The mechanism is that many small caches remain idle because each stores too few low-probability items to be useful in a typical batch. For $\gamma \ge 1$9, the lower bound depends only on total memory $M = m^{\mu}$0, not on the distribution of individual $M = m^{\mu}$1, and KS+MLP achieves the same order as that lower bound provided the top $M = m^{\mu}$2 caches have equal size (Reddy et al., 2018). Strong skew makes even small caches useful because replicas of very hot items are almost always exercised.

Granularity interacts with the same theme. CommCache introduces Sub-query Fragmentation, in which a query $M = m^{\mu}$3 is decomposed into independently executable sub-queries $M = m^{\mu}$4, represented as

$M = m^{\mu}$5

A central query analyzer builds query evaluation trees, promotes frequently reused sub-queries into cached objects, and relocates them among community cache locations during a maintenance phase (Venkata et al., 2019). This is not a transparent OS page cache; it is an application-level, query-aware cache. Yet it provides a precise workload-driven notion of cache granularity. Under dynamic workloads, SQF recovers faster than semantic or full-query caching, and in later epochs it surpasses semantic caching by approximately $M = m^{\mu}$6 in percentage of data found in cache; under Uniform workloads, semantic caching can have up to approximately $M = m^{\mu}$7 lower response time than SQF (Venkata et al., 2019). The broader lesson is that DPC effectiveness depends not only on capacity but also on whether the cached unit aligns with reuse structure.

4. Coherence, write-back semantics, and OS integration

Any DPC that operates below the application layer inherits the semantics of the local page cache. A macroscopic model of the Linux page cache captures the active and inactive LRU lists, dirty ratios, foreground flushes under memory pressure, and periodic background writeback. Reads and writes operate on “data blocks,” defined as subsets of file pages accessed in the same I/O operation, and the model supports both writeback and writethrough modes for local and network-based filesystems. Incorporating these mechanisms into simulation reduces error by up to an order of magnitude relative to cacheless simulation (Do et al., 2021). This establishes that DPC design cannot ignore local page-cache dynamics; it must either preserve them or replace them with an equally explicit policy.

DistFUSE addresses the long-standing conflict between write-back kernel caching and strong consistency in distributed FUSE filesystems. It is described as the first FUSE-based distributed file system that relies on a write-back kernel-based page cache for performance and provides strong consistency. The core mechanism is to offload userspace lock management to the kernel driver, extending per-inode metadata with a distributed lease field and adding fuse_acquire_dist_lease() and fuse_release_dist_lease() operations. The lease manager enforces per-file read and write leases; on revocation, dirty pages are flushed and caches invalidated before another node acquires the lease. DistFUSE improves throughput by up to $M = m^{\mu}$8 compared to baseline approaches (Li et al., 23 Mar 2025).

The CXL-based DPC line takes a different route. It enforces a single-copy invariant at page granularity: each file page has exactly one owner node holding the sole resident DRAM copy, and other nodes access it via CXL-based remote mappings rather than replicas. The directory protocol uses the states $M = m^{\mu}$9, $\mu \ge 1$0, $\mu \ge 1$1, $\mu \ge 1$2, and $\mu \ge 1$3, while hardware maintains cache-line coherence across hosts within shared regions. In the strong-consistency mode, DPC_SC uses a two-step write protocol with FUSE_DPC_LOOKUP_LOCK and FUSE_DPC_UNLOCK; in relaxed mode, local writable copies are allowed with weaker semantics. Across the reported workloads, this DPC delivers speedups of up to $\mu \ge 1$4, with a geometric-mean speedup of $\mu \ge 1$5 (Bergman et al., 21 Apr 2026).

5. Representative systems and deployments

The literature uses the term and its near-equivalents across several system strata, from middleware query caches to kernel-integrated page caches.

System	Cache unit	Salient mechanism
CommCache (Venkata et al., 2019)	Sub-query result	Central query analyzer, SQF, migration across community cache locations
StashCache (Weitzel et al., 2019)	Files and 24 MB CVMFS chunks	Origins, redirectors, caches, clients; federated opportunistic storage
DistFUSE (Li et al., 23 Mar 2025)	Kernel page-cache pages	Write-back page cache with kernel-managed distributed leases
DPC over CXL (Bergman et al., 21 Apr 2026)	Page-granular owner-mapped frames	Single-copy invariant with remote CXL mappings

StashCache is a concrete distributed caching federation for scientific workloads. Its architecture has four roles: origins, caches, redirectors, and clients. Caches are deployed at 6 universities and 3 Internet2 backbone PoPs in the U.S., with at least one cache in Europe, and each cache is guaranteed at least 10 Gbps networking and several TBs of caching storage (Weitzel et al., 2019). Clients select the “nearest” cache via GeoIP. CVMFS provides a read-only POSIX interface and downloads data in 24 MB chunks; if an application reads only portions of a file, CVMFS downloads only those portions. The deployment results show that after a local StashCache cache was installed at Syracuse, WAN usage decreased from 14.3 GB/s to 1.6 GB/s. Small files could still favor HTTP proxies because stashcp has higher startup cost, but for 2.3 GB and 10 GB files StashCache was often faster, depending on site network topology (Weitzel et al., 2019).

CommCache illustrates a different deployment pattern. It uses a central query analyzer, distributed data stores, and 20 cache units in the evaluation. Its active phase captures and fragments queries, while its maintenance phase analyzes usage statistics, relocates sub-queries, and performs eviction and consolidation. Because the caching unit is a sub-query result rather than a raw page, it is best understood as a query-aware extension rather than a transparent OS page cache (Venkata et al., 2019).

6. Extensions, alternative designs, and open boundaries

Two prominent extensions move beyond homogeneous DRAM caching. The first is hybrid memory. A stochastic model of hybrid page cache systems studies D-Cache, a DRAM-based page cache, and N-Cache, an NVM-based page cache such as PCM. Under the budget constraint

$\mu \ge 1$6

the analysis compares flat and layered architectures. In the flat architecture, DRAM and NVM are peers and pages never migrate between them; in the layered architecture, DRAM acts as a cache for NVM. For the common setting of the performance-price ratio of PCM, flat architecture is the better choice, but layered architecture outperforms if PCM write performance can be significantly improved in the future (Ju et al., 2016). For DPC, this suggests that node-local hybrid caches and cross-node distributed caches should not be designed independently.

The second extension is coded caching over distributed storage. In a system with $\mu \ge 1$7 data servers and optional parity servers, striping or whole-file placement can be combined with coded delivery. Under striping, the per-server peak rate scales by $\mu \ge 1$8 without parity and by $\mu \ge 1$9 with $\widetilde m = \rho m$0 parity servers. Under whole-file placement with two data servers and one parity server, a peak rate $\widetilde m = \rho m$1 is always achievable, and $\widetilde m = \rho m$2 is achievable when $\widetilde m = \rho m$3 is even and pairing is ideal (Luo et al., 2016). This is not a conventional page-cache protocol, but it formalizes an alternative to plain replication: redundancy can be used simultaneously for fault tolerance, load balancing, and coded delivery.

Several assumptions recur across the literature and delimit current results. Many analytical models assume fixed popularity and i.i.d. requests, equal object size, and either one request per cache per slot or tightly bounded service capacity (Reddy et al., 2018). Load-balancing analyses often assume static placement within the analysis horizon (Siavoshani et al., 2017). Strong-consistency designs such as DistFUSE do not deeply explore detailed crash-recovery protocols, even though they suggest standard Paxos or Raft replication for the lease manager (Li et al., 23 Mar 2025). The CXL-based DPC results rely on a CXL emulation framework because widespread multi-host CXL 3.0 hardware is not yet assumed (Bergman et al., 21 Apr 2026). A plausible implication is that DPC remains less a single architecture than a research area defined by one persistent objective: to make physically distributed memory behave like a coherent, high-capacity, workload-effective page cache without paying the full cost of redundancy, contention, or stale state.