Specialized Caching: Disk, Multi-Instance & Graph

• Specialized caching systems are advanced storage solutions that use workload- and topology-aware strategies to overcome the limitations of generic cache policies.
• Disk-based and multi-instance caches deploy adaptive prefetching, tailored eviction policies, and load balancing to improve throughput and reduce latency in heterogeneous environments.
• Graph-structured caches leverage data clustering, recursive layering, and fine-grained eviction mechanisms to enhance query performance in graph workloads and analytics.

Specialized caching systems address the limitations of general-purpose cache policies in high-performance storage, distributed inference, and graph processing environments. They integrate workload- and topology-aware placement, sophisticated eviction and prefetch policies, and adaptive data organization, enabling efficient handling of nonuniform access patterns, multi-instance or multi-tenant scenarios, and hardware heterogeneity. Recent research spans disk-aware caches, multi-instance (multi-access) caches, and graph-structured caches, each optimizing locality, throughput, and scalability within their target domains.

1. Disk-Based and Multi-Tier Caching Architectures

Disk-based caching systems are engineered to operate over SSD/NVMe tiers or persistent storage backends, balancing performance, cost, and endurance.

Example: Shared Disk KV Cache for LLM Inference

Shared RAG-DCache enables multi-instance LLM retrieval-augmented generation services via a disk-resident key-value tensor cache, which stores document-level precomputed “past_key_values” for efficient LLM context assembly (Lee et al., 16 Apr 2025). It combines:

• A centralized KV Cache Manager with an in-RAM LRU cache (capacity ~16 GB) and SSD-backed storage for scale.
• Proactive prefetching keyed on expected queueing delay; hot entries are loaded into RAM/SSD based on retrieval frequency (only 3.1–31.4% of chunks cover 50% of queries across datasets).
• LRU eviction in RAM with no on-disk eviction (append-only unless corpus updated).
• Experimentally, this architecture attains up to 71% throughput increase and 65% lower latency over baseline multi-instance LLM serving.

Example: Adaptive Block Allocation and Grouping

AdaCache implements a disaggregated rack-scale cache for cloud block storage, supporting adaptive cache block sizing based on request length (Yang et al., 2023). Variable-size allocations are organized in fixed-size “groups” to minimize fragmentation and enable a two-level replacement policy (group-level LRU + block-level LRU). The result is improved I/O latency (e.g., 19–20% lower read latency vs. 32 KiB fixed-size cache), reduction in backend I/O volume (74% less), and major metadata memory savings.

Example: dLSM Compaction Buffer

In LSM-tree–based key-value stores, cache invalidation by disk compactions degrades read performance. The dLSM system inserts a write-once, append-only on-disk “compaction buffer” for hot or recently written data. This buffer decouples compaction rates from cache invalidation frequency, reducing invalidations by a factor ~1/(1+r) (r=size-ratio per level) and sustaining 5–8x random-read throughput under heavy write loads on HDD (Guo et al., 2016).

2. Multi-Instance and Hierarchical Load-Balanced Caches

Multi-instance, multi-access, and hierarchical cache designs focus on scalability and equitable load distribution in distributed environments.

Example: DistCache—Provable Load-Balanced Multi-Instance Cache

DistCache achieves linear throughput scaling in large-scale distributed storage by employing a two-layer cache topology (rack-local and global) and independent hash-based hot object partitioning (Liu et al., 2019). Core mechanisms include:

• Each hot object is cached in two nodes (one per layer) using independent hashes, minimizing duplication.
• Each client query chooses between its two hash-determined cache nodes, routing to the less-loaded using the “power-of-two-choices” (decentralized, O(1) per-packet cost).
• Formal analysis shows cache throughput T(m) = Ω(m); experimental results demonstrate scaling under skewed workloads, minimal cache coherence traffic, and robust failure recovery.
• Extensions generalize to multi-tier hierarchies and graph workloads via recursive layering and independent hash partitioning.

Example: Multi-Access Coded Caching via Conflict Graphs

Multi-access coded caching systems with arbitrary user–cache access topology are modeled as index-coding problems on conflict graphs (Yang et al., 15 Jan 2026). Each user may access an arbitrary subset of L caches. Conflict graphs capture decoding dependencies, reducing broadcast code construction to graph coloring (minimum color count equals minimum load). Key contributions:

• The DSatur greedy coloring heuristic delivers loads within 1–2% of the index-coding converse bound but at high complexity.
• A learning-based Graph Neural Network (GNN) approach achieves near-optimal multicast construction with ~20–30–fold lower runtime.
• Direct applicability to graph-structured storage topologies and extension to disk-block caches.
• The unified graph-theoretic framework accommodates heterogeneous cache arrangements and arbitrary topologies.

3. Graph-Structured Caching: Algorithms and Performance

Graph-structured caches exploit the structure of the workload or data retrieval graph—notably in database engines, nearest neighbor search, and analytics.

Example: GoVector—Phase-Aware and Vector-Local Disk Cache

GoVector addresses high-dimensional vector nearest neighbor (ANN) search over graph indices by combining:

• Hybrid two-tier caching: static cache for entry-points and multi-hop neighbors (Phase I), and a dynamic cache of adaptively prefetched nodes for fine-grained search (Phase II), both in RAM.
• Similarity-driven on-disk node reordering via k-means clustering reduces cross-page fetches.
• Prefetching in Phase II batches nodes/adjacent pages by cluster to maximize spatial locality.
• At 90% recall, GoVector reduces I/O by 46%, boosts throughput by 1.73x, and reduces query latency by 42% versus state-of-the-art systems (Zhou et al., 21 Aug 2025).

Example: One-Hop Sub-Query Result Cache for Graph Databases

Graph database systems benefit from keying the cache at the level of one-hop sub-queries (from a vertex along edges matching property predicates) (Nguyen et al., 2024). In this model:

• Each cache entry represents the result of a parameterized one-hop traversal, stored as an immutable vertex ID set.
• On-cache-miss, the system performs the traversal and asynchronously populates the cache; write transactions install invalidate routines for related cache keys.
• This granularity yields tractable invalidation—crucial for transactional isolation and ACID compliance.
• Production workloads in eCommerce graphs show 2–4.5x improvements in tail latency for read and write transactions when combined with query rewriting.

Example: Cache-Aware Graph Analytics on Conventional Hierarchies

Techniques such as frequency-based clustering (packing high-degree vertices in contiguous physical order) and CSR segmenting (partitioning the graph so per-segment data fits in the last-level cache) maximize cache-line utilization and convert irregular accesses to sequential DRAM traffic (Zhang et al., 2016). When combined, these approaches deliver up to 5× speedup for PageRank and 2× for Betweenness Centrality over optimized baselines, with minimal code changes.

4. Advanced Workload-Aware Management and Replacement

Specialized caches increasingly exploit workload semantics and offline or online profiling to inform partitioning, replacement, and prefetch strategies.

Example: Useful Reuse Distance and Per-VM Policy

ECI-Cache introduces the Useful Reuse Distance (URD), which only counts accesses that make a block likely to be reused by a read (RAR/RAW, ignoring WAW/WAR), unlike traditional reuse distance (Ahmadian et al., 2018). This enables fine-grained cache block allocation and per-VM write policy (Write-Back vs. Read-Only), improving performance-per-cost by 30% and SSD endurance by 65% in multi-tenant environments.

Example: Workflow-Aware KV Caching for Agentic LLMs

KVFlow introduces the Agent Step Graph to model agent dependencies and “steps-to-execution” in LLM-based workflows (Pan et al., 10 Jul 2025). These values define fine-grained eviction priorities in a prefix-tree KV cache, ensuring future-needed entries are retained. Prefetch is overlapped with active computation by traversing the step graph. Under high concurrency and large prompt workloads, KVFlow achieves 1.83–2.19× speedup over hierarchical radix and LRU-based policies.

Example: Domain-Specialized Cache Management (GRASP)

For graph analytics on power-law graphs, GRASP leverages static degree-based vertex reordering and hardware-level Address-Bound Registers to label vertex data cache lines as high-/moderate-/low-reuse, biasing insertion and promotion policies at the LLC (Faldu et al., 2020). This lightweight scheme outperforms complex dynamic predictors, sustaining up to 10% speedup with negligible hardware cost.

5. Implementation Strategies and Scalability

Contemporary specialized caching frameworks deploy in heterogeneous software environments and exploit hardware primitives:

• Many reside in user space and interact directly with RDMA/NVMe-oF, public cloud APIs, or programmable switch ASICs (e.g., DistCache, AdaCache).
• Data structures include hybrid RAM–SSD tiers, fixed-size groups for organizing variable-size cache blocks, tree/radix structures for prefix caches, and hash tables for O(1) lookups.
• Eviction and replacement often layer multiple policies (e.g., group- and block-LRU in AdaCache, per-node priorities in KVFlow).
• Multi-instance and distributed deployments frequently rely on shared coordination layers (e.g., centralized managers, service coordinators) and leverage persistent KV stores (e.g., FoundationDB) for global cache coherence and fault tolerance (Nguyen et al., 2024).
• Analytical models quantify cache performance in terms of load balancing (matching/expander arguments), I/O reduction (phase-local hit rates, expansion factor in segmented processing), and endurance/cost (write amplification and capacity models).

6. Trade-Offs, Limitations, and Research Directions

Specialized cache systems introduce trade-offs around management overhead, hardware dependencies, and generality:

• Many algorithms demand fine-grained real-time profiling, nontrivial metadata management, or cross-layer co-design; others, such as GRASP and frequency-based clustering, prefer static analysis and minimal runtime cost.
• Disk-based and persistent caches face endurance and fragmentation constraints, which are mitigated by adaptive block sizing and group allocation, but further work remains (e.g., dynamic resizing, wear leveling).
• Caches tailored to highly skewed workloads may provide limited benefits for uniform access patterns; the generalization to dynamic or streaming data is underexplored.
• Graph-structured caches are a natural fit for workloads with graph-theoretic access patterns but introduce complex invalidation, consistency, and sharding considerations at multi-instance or geo-distributed scales.

Overall, the integration of workload-awareness, explicit topology or dependency modeling, and adaptability to hardware/storage layout distinguishes specialized caching from conventionally engineered caches, yielding substantial and rigorously quantified gains across a broad array of performance metrics. These systems provide a foundation for further advances in tiered storage management, disaggregated computing, and high-throughput inference serving.