IndexCache: Integrated Indexing & Caching

Updated 3 July 2026

IndexCache is a unified approach that combines fast lookup index structures with cache memory to minimize recomputation and I/O.
It is applied in diverse domains including coded caching in networks, transformer-based LLMs with sparse attention, and hardware-accelerated SSD search.
By leveraging techniques such as greedy layer selection, B-skiplist design, and learned token retention, IndexCache significantly enhances system efficiency.

IndexCache is a general term applied to methods, data structures, and system architectures that integrate indexing and caching to minimize computational overhead in repeated or memory-constrained query processing. The term appears prominently in several domains: coded caching in network information theory, approximate-indexed caches for similarity search, high-throughput dynamic indexing structures for large-scale data processing, hardware-accelerated in-storage index matching, and KV-cache management in long-context LLMs. The unifying theme is the deliberate blending of index structures—providing fast, selective lookup—and cache memory—minimizing recomputation or I/O, with the intent to optimize latency, throughput, and/or resource utilization across layers of a processing system.

1. IndexCache in Coded Caching and Index Coding

In the coded caching context, IndexCache refers to the explicit formulation of the content delivery (“delivery phase”) as an instance of the index coding problem. For a system with $N$ files, $K$ users, and cache memories of size $M$ files per user, the standard two-phase approach consists of an uncoded placement phase (simple bit-copying) and a coded delivery phase. Once user demands are revealed, the broadcast transmission can be mapped to an index coding instance where the subfiles not already cached form the “messages,” and each user’s side information is the cache content. The fundamental limit on the transmission rate under uncoded placement, denoted $R_u^*(M)$ , is described by a piecewise-linear function parameterized by combinatorial coefficients $c_q$ :

$R_u^*(M) \geq \max_{q=0,...,K} c_q + (c_q - c_{q-1}) (KM/N - q),$

where $c_q = \left[ \binom{K}{q+1} - \binom{K-\min(K,N)}{q+1} \right] / \binom{K}{q}$ and $(M_q, R_q)=(qN/K, c_q)$ . These limits are tight for the Maddah-Ali–Niesen (MAN) scheme when $N \geq K$ and strictly improve upon the composite coding bound for $N < K$ via distributed source coding and non-unique decoding. Any improvement beyond this bound requires coded placement, with network-load improvements capped at a factor of two compared to uncoded placement (Wan et al., 2020, Wan et al., 2017).

2. Layer-Reuse IndexCache for Sparse Attention in LLMs

In transformer-based LLMs with sparse attention (specifically DeepSeek Sparse Attention), an IndexCache mechanism leverages the high redundancy in top- $K$ 0 attention selection between consecutive layers. Instead of running the expensive $K$ 1 per-layer indexer at every layer, layers are partitioned into Full (“F”, run indexer) and Shared (“S”, reuse previous indices) layers. Formally, for a layer pattern $K$ 2 ( $K$ 3, $K$ 4), layers with $K$ 5 simply reuse the top- $K$ 6 indices from the nearest preceding F layer. This reduces indexer cost from $K$ 7 to $K$ 8 for $K$ 9 retained indexers. Two approaches to optimizing the pattern are proposed:

Training-free: Greedy search over layers to minimize language modeling loss given a calibration set, with no weight updates.
Training-aware: Multi-layer distillation, wherein each indexer is distilled against the mean attention distribution over the layers it serves; loss:

$M$ 0

Experiments on 30B- and 744B-parameter DSA models demonstrate up to 75% reduction in indexer work, $M$ 1 prefill and $M$ 2 decode speedup without measurable degradation in accuracy (Bai et al., 12 Mar 2026).

3. B-Skiplist: Locality-Optimized IndexCache for Key-Value Stores

Cache-friendliness in indexing structures is addressed by the B-skiplist, which blocks $M$ 3 key-value pairs per node to minimize last-level cache (LLC) misses. The node format

$R_u^*(M)$ 6

results in horizontal scans (across $M$ 4 entries) fitting into one or two cache lines and only one pointer dereference per vertical traversal. A top-down, single-pass algorithm realizes $M$ 5 cache-line transfers per operation, matching B-trees. A reader–writer hand-over-hand locking scheme gives high concurrency and low tail latency. Experiments demonstrate $M$ 6– $M$ 7 throughput gains and $M$ 8– $M$ 9 latency reduction vs. unblocked skiplists, and performance competitive with cache-optimized B-trees/Masstree. Aligning blocked index layouts with cache granularity is cited as key for designing in-memory IndexCache structures in key-value systems (Luo et al., 29 Jul 2025).

4. Indexed Caching in Distributed Data Processing Systems

For distributed data analytics frameworks (e.g., Apache Spark), the Indexed DataFrame paradigm attaches a mutable, write-enabled in-memory index—implemented as a concurrent hash-trie (cTrie)—to each partition of an in-memory data cache. This “IndexCache” supports $R_u^*(M)$ 0 expected-time point lookups and avoids repeated $R_u^*(M)$ 1 scans or hash-table builds for each query or join. APIs such as createIndex(col), getRows(key), and index-assisted joins integrate seamlessly via Catalyst optimizer rewrites. Empirical results show 3–20× speedups on join-heavy workloads and point lookups, and only modest memory (<2% per partition) and build latency overhead (Uta et al., 2021).

5. Hardware-Accelerated IndexCache: Search-in-Memory (SiM)

At the storage layer, the SiM chip implements IndexCache by embedding bitwise parallel match (BPM) units within the NAND flash pipeline of solid-state drives. A 64-bit query and mask are broadcast per page; all 512 eight-byte slots are matched in parallel, returning a 512-bit hit bitmap instead of the entire page. Host software then selectively gathers matching chunks. This in-flash SIMD-style filtering reduces I/O volume, slashes energy, and lowers observed read/write latency (median read-latency reduced up to 89%; 99th percentile by 85%; energy savings up to 45%). SiM supports B $R_u^*(M)$ 2-tree leaf lookups, hash-bucket queries, and bit-sliced bitmap range queries through two NVMe-DSM commands (Search/Gather), with a typical bulk-matching latency of less than 1µs (Chen et al., 2024).

6. Approximate Similarity IndexCache

For caching in high-dimensional similarity search, ACAI (Ascent Caching with Approximate Indexes) merges independent approximate nearest neighbor (ANN) indexes (e.g., HNSW/FAISS) for both the local cache and remote catalog. Requests are scored by local and remote similarity, with the cache state adaptively managed by an online mirror ascent procedure over a fractional relaxation of set selection, rounded randomly to integral cache vectors. ACAI achieves $R_u^*(M)$ 3-regret competitive guarantees, with empirical gains of 10–30% over state-of-the-art baselines, especially with tight caches or high remote fetch cost. Indexing accounts for ~80–90% of ACAI’s advantage; OMA updating provides the remainder (Si-Salem et al., 2021).

7. Learned IndexCache for KV-Eviction in LLM Inference

In efficient long-context LLM inference, IndexCache encompasses learnable indexer-based token retention and latent memory for KV-cache eviction. The indexer assigns a per-token importance score via a lightweight down-projection MLP, trained to distill max attention relevance from the full model:

$R_u^*(M)$ 4

Tokens below a retention threshold are evicted and optionally compressed into a latent memory matrix $R_u^*(M)$ 5, updated online. The memory emits learned residuals to compensate for lost attention. IndexMem demonstrates up to 25-point gains on RULER and superior stability on NIAH/LongBench compared to KeyDiff, SnapKV, or heuristic recency policies, especially when stringent cache budgets or long-range retrieval are required (Yang et al., 25 May 2026).

These diverse instantiations of IndexCache clarify that, irrespective of the domain—information-theoretic coded caching, in-memory distributed analytics, hardware-accelerated SSD search, similarity search, or KV-pruning for LLMs—the principle remains the same: integrate a selective, low-latency index structure with cache mechanisms to minimize redundant computation, I/O, or memory, while maintaining rigorous performance guarantees. Recent advances leverage distributed source coding, greedy/differentiable selection and residual memory, concurrency-optimized blocking, and hardware-level matching—demonstrating that IndexCache is both an algorithmic and systems concept, foundational to high-performance data-intensive computation.