Table-Compatibility Cache

• Table-compatibility caches are defined as architectures for storing, reusing, and invalidating cached table data using explicit compatibility relations.
• They are applied in relational DBMS, machine learning recommendation systems, and LLM inference engines to enhance scalability and selective data retrieval.
• Key algorithms include variant enumeration for queries, revision-based invalidation, and reuse-aware optimization, achieving significant performance improvements.

A table-compatibility cache refers to a class of cache architectures, mechanisms, and algorithms that enable storage, reuse, and fine-grained invalidation of cached objects or intermediate results derived from structured data tables, with explicit compatibility relations guiding when caches can be reused, shared, or evicted. These systems arise in database management, machine learning recommendation models, web application backends, and LLM inference engines, unified by a requirement for scalable, highly selective caching for relational, tabular, or embedding data.

1. Formal Definitions and Compatibility Relations

Table-compatibility in the caching context specifies under what conditions queries or objects may reuse or invalidate cached entries. Compatibility is rigorously defined in several settings:

Relational Table Query Caching

For single-table setups, compatibility follows a subspace intersection rule. Represent a table of arity $k$ as $S = D_1 \times D_2 \times \dots \times D_k$. A query $q$ is a $k$-tuple over $D_1 \cup \dots \cup D_k \cup \{*\}$, denoting selected or wildcard attributes. The subspace of $q$ is: $$\text{subspace}(q) = \{ r \in S \mid \forall i,\; r[i] = q[i] \text{ or } q[i] = * \}$$ A write query $q_w$ and read query $q_r$ are incompatible (write may invalidate read) iff the intersection of their subspaces is nonempty: $$(q_w, q_r) \in E \Longleftrightarrow \exists r \in S:\; r \in \text{subspace}(q_w) \cap \text{subspace}(q_r) \Longleftrightarrow \forall i,\; q_w[i] = q_r[i] \lor q_w[i]=* \lor q_r[i]=*$$ This supports infinite TTL and efficient, fine-grained invalidation for arbitrary predicates (Łopuszański, 2023).

Hash Table Caching in DBMSs

HashStash externalizes pipeline-breaking hash tables, tagging them by lineage (operator type, join/group-by keys, predicates). Compatibility is established via bisimilarity on lineage-graph and predicate containment. Four cases are handled by the reuse-aware optimizer (Dursun et al., 2016):

• Exact Reuse: Identical predicates/group-bys.
• Subsuming: Cache table contains a superset of required data.
• Partial: Cache table covers a subset; missing entries are inserted.
• Overlapping: Partial overlap; rewrite to augment/validate both tables.

LLM Inference and Embedding Tables

TableCache for text-to-SQL quantifies compatibility by exact table-set overlap. Given query $i$ touching tables $\tau_i$, represent as $I_i \in \{0,1\}^m$; the Hamming distance quantifies table overlap: $d_{i,j} = \sum_{k=1}^m (I_i \oplus I_j)[k]$ $d_{i,j}=0$ indicates full compatibility for KV cache reuse (Su et al., 13 Jan 2026).

2. Architectural Foundations and Data Structures

Cache Daemon for Relational Tables

SQLcached is a POSIX daemon embedding a pure in-memory SQLite (B-tree). No fixed schema is imposed; arbitrary tables can be created to suit the cached objects. Data structures leverage balanced B-trees, offer SQL-based CRUD and transaction support, and run entirely in RAM. Multiple instances can be horizontally scaled behind load-balancers (0910.0187).

HashStash Table-Cache Manager

A global cache records pointers to hash tables, each tagged with statistics (number of entries, tuple width, input sizes) and lineage. Eviction applies coarse-grained LRU; all hash tables live in extendible-hash layouts with amortized efficient resizing (Dursun et al., 2016).

TableCache for LLM Engines

TableCache precomputes KV caches for each table chunk and organizes them in a Trie structure (character or token level). Each trie leaf links to a table’s cache block; lookup for active tables is linear in the input length. Hot blocks are managed and prefetched on GPU during inference (Su et al., 13 Jan 2026).

Metadata and Versioning

Single-table compatibility caching employs per-pattern revision counters, digest-based query entries, and version vectors/joins to enable selective invalidation. If any read cache’s revision is stale in any coordinate, it is invalidated and refreshed (Łopuszański, 2023).

3. Selection, Reuse, and Invalidation Algorithms

Query Select and Invalidate Logic

A central innovation for compatibility caches is using all possible variants for a query pattern to track dependency:

• For a query $q$ on $k$ columns, enumerate up to $2^k$ patterns.
• Each pattern is associated with a revision counter, incremented on writes (insert/delete).
• The cache stores results tagged by the current vector of relevant revision counters.
• Selecting reads back all corresponding revisions; cache entries are returned only if their version dominates (is not stale) (Łopuszański, 2023).

Pseudocode in the original text describes explicit procedures for variant enumeration (tail-recursive), revision fetch/add, select, invalidate, insert, and delete. These guarantee correct propagation of dependencies in $O(2^k)$ time and space.

Reuse-aware Optimization

HashStash’s optimizer integrates table-cache selection into plan enumeration:

1. For each logical plan partition, enumerate compatible cached tables plus fresh builds.
2. Rewrite operators to probe, extend, or post-filter based on compatibility.
3. Evaluate cost models (see section below) and select minimal cost plan (Dursun et al., 2016).

Cache Lookups in LLM Inference

Given a token sequence, TableCache’s trie matching algorithm finds all referenced tables in $O(n)$, fetches corresponding KV blocks, and forms a concatenated prefix for model inference. Batch reranking prioritizes queries with maximal table-set overlap, minimizing GPU cache swapping (Su et al., 13 Jan 2026).

4. Cost Modeling, Performance Trade-offs, and Evaluation

Cost Models

HashStash computes reuse and build-new operator costs leveraging hash table statistics, data movement, cache effects, and predicate overhead: $$C_{\text{RHJ}}(HT_c) = c_{\text{resize}} + (|Builder| \cdot (1-\text{contr})) c_i + |Prober| c_r + (|HT_c| \cdot \text{overh}) c_\sigma$$ Contrasts with fresh build and aggregate costs to choose reuse only when contribution $\geq 70\%$ (Dursun et al., 2016).

Empirical Results

• HashStash achieves up to $2\times$ speedup over non-reuse and $1.4\times$ over materialize-based reuse under high reuse, with intermediate memory savings and minimized cost for multi-query batch plans (Dursun et al., 2016).
• TableCache shows up to $3.62\times$ improvement in TTFT (Spider Text-to-SQL) with negligible (<1%) accuracy drop. Ablation shows precompute, reranking, and pipelining contribute complementary improvements (Su et al., 13 Jan 2026).
• Mixed-precision embedding caches drive $3\times$–$7\times$ memory reduction with neutral accuracy for $1\%$–$5\%$ cache ratios, and $16\%$ end-to-end training speedup over unified memory baselines in recommendation models (Yang et al., 2020).
• Versioned compatibility caches reach $>90\%$ hit ratios for read-heavy web applications with sub-second freshness guarantees (Łopuszański, 2023).

5. Practical Considerations and Implementation Insights

Fine-grained Expiry, Data Pruning, and Bulk Ops

SQLcached supports fine-grained expiry via arbitrary SQL WHERE predicates, enabling selective invalidation (expire per-page, per-user, or per-region). TTL-based expiry, table-size controls, and operation-based purges can be configured. Indexes (e.g., B-tree over timestamp) accelerate range scans for expiry, yielding $O(\log n + K)$ algorithmic complexity (0910.0187).

Pattern set $P$ in versioned caches can be statically trimmed by analyzing read/write query logic to minimize revision counter space. Columns unused in equality search are pruned, reducing the variant explosion. Inequalities over small domains are decomposed to binary trees for space-optimal revision counters (Łopuszański, 2023).

TableCache maintains bulk cache update capability, LRU/FIFO/LFU eviction policies, and micro-batched GPU compute/memory pipelining (Su et al., 13 Jan 2026).

Concurrency and Correctness

Formal concurrency correctness is established by ensuring

• Monotonic increments of revision counters,
• Version vectors computed per query are always fresh,
• Atomicity ensures that completed writes strictly invalidate stale reads.

Freshness is bounded by the worst-case RPC latency $\varepsilon$; cached selects may serve stale data no older than $\varepsilon$ (Łopuszański, 2023).

Single-threaded SQLcached employs serializable updates and SQLite locking to prevent deadlocks, limiting throughput to sequential request processing (0910.0187).

Multi-threaded hash table reuse (beyond HashStash’s prototype) is an area suggested for future investigation, requiring reader–writer protocols (Dursun et al., 2016).

6. Extensions, Limitations, and Research Directions

HashStash and TableCache approaches suggest several generalizations:

• Extending compatibility caches to bitmaps (for selection filters), sort-merge runs (for order-preserving operations), partial B-trees (for adaptive indexing), and beyond hash-based joins/aggregates (Dursun et al., 2016).
• Handling data skew via dynamic bucket statistics to prevent bucket overload.
• Generalizing schema matching for compatibility across diverse group-by and join sets.
• Employing cost-aware multi-versioning for hybrid cache entries serving subsuming and partial requests.
• Machine-learned cost models adapting cache metrics to dynamic hardware and workload environments (Dursun et al., 2016).
• Further integration of primary–foreign key graphs for cross-table semantic compatibility in LLM prompt construction and KV cache partitioning (Su et al., 13 Jan 2026).

A plausible implication is that table-compatibility caches with formalized compatibility relations, versioned invariants, and selective reuse logic are foundational for the next generation of data-intensive systems, effective in both low-latency services and high-throughput analytical and machine learning applications.

7. Comparative Analysis with Non-Compatible Caches

Table-compatibility caches distinguish themselves by:

• Supporting complex, predicate-driven invalidation and expiry versus global TTL or flush,
• Enabling fine-grained reuse for both full and partial table/query overlap,
• Maintaining correctness under concurrent operations via explicit compatibility relations,
• Trading raw throughput for rich query support, selective data retrieval, and memory efficiency,
• Achieving substantial empirical gains in analytical DBMS, LLM inference, and large-scale embedding models.

Systems such as memcached deliver higher throughput for simple GET/SET but lack SQL expressivity, arbitrary predicate expiry, and table-structural compatibility (0910.0187). Compatibility-driven tables enable efficiency and correctness in workloads where shared data patterns, partial reuse, and precise invalidation dominate benefits.

---

Key references:

• (0910.0187): SQLcached and relational cache daemon design
• (Dursun et al., 2016): HashStash and table-cache for main memory DBMSs
• (Yang et al., 2020): Mixed-precision embedding cache in recommender systems
• (Su et al., 13 Jan 2026): TableCache for LLM Text-to-SQL inference optimization
• (Łopuszański, 2023): Formal model and algorithm for table-compatibility cache invalidation in web backends