---
title: 'Map-Rule Cache: Patterns and Applications'
url: https://www.emergentmind.com/topics/map-rule-cache
type: topic
---

# Map-Rule Cache: Patterns and Applications

Searching arXiv for the cited work and closely related cache papers to ground the article.
In the literature, a map-rule cache can be understood as a cache whose reusable unit is either an explicit mapping or a compact map-like artifact, and whose correctness depends on explicit admission, validation, or eviction rules. In inter-domain routing, this takes the form of a cache of temporal locator↔identifier bindings used to preserve forwarding speed after semantic decoupling of identity and location. In retrieval-augmented generation, it takes the form of an answer cache whose entries are reused only when four safety gates hold. In long-context LLM agents, it appears as a bounded context map that stores reusable orientation knowledge about a recurring external context. Across these settings, the common problem is to reduce repeated lookup or generation cost without sacrificing miss-rate predictability, evidence validity, or prompt-budget discipline [1312.1378] [2605.27494] [2605.19932].

## 1. Conceptual scope

A useful way to organize the subject is to distinguish the cached object from the rule system that governs it. In the routing setting, the cached object is a binding between namespaces, such as an RLOC↔EID entry, and the governing mechanism is primarily an LRU replacement policy analyzed through working-set theory. In the RAG setting, the cached object is a tuple \((q^c, e^c, a^c, \sigma^c)\) containing a query, its embedding, an answer, and an evidence signature, and the governing mechanism is a four-gate admission rule requiring simultaneous agreement on query similarity, evidence overlap, source-version validity, and lexical or judge-based support. In the long-context agent setting, the cached object is an entry \(r_i = (id_i, section_i, content_i, tokSize_i, score_i, ts_i)\) in a persistent context map, and the governing mechanism is a programmable policy composed of a Distiller, a Cartographer, and a priority-based Evictor [1312.1378] [2605.27494] [2605.19932].

| Setting | Cached object | Governing rule structure |
|---|---|---|
| Inter-domain routing | locator ↔ identifier binding | LRU with working-set-based sizing |
| RAG answer reuse | \((q,e,a,\sigma)\) entry | four admission gates \(G1\)–\(G4\) |
| Long-context agents | context-map entry \(r_i\) | Distiller, Cartographer, Evictor |

This organization suggests that “map-rule cache” is less a single protocol artifact than a recurring design pattern: a cache is only useful when the object being reused remains semantically valid under a compact rule system. The routing paper treats validity statistically through locality and stationarity; the RAG paper treats validity through evidence-grounded gating; the agent paper treats validity through orientation-value scoring and budgeted retention.

## 2. Working-set model for Loc/ID mapping caches

The analytical treatment of Loc/ID mapping caches begins from a discrete reference model. References are indexed as \(t=1,2,\dots\), a window of length \(T\) denotes the last \(T\) references, and each cache entry stores a temporal binding between a locator and an identifier. Bindings are installed on a per-use basis: on a miss, the router fetches the mapping from the control-plane system and stores it. The model assumes an essentially unending reference string, stationarity, and asymptotic independence; under these conditions, classic working-set results apply and cache behavior under LRU is well modeled by the working-set size and miss-rate functions [1312.1378].

Let \(N\) be the universe of possible destinations and \(\rho = r_1 r_2 \cdots r_t \cdots\) the reference string, with each \(r_i \in N\). The working set at time \(t\) over window \(T\) is

$$
W(t,T)=\{\,\text{distinct prefixes referenced in }[t-T+1,\;t]\}\,,
$$

with size \(w(t,T)=|W(t,T)|\). The average working-set size is

$$
s(T)=\lim_{k\to\infty}\frac{1}{k}\sum_{t=1}^k w(t,T)\,.
\tag{1}
$$

Under mild regularity, the instantaneous miss rate per reference is

$$
m(T)=\frac{d}{dT}s(T)\,,
$$

and the inter-reference distance density satisfies

$$
f(d)=-\frac{d^2}{dT^2}s(T)\big|_{T=d}\,.
$$

Empirically, \(s(T)\) is observed to be piecewise linear in log-log scale, which motivates the fit

$$
s(T)\approx e^{\beta}\,T^{\alpha},
\qquad (0<\alpha\le 1,\;\beta\ge 0)\,.
\tag{2}
$$

Here \(\alpha\) captures how strongly the working set grows with time, and \(\beta\) sets the scale. For an LRU cache of fixed size \(C\), the steady-state miss rate equals \(m(T)\) evaluated at the unique \(T\) such that \(s(T)=C\). Inverting the power law yields

$$
T(C)=\exp\!\bigl(-\beta/\alpha\bigr)\,C^{1/\alpha}\,,
$$

and therefore

$$
P_{\rm miss}(C)=m\bigl(T(C)\bigr)
=e^{\beta/\alpha}\,\alpha\,C^{\,1-1/\alpha}\,.
\tag{3}
$$

The key parameters are the arrival rate \(\lambda\), the fitted coefficients \(\alpha\) and \(\beta\), the cache size \(C\), and the expected miss probability \(P_{\rm miss}(C)\). The model’s importance lies in separating shape from rate: \(\lambda\) affects absolute rates but not the shape of \(s(T)\), whereas \(\alpha\) and \(\beta\) encode locality.

## 3. Applicability, validation, and router sizing

The theory is not presented as universally applicable; it is conditioned on the reference process satisfying the three working-set constraints. In practice, stationarity is tested by extracting multiple working-set curves \(w(t,T)\) with different start times \(t-T\) and verifying that, for each fixed \(T\), \(w(t,T)\) is approximately Gaussian and clusters, or by applying time-series stationarity tests such as augmented Dickey-Fuller to inter-reference distances. Because the sequence satisfies asymptotic independence, LRU is treated as approximating the optimal offline policy. For deployment, the prescribed workflow is to measure a day or more of packet-header traces, aggregate by destination prefix, compute \(s(T)\) via a standard working-set algorithm such as Denning’s stack-distance method, fit \(\alpha,\beta\), and then use equation (3) to size the cache for a target miss rate [1312.1378].

Validation is performed by emulation on four 24 h egress traces: upc 2009, upc 2011, and upc 2012 from a 2 Gbps campus link, and cesca 2013 from a 10 Gbps academic backbone. Each IP address is resolved to a BGP-table prefix from RouteViews, with stale-filtering of more specifics. The emulator uses pure LRU, infinite TTL, and varying cache sizes from a few hundred entries up to approximately \(|\Psi|\), the daily-visited prefix set. The tested sizes range from a few hundred entries up to approximately \(50\%\) of the ISP’s routing table, normalized by \(|BGP_\phi|\). Over four orders of magnitude in \(C\), the analytical curve remains within a few \(0.01\%\) absolute miss error of the emulator. The reported mean absolute error is below \(0.02\%\) miss, and the worst-case deviation is below \(0.1\%\) at the largest \(C\), where cold-start bias and few data points in \(s(T)\) fitting reduce precision [1312.1378].

These results give the model direct operational significance. Under normal traffic, LRU is described as near-optimal, and a cache of approximately \(10\%\) of the global BGP table yields miss rates \(\lesssim 0.2\%\), i.e. below \(1\) k miss/s at \(500\) k pkt/s. The size-versus-miss trade-off follows the power law in equation (3), with pronounced knee points; small increases in \(C\) yield large miss reductions until those knees are reached. A recurrent misconception is that map-cache provisioning is an ad hoc exercise. The analytical model argues the opposite: once locality is verified empirically, cache dimensioning becomes a parametric exercise in fitting \(s(T)\) and reading off \(P_{\rm miss}(C)\).

## 4. Cache-polluting scans and robustness limits

The same framework is extended to cache-polluting traffic by introducing an attacker-selected set \(\Omega \subset BGP_\phi\) of prefixes to scan, a legitimate-to-attack rate ratio \(\tau = \frac{1}{1+\rho}\), and an overlap parameter \(\delta = |\Omega \cap \Psi|/|\Psi|\), ranging from \(0\) for disjoint scans to \(1\) for full overlap. If \(u\) denotes the total number of packets seen, the combined working-set size becomes

$$
s_a(u)=
\begin{cases}
s(\tau u)+\bigl(1-\tau-\tfrac{\tau\,\delta\,s(u_k)}{u_k}\bigr)\,u
& u<u_k \\[6pt]
(1-\delta)\,s(\tau u)+|\Omega|
& u\ge u_k
\end{cases}
\quad,\;
u_k=\frac{|\Omega|}{1-\tau}\,,
\tag{4}
$$

and differentiation gives the miss-rate expression

$$
m_a(u)=
\begin{cases}
\tau\,m(\tau u)+\bigl(1-\tau-\tfrac{\tau\,\delta\,s(u_k)}{u_k}\bigr)
& u<u_k \\[4pt]
\tau\,(1-\delta)\,m(\tau u)
& u\ge u_k
\end{cases}\,.
\tag{5}
$$

The first term \(s(\tau u)\) is the contribution of legitimate traffic scaled to \(\tau u\); the linear term corresponds to new attack prefixes minus overlaps [1312.1378].

Empirically, the attack model is tested on upc 2012 with \(\rho \in \{0.01, 0.1, 0.5\}\) and \(\delta = 0,1\). Overlapping scans, \(\delta=1\), are reported as the most damaging: the miss rate stays above \(1\%\) unless the cache is at least approximately \(0.6\times |BGP|\). Even a \(1\%\) attack, \(\rho=0.01\), raises the miss rate by one order of magnitude compared to normal operation, from \(0.1\%\) to \(1\%\). The analytical attack model tracks emulation within below \(0.05\%\) absolute error [1312.1378].

The design consequences are explicit. Overlapping scans are described as worst-case because remedying them by raw capacity alone would require caches of size \(\gtrsim |BGP|\), i.e. essentially the full prefix space, which is impractical. Detection can therefore be based on comparing the instantaneous miss rate against the predicted \(P_{\rm miss}(C)\) and raising an alert when it exceeds a threshold. The proposed mitigations are a two-level cache that protects top entries from eviction and evicts into a slower or shadow cache, per-user rate limiting for new prefixes, and more sophisticated eviction policies such as frequency-biased schemes at the cost of metadata complexity. This directly counters the common assumption that attack resilience is merely a question of more DRAM.

## 5. Four-gate answer caches in retrieval-augmented generation

In retrieval-augmented generation, a map-rule cache is instantiated as a grounded answer cache layered above a retriever, an LLM generator, and an embedding model. The architecture has a stage-2 retrieval cache and a stage-3 answer cache. After a query \(q\) is embedded as \(e=\mathrm{emb}(q)\), the system performs retrieval, deduplicates and orders chunks, computes an evidence signature \(\sigma\), and looks up the nearest cached answer entry \(h\). The cached answer \(a^c\) is reused only if all four admission gates hold simultaneously; otherwise the system compresses evidence, regenerates with the LLM, and inserts \((q,e,a,\sigma)\) into the answer cache. A hit in the retrieval cache does not by itself authorize answer reuse: only a stage-3 answer-cache hit with all gates satisfied bypasses generation [2605.27494].

The four gates are defined formally. The query-similarity gate requires

$$
\cos(e,e^c)\ge \tau_q\,,
$$

with pre-normalized embeddings. The evidence-overlap gate requires the Jaccard overlap of chunk-hash sets,

$$
J(\sigma,\sigma^c)=
\frac{|H(\sigma)\cap H(\sigma^c)|}
{|H(\sigma)\cup H(\sigma^c)|}
\ge \tau_e\,.
$$

The source-version-validity gate requires matching version tags for every chunk hash in the intersection of \(H(\sigma)\) and \(H(\sigma^c)\). The lexical-support gate defines

$$
\tau(x)=\{\,t\in\mathrm{toks}(x): |t|\ge 3 \;\wedge\; t\notin\mathcal{W}\,\}
$$

and the support score

$$
S(a^c,C)=
\frac{|\tau(a^c)\cap \bigcup_{c\in C}\tau(c)|}
{|\tau(a^c)|}\,,
$$

with admission if \(S(a^c,C)\ge \tau_s\). The lexical gate can be replaced by a binary supported/not-supported LLM judge without altering the policy structure [2605.27494].

The operator-facing safety metric is the unsafe-served rate. For \(N\) total queries, with indicator \(1_i^{ac}\) for whether query \(i\) was served from the answer cache and \(1_i^{dis}\) for whether the served answer disagrees with gold, the answer-cache hit rate is \(\frac{1}{N}\sum_i 1_i^{ac}\), the unsafe-served rate is \(\frac{1}{N}\sum_i (1_i^{ac}\cdot 1_i^{dis})\), and the conditional false-hit rate is \(USR/aHR\). The paper identifies USR as the production quantity to watch because it directly measures the fraction of all queries that received a wrong cached answer [2605.27494].

The empirical results are framed as safety-first rather than hit-rate-first. Across \(2\) datasets and \(12{,}000\) real-LLM generations using Qwen2.5-7B-Instruct on vLLM with Automatic Prefix Caching, GroundedCache drives USR to \(0.0\%\) on every HotpotQA regime, compared with \(15\)–\(35\%\) under naive caching, and to \(1.5\%\) on mtRAG document drift, compared with \(51.5\%\). It yields a \(34\times\) reduction on the design-point adversarial regime and \(3\)–\(10\times\) reductions across the other mtRAG regimes, while end-to-end p50 latency stays within \(1.04\)–\(1.07\times\) of a no-cache RAG baseline. In the summarized latency-versus-USR table, HotpotQA records \(1.09\) s and \(0.0\%\) for no-cache RAG, \(0.56\) s and \(17.2\%\) for naive cache, \(0.74\) s and \(12.5\%\) for no-support, and \(1.05\) s and \(0.0\%\) for GroundedCache; mtRAG records \(1.13\) s and \(0.0\%\), \(0.65\) s and \(26.1\%\), \(0.83\) s and \(18.2\%\), and \(1.06\) s and \(6.3\%\), respectively. A per-gate ablation identifies the lexical support gate as the load-bearing safety mechanism, with \(G1\), \(G2\), and \(G3\) providing defense in depth at near-zero cost [2605.27494].

## 6. Context maps as orientation caches for long-context agents

A different instantiation of the same general pattern appears in long-context LLM agents. Here the cache is not a binding table or an answer store, but a persistent context map capturing reusable orientation knowledge about a recurring external context. Formally, the context map at time \(t\) is

$$
M_t = \{r_1, r_2, \dots, r_k\}\,,
$$

where each entry is a tuple

$$
r_i=(id_i, section_i, content_i, tokSize_i, score_i, ts_i)\,.
$$

The permitted sections are \(\{\text{context\_roadmap}, \text{context\_understanding}, \text{domain\_constants}, \text{parsing\_schema}, \text{reusable\_results}\}\). The total token count is

$$
T_t=\sum_{r\in M_t} tokSize(r)\,,
$$

and the map enforces \(T_t \le B\), where \(B\) is a fixed token budget, for example \(1{,}024\) tokens [2605.19932].

Maintenance is performed by three modules. The Distiller receives the execution trajectory \(traj_t\) and current map \(M_{t-1}\), produces a diagnosis, tags existing entries as helpful, harmful, neutral, or stale, and extracts candidates \(C_t=\{c_1,\dots,c_m\}\) with scores \(score_j = LLM\_score(c_j \mid traj_t, M_{t-1}) \in [0,1]\). It examines REPL steps, code calls, sub-LLM calls, and self-analysis comments, and is instructed to assign high scores to transferable orientation facts and low scores to one-off task facts. The Cartographer computes a minimal set of edit operations \(E_t\) consisting of \(ADD\), \(REPLACE\), and \(DELETE\), then applies them to obtain \(M_{\text{temp}}\). The Evictor enforces the budget by sorting entries by ascending priority \(\pi(r)=score_r\), breaking ties by oldest timestamp, and removing the lowest-value entries until \(\sum tokSize \le B\). If the total number of user questions is \(n\) and the map evolves only for the first \(m \le n\) queries, the workflow initializes \(M_0=\varnothing\), prepends \(M_{t-1}\) to the system prompt at each iteration, runs the agent loop, and updates the map only while \(t \le m\); after that, the map is frozen [2605.19932].

The reported performance gains are substantial. Using GPT-5-mini inside the RLM agent, PEEK improves long-context aggregation on OOLONG from \(30.3\%\) to \(58.1\%\) on TREC-Q-coarse, from \(46.5\%\) to \(69.4\%\) on AGNews, and from \(23.0\%\) to \(57.0\%\) on Yahoo. On CL-bench, the solving rate rises from \(14.0\%\) to \(26.0\%\), while rubric accuracy rises from \(54.5\%\) to \(63.4\%\). Relative to ACE, iteration counts drop from \(523\) to \(378\) on TREC-Q, from \(491\) to \(398\) on AGNews, and from \(487\) to \(349\) on Yahoo, corresponding to \(145\), \(93\), and \(138\) fewer iterations. Total cost is also lower: on OOLONG, ACE costs \(5.8\times\)–\(1.7\times\) more than PEEK while scoring \(7.8\)–\(15.0\) percentage points lower in accuracy; on CL-bench, ACE costs \(1.4\times\) more and still trails by \(6.0\) percentage points in solve rate and \(9.9\) points in rubric accuracy. The method is stated to lie on the Pareto frontier of accuracy versus iterations and accuracy versus total dollar cost in all four benchmarks [2605.19932].

Generalization results indicate that the cache policy is model- and agent-agnostic. With GPT-5.5 as the base LM in the same RLM agent, TREC-Q improves from \(35.1\%\) to \(78.2\%\), AGNews from \(52.3\%\) to \(81.6\%\), Yahoo from \(30.0\%\) to \(71.0\%\), and CL-bench solve/rubric from \(32.0\%/62.4\%\) to \(38.0\%/65.6\%\). With Qwen3-Coder in RLM, the corresponding changes are \(42.0\%\) to \(56.0\%\), \(53.0\%\) to \(65.6\%\), \(32.0\%\) to \(58.0\%\), and \(2.0\%/47.3\%\) to \(6.0\%/48.1\%\). Replacing RLM with the OpenAI Codex CLI agent using GPT-5-mini yields \(32.0\%\) to \(76.0\%\) on TREC-Q, \(44.7\%\) to \(80.3\%\) on AGNews, \(22.0\%\) to \(74.0\%\) on Yahoo, and \(30.0\%/70.4\%\) to \(34.0\%/76.5\%\) on CL-bench solve/rubric [2605.19932].

## 7. Cross-domain interpretation

Taken together, these studies support a broader interpretation of map-rule caching. The routing model shows that reuse can be characterized analytically when locality assumptions hold and the replacement policy is simple. The grounded answer cache shows that high hit-rate alone is not an adequate objective when cached outputs can become semantically unsafe; safety requires explicit evidence-sensitive gates and an operator-facing error metric. The context-map cache shows that, in repeated long-context workloads, the object worth caching may be neither a full trajectory nor raw context, but a compact orientation artifact whose value is enforced through priority and token-budget rules [1312.1378] [2605.27494] [2605.19932].

This suggests three cross-cutting principles. First, cacheability is a semantic property, not merely a storage property: an RLOC↔EID binding, a cached answer, and a context-map entry are each reusable only under domain-specific invariants. Second, admission and eviction rules are central design objects rather than implementation detail. In the routing case, the governing rule is effectively LRU plus a traffic model; in GroundedCache, it is a conjunctive four-gate policy; in PEEK, it is a programmable maintenance stack culminating in priority eviction. Third, the right evaluation metric depends on failure mode. For mapping caches, miss probability and attack-induced miss inflation are decisive; for answer caches, USR is decisive; for context maps, solve rate, rubric accuracy, iteration count, and total cost matter jointly.

A final misconception common to all three areas is that caching is intrinsically a latency optimization. The networking results show that cache provisioning is also a control-plane scalability problem and an attack surface. The RAG results show that naive caching can be faster yet unsafe. The long-context results show that a small constant-sized prompt artifact can outperform larger or more passive context-preservation strategies. In that sense, a map-rule cache is best understood as a governed reuse mechanism: a cache whose utility depends on the quality of the rules that decide what is retained, when it is valid, and how it is consumed.

Source: https://www.emergentmind.com/topics/map-rule-cache