---
title: Cache Sharing in Multi-Agent Systems
url: https://www.emergentmind.com/topics/cache-sharing-across-agents
type: topic
---

# Cache Sharing in Multi-Agent Systems

Cache sharing across agents refers to collaborative strategies, architectural mechanisms, and optimization algorithms that allow multiple autonomous entities—typically LLM-based agents or distributed RL nodes—to read from, write to, or coordinate over shared computational and storage caches. The goal is to substantially reduce redundant computation, memory consumption, and data transfer, while preserving task-specific correctness, agent isolation, and scalability. Modern research defines and implements cache sharing not just via low-level tensor reuse (as in transformer KV caches), but more generally through techniques spanning semantic vector caches, lossy compression, quantized handoff, copy-on-write semantics, asynchronous collective reuse, and adaptive multi-agent prioritization. The field emphasizes pragmatic trade-offs among throughput, hit rate, memory utilization, consistency, and security, especially in large-scale or highly dynamic multi-agent workflows.

## 1. Design Principles and Fundamental Abstractions

Effective cache sharing relies on a precise specification of what is shared, who may access or mutate the cache, and how correctness and efficiency are maintained. Leading systems introduce the following abstractions:

- **Semantic Elements (SEs):** Each agent-issued remote call or query is encapsulated as an SE: a tuple $(q, r, e_q, \ell, c, s, \mathrm{size}, \mathrm{freq})$ comprising the raw query $q$, the response $r$, a semantic embedding $e_q$, performance metadata (latency $\ell$, monetary cost $c$, staticity score $s$), the response size, and access frequency. SEs populate the semantic cache [2509.17360].
- **KV-Cache Blocks:** Transformer-based agents maintain key–value (KV) caches at each layer/head for each prefix or trajectory segment. Cache block granularity is critical for inter-agent reuse and fine-grained management [2602.01053, 2604.24971, 2604.03143].
- **Asymmetrically Compressed Shared Pools:** Systems such as PolyKV maintain a single, lossy-compressed cache pool that is concurrently accessed by many agents, decoupling per-agent inference from redundant full-precision cache instantiation [2604.24971].
- **Copy-on-Write (CoW) and Dual-Tier Indexing:** ForkKV implements an OS-inspired CoW mechanism, separating shared base KV cache from agent-specific residuals and using scalable radix trees for per-agent divergent writes [2604.06370].
- **Anchor Pools for Offset Correction:** KVCOMM enables offset-aligned cache handoff by dynamically maintaining anchor pools mapping context embeddings to empirically observed cache offsets, facilitating cross-context reuse under diverse prefix extensions [2510.12872].

## 2. Semantic Caching and Retrieval Algorithms

Semantically aware agent cache sharing seeks to maximize reuse not just for syntactically identical queries but also for semantically similar or paraphrased statements, tool invocations, or prompts.

- **Semantic Retrieval Indexing (Sine):** A two-stage retrieval mechanism: (a) vector-similarity search using approximate nearest neighbor (ANN) over semantic embeddings to identify candidate cache elements, followed by (b) an LLM-powered judger that semantically validates the relevance of those candidates, enforcing a calibrated precision threshold (e.g., ≥99%) [2509.17360].
- **Semantic Cache Hit Definition:** A cache hit is realized only if both the embedding similarity and semantic judger jointly validate the match: 
  $$
  \mathrm{Hit}(q,SE) = \mathbf{1}[ d(e_q, e_{SE}) \le \tau_{sim} \wedge J(q, r_{SE}) = \mathrm{true} ].
  $$
  This ensures precise semantic correctness in multi-agent tools and eliminates false positives [2509.17360].
- **Micro-caching and Attention-Guided Reuse:** Agent-centric data fabrics rely on encode–lookup-evict policies over semantic micro-caches (vector-indexed small fragments), coordinated by attention-guided routers and cross-agent cache managers that propagate high-utility cache fragments across agents on semantic overlap [2512.09548].

## 3. Shared KV-Cache Mechanisms

State-of-the-art transformers in multi-agent systems maintain large-scale KV caches, whose naive per-agent replication is infeasible. Modern solutions include:

- **KV-Cache Pooling and Decompression:** PolyKV’s SharedKVPool compresses all key tensors at int8 (q8_0) and value tensors via 3-bit Fast Walsh-Hadamard Transform (TurboQuant MSE), achieving stable 2.91× compression while preserving softmax stability and downstream accuracy ($\Delta$PPL typically <1%) [2604.24971].
- **Copy-on-Write and ResidualAttention:** ForkKV’s CoW decouples agent-specific low-rank activations from a shared base, and reconstructs disaggregated KV cache directly within on-chip SRAM via ResidualAttention. This achieves >10× memory savings and up to 3× throughput under multi-LoRA workloads, with minimal quality degradation [2604.06370].
- **Low-Rank Adapter KV Sharing:** LRAgent decomposes KV into base and low-rank adapter components, sharing the base across agents and, subject to adapter orthogonality, sharing the low-rank cache as well—materializing agent-specific deltas only as needed. The Flash-LoRA-Attention kernel avoids explicit expansion of low-rank cache, maintaining high throughput [2602.01053].
- **Quantized Handoff and Mixed-Precision Cards:** QKVShare formalizes agent-to-agent cache transfer as a quantized CacheCard with token-level mixed-precision (adaptive bit-width from 2–16), enabling up to 2.8× density over FP16 and sublinear handoff latency scaling (TTFT reduced from 1,030 ms to 397 ms at 8K tokens) while preserving multi-hop accuracy within 1–3% of baseline [2605.03884].
- **Diff-Aware KV Sharing:** TokenDance exploits synchronized all-gather patterns among agents, collectively computing a master KV-cache and storing per-agent block-sparse diffs, yielding 11–17× compression and scaling to 2.7× more agents under SLO than prefix caching [2604.03143].

## 4. Eviction, Prefetching, and Dynamic Resource Management

Multi-agent cache sharing necessitates advanced eviction and data movement strategies to avoid catastrophic contention while targeting near-optimal reuse.

- **Value-Weighted and Lifecycle-Aware Eviction:** Algorithms like LCFU (Least Cost-Efficient and Frequently Used) and PBKV’s Score(c) combine access frequency, cost, retrieval latency, and staticity to prioritize entries. Hierarchical eviction ensures that retired cache (not needed by any live workflow) is always evicted before high-score active cache [2509.17360, 2605.06472].
- **Predictive and Proactive Prefetching:** Markov chain models or agent-embedding sequence predictors guide asynchronous background fetches of likely-to-be-used cache entries, always loading into otherwise idle memory to avoid displacing high-value live cache [2509.17360, 2605.06472].
- **Elastic Memory Pooling and Peer-to-Peer Exchange:** SwiftCache orchestrates on-server, NVLink-backed sharing of idle GPU memory, streaming prefix cache blocks from low-demand (worker) models to high-demand (master) ones, pipelined per-layer. This decouples context length support from single-model HBM constraints and achieves up to 3.98× extension in supported context length relative to conventional strategies [2606.16135].
- **Agent-Centric Partitioned and Time-Aware Scheduling:** Tokencake uses a space scheduler for partitioning GPU cache across critical vs. non-critical agents, and a time scheduler for proactively offloading stalled agent caches (e.g., during function calls) with predictive upload, maximizing effective GPU utilization (boosting cache occupancy by ~17%) and reducing end-to-end latency by 47% [2510.18586].

## 5. Security, Consistency, and Correctness Considerations

Integrating shared cache mechanisms into multi-agent systems introduces new attack surfaces and necessitates robust correctness guarantees.

- **KV-Cache Integrity and Tamper Detection:** When latent agents transmit full KV-cache state (e.g., in latent communication protocols), the system must cryptographically bind each cache payload to session, specialist identity, payload digest, and visible commitment via HMAC-SHA256 manifests. This achieves 100% detection of tampered payloads versus pure text-based verification, which can be bypassed [2606.28958].
- **Cross-Agent Consistency and Risk-Aware Reuse:** For judge-centric tasks, naive KV reuse may degrade "judge consistency rate" (JCR), as block-diagonalization weakens cross-candidate interactions. Dedicated interaction-preserving strategies or conservative meta-reasoning gating are necessary to ensure correct candidate selection [2601.08343].
- **Locking and Deadlock Avoidance in Non-LLM Domains:** In multi-agent path-finding (e.g., L-MAPF-CM), shared cache cells (for items or paths) are managed with per-agent locking protocols and six-state machines, guaranteeing deadlock- and starvation-freedom even with aggressive agent concurrency [2501.02803].

## 6. Applications, Empirical Performance, and Scalability

Empirical results across a range of domains and agent architectures confirm the impact and trade-offs of cache sharing strategies:

- **LLM Workloads:**
    - Asteria shows cache hit rates up to 85–95% with 3.6× throughput gain and 6× improvement in cost-per-request over non-semantic caching on search and code generation tasks [2509.17360].
    - PolyKV demonstrates >97% memory savings and near-constant perplexity (even improving as context grows) for up to 15 agents with shared compressed cache pools [2604.24971].
    - TokenDance achieves up to 2.7× more agents serviced under SLO and 11–17× per-agent cache compression [2604.03143].
- **Edge and On-Device Scenarios:** QKVShare supports high agent density with controlled accuracy loss (≤3%) and handoff latencies hundreds of ms below re-prefill at 8K tokens [2605.03884].
- **RL and Caching Networks:** AccMER achieves 20–25% training time reduction via windowed high-priority replay buffer sharing, significantly reducing cache/TLB misses with maintained learning performance in centralized multi-agent RL [2306.00187].
- **Wireless Coded Caching:** MARL-based coded cache sharing in wireless networks achieves up to 20% lower network load than static policies under nonuniform spatial demand, by learning contiguous, non-redundant codeword placements [2104.06724].

## 7. Limitations, Open Challenges, and Future Directions

Despite substantial advances, several areas remain active for research and refinement:

- **Workflow and Domain Generality:** Systems like PBKV target dynamic agent workflows, but broader adoption requires further decoupling from static DAGs and improved cross-agent semantic prediction [2605.06472].
- **Consistency vs. Reuse Trade-Offs:** Empirically, high reuse may silently degrade decision invariance in judge-centric aggregation; safe deployment requires hybrid approaches that combine on-demand dense prefill with risk-aware, instance-level gating [2601.08343].
- **Security:** As cache-sharing extends to latent memory handoff, cryptographic payload binding and auditability become non-negotiable, with ongoing work on trusted execution and semantic anomaly detection [2606.28958].
- **Overhead of Compression/Decompression:** Lossily compressed pools (e.g., PolyKV) may add pipeline latency, and decompression cost, though amortized, should be benchmarked under high concurrency.
- **Hardware and Heterogeneity:** Techniques such as SwiftCache and Tokencake assume fast-server interconnects (NVLink), and further research is needed for heterogeneous, cloud-edge, and mobile deployments [2606.16135, 2510.18586].
- **Controller Adaptivity and Scalability:** Many quantization and prediction heuristics require further ablations to robustly navigate high-depth, multi-hop, or dynamic-topology agent networks [2605.03884].

Cache sharing across agents is now central to enabling efficient, scalable, and semantically rich multi-agent systems—spanning LLM inference at web and edge scale, reinforcement learning, distributed caching networks, embodied robotics, and beyond. The collective trajectory of the field is toward dynamic, prediction-driven, trust-preserving sharing protocols that adapt fluidly to agent workload, workflow structure, and infrastructure constraints.

Source: https://www.emergentmind.com/topics/cache-sharing-across-agents