Structured Memory Access Control

• Structured Memory Access Control is a paradigm for enabling efficient multi-agent cache reuse through semantic encoding, quantization, and coordinated sharing mechanisms.
• It employs advanced methods like copy-on-write, predictive eviction, and LLM-based semantic validation to maintain correctness and adapt to dynamic workloads.
• Empirical results show significant improvements in memory efficiency, latency reduction, and throughput in multi-agent architectures using these structured access strategies.

Cache sharing across agents refers to the suite of architectural, algorithmic, and systems strategies that enable multiple agents—whether LLM-based, classical, or reinforcement learning (RL) driven—to efficiently reuse, coordinate, and manage caches representing context, intermediate computations, external tool calls, or data fragments. The central objective is to minimize redundant computation, reduce memory use, and lower data movement cost, all while maintaining correctness and adaptability across heterogeneous workloads. The field now encompasses token/key–value (KV) caches for transformers, semantic tool-result or knowledge caches, compressed pools for multi-agent inference, offline experience replay, distributed edge environments, and classical cooperative settings. Recent systems achieve substantial savings in both memory and latency by leveraging the high degree of overlap, semantic similarity, and workflow structure among agents, even as they contend with synchronization, correctness, cache divergence, and threat surfaces.

1. Architectural Paradigms for Multi-Agent Cache Sharing

Modern architectures for cache sharing across agents fall into several distinct categories:

• Shared Semantic Caches: Representing queries, tool invocations, or data lookups as embedding-based semantic elements, enabling cross-agent reuse where queries are only semantically similar, not identical. Asteria exemplifies this class, employing a two-stage approximate nearest neighbor (ANN) plus LLM-based validation pipeline to define semantic cache "hits" and build cross-region knowledge caches with performance-aware eviction and prefetching (Ruan et al., 22 Sep 2025).
• Shared and Disaggregated KV Caches for LLM Inference: In multi-agent, multi-LoRA, or pipeline workflows, systems such as ForkKV and LRAgent split the KV cache into shared "base" (backbone) and agent-specific "residual" or low-rank deltas, applying OS-inspired Copy-on-Write (CoW) or low-rank sharing to decouple storage and reconstruct context on-the-fly. DualRadixTree indexes, low-rank kernels (e.g., ResidualAttention and Flash-LoRA-Attention), and copy-on-demand mechanisms deliver both memory and compute savings at scale (Wang et al., 7 Apr 2026, Jeon et al., 1 Feb 2026).
• Compressed and Asymmetrically Quantized Shared Pools: PolyKV proposes a single, asymmetrically compressed shared cache pool—Keys in int8 for softmax stability; Values in 3-bit TurboQuant MSE—permitting multi-reader, lossy but high-fidelity injection by many concurrent agents, yielding over 97% memory savings while keeping perplexity degradation sub-1% (Patel et al., 27 Apr 2026). QKVShare allows quantized handoff of latent context via CacheCard artifacts with adaptive per-token bit width to support multi-agent on-device settings (Honavar et al., 5 May 2026).
• Collective and Diff-Aware Sharing: For synchronized multi-agent workflows, TokenDance exploits All-Gather synchronization to pay the KV reuse cost once and store sibling agent caches as sparse diffs against a master, resulting in 11–17× storage compression and sublinear compute cost scaling with agent count (Bian et al., 3 Apr 2026).
• Cross-Workflow and Dynamic Sharing: PBKV employs workflow-predictive eviction and prefetching, using K-step agent-invocation prediction to retain only high-reuse-potential cache in fast memory, supporting dynamic call graphs and robust performance under highly variable agent orchestration (Zheng et al., 7 May 2026).
• Cooperative RL-Based and Distributed Caching: Classical cooperative cache networks use multi-agent RL (e.g., CoM-Cache, MARL-coded caching) to coordinate placement, eviction, and coded content distribution among wireless or edge caches, with explicit MDP formulations and local–global reward coupling (Rezaei et al., 2018, Pedersen et al., 2021).
• Plan and Experience Caches: Embodied AI planning leverages per-agent plan transition caches (AgenticCache), while distributed RL experience replay benefits from reuse-aware and high-priority transition selection to boost cache locality and minimize bandwidth contention (AccMER) (Kim et al., 27 Apr 2026, Gogineni et al., 2023).

System-level innovations—such as GPU memory donation with cross-model NVLink streaming (SwiftCache (Hu et al., 15 Jun 2026)), and dynamic memory/time partitioning to avoid cache contention (Tokencake (Bian et al., 21 Oct 2025))—further enable high concurrency and context length by treating cache as a first-class, shared resource.

2. Core Methodologies: Design Patterns and Algorithms

Cache sharing designs typically hinge on three methodological pillars:

• Representation and Indexing: Semantic caches encode queries, tool calls, or data shards as vector embeddings. Approximate Nearest Neighbor indices or vector search (e.g., Faiss, HNSW) efficiently find potential reuse candidates, and further reranking or LLM-based validation checks for semantic fidelity (Asteria, Agent-Centric Data Fabric (Giurgiu et al., 10 Dec 2025)).
• Sharing and Reuse Protocols: Systems vary between:
  • Eager global sharing—single shared object or pool is written once and injected/mapped into agent contexts (PolyKV).
  • Disaggregated or partial sharing—base context is shared; per-agent (LoRA or workflow delta) is maintained separately and combined at read-out (ForkKV, LRAgent).
  • Diff or Anchor-based correction—reuse is possible only if local correction or delta is available; as in KVCOMM, where anchor pools track offset-variance to align caches under non-identical prefixes (Ye et al., 14 Oct 2025).
• Eviction and Prefetching: Cache entries are scored by expected future utility, frequency, cost, or reuse potential (Asteria’s LCFU, PBKV’s K-step lookahead, agent-centric utility functions). Secure and efficient sharing depends on robust policies—e.g., lifecycle-first/score-second eviction to manage dynamic, loop-heavy workflows (Zheng et al., 7 May 2026)—plus conservative prefetching that leverages otherwise idle memory bandwidth without displacing high-value active cache.

Pseudocode for these mechanisms often features clearly-defined scoring/value formulas, deterministic eviction policies (evict entries with lowest computed value), and prediction-driven offload/prefetch intertwined with real-time agent/scheduler signals.

3. Correctness, Threat Surfaces, and Integrity Mechanisms

Ensuring semantic correctness and system integrity is a central concern:

• Semantic Validation and Correctness: Binary cache hit definitions combine embedding similarity (e.g., cosine distance) with an LLM-powered semantic judger, ensuring that only validated semantic matches are returned, with fallback on cache miss to remote or tool call. Retrieval thresholds (such as $\tau_\mathrm{sim}, \tau_\mathrm{lsm}$) are continuously recalibrated to satisfy high-precision targets (>99%) (Ruan et al., 22 Sep 2025).
• Security Against Tampering: When cache states or KV memories are explicitly shared as part of the agent communication protocol (e.g., via hidden latents in multi-agent QA), threats arise from adversaries who can inject, corrupt, or substitute cache data. Integrity must be protected at transport via cryptographically strong HMAC manifests binding agent ID, session, model, visible commitments, tensor metadata, and payload digest. All tampered payloads must be dropped or downgraded to visible-only fallback immediately upon failed verification (Brito et al., 27 Jun 2026).
• Trade-offs for Lossy Compression: Asymmetric quantization of keys and values (PolyKV) or adaptive quantization (QKVShare) trades minor degradation in perplexity or accuracy for order-of-magnitude memory reduction, but empirical results show loss remains sub-1% and actually decreases as more repetitive or coherent tokens are shared across agents (Patel et al., 27 Apr 2026, Honavar et al., 5 May 2026).
• Consistency and Robustness in Workflow-Level Sharing: Correctness is preserved via deterministic cache update/evict policies and by guarding against prediction or prefetch errors (via graceful fallback and careful separation of active/retired cache). In judge-centric LLM workflows, naive reuse can undermine selection invariance (low JCR) even as accuracy remains nominal, necessitating explicit meta-reasoning gating, interaction-preserving strategies, or selective cross-block recomputation to maintain trust and auditability in agent aggregation (Liang et al., 13 Jan 2026).

4. Quantitative Impacts and Benchmarks

Extensive evaluations across all domains demonstrate strong, generalizable benefits:

• Throughput and Scalability: TokenDance supports up to 2.7× more concurrent agents under strict latency SLOs (e.g., QPS=10) and achieves 1.9× prefill speedup over per-request caching. ForkKV and LRAgent deliver 3.0× and up to 4.4× throughput improvements at scale by physically sharing massive context blocks (Bian et al., 3 Apr 2026, Wang et al., 7 Apr 2026, Jeon et al., 1 Feb 2026).
• Memory Efficiency: PolyKV achieves up to 97.7% reduction in total KV memory usage for Llama-3-8B workloads (e.g., 19.8 GB to 0.45 GB for 15 agents at 4K tokens) (Patel et al., 27 Apr 2026). LRAgent and ForkKV both reduce practical KV memory demand to $\sim$1/N of non-shared settings, with negligible accuracy loss (<1.6 pp).
• Latency and Latency Variance: SwiftCache realizes 54–69% reduction in P99 time-to-first-token (TTFT) on real-world multi-turn workloads, by importing blocks via high-bandwidth NVLink with near-zero (<0.1%) overhead relative to prefill time (Hu et al., 15 Jun 2026). PBKV delivers up to 1.85× workflow speedup over LRU, with GPU hit rates as high as 79.9% on dynamic workflows (Zheng et al., 7 May 2026). Tokencake can reduce end-to-end latency by more than 47% by combining agent-aware scheduling and proactive offload with reserved critical-path partitions (Bian et al., 21 Oct 2025).
• Empirical Soundness: Systems carefully quantify and bound any possible quality loss under compression, reuse, or quantization—PolyKV and QKVShare report mean BERTScore F1 ≈ 0.928–0.970 and <1.5% accuracy delta even under 5-hop repeated handoffs (Patel et al., 27 Apr 2026, Honavar et al., 5 May 2026). AgenticCache demonstrates 65% simulation latency reduction in embodied multi-agent benchmarks by eliminating redundant LLM calls via local, per-agent plan transition caching (Kim et al., 27 Apr 2026).

5. Integration with Execution and Orchestration Frameworks

Cache sharing has become a central design criterion in next-generation agentic and data management fabrics:

• Data Fabrics: Agent-centric data architectures embed semantic micro-caches per agent, but rely on an orchestration layer (router, cross-agent cache manager, prefetcher, quorum server) to coordinate context-driven sharing, attention-guided data prefetch, and early return protocols. Semantic cache entries are indexed by vector ANN, scored for utility, and shared among agents when context embeddings exhibit high cosine similarity above a sharing threshold (Giurgiu et al., 10 Dec 2025).
• Workflow- and Policy-Driven Serving: Prediction-based and context-aware sharing (PBKV, Tokencake) leverage knowledge of agent invocation graphs, adaptive memory partitioning, and time-aware offload or upload, requiring only lightweight meta-information communication. Real-time load, priority, and execution forecasts feed into deterministic yet adaptive partitioning and scheduling policies.
• RL Systems and Cooperative Networks: In RL, key/replay buffer sharing via windowed, priority-aware reuse patterns (AccMER) delivers 15–25% overall training acceleration at scale without disrupting convergence; decentralized, locality-aware policies (e.g., CoM-Cache, MARL coded caching) outperform naive independent learning and classical eviction by exploiting network interactions and coded fragments (Gogineni et al., 2023, Rezaei et al., 2018, Pedersen et al., 2021).

6. Limitations, Failure Modes, and Open Directions

Despite demonstrated gains, challenges remain:

• Agent Divergence: Methods like TokenDance and collective reuse depend on high shared-context overlap. Divergence or drift in agent context induces large diffs and diminishes amortization. Semi-structured, asynchronous, or highly dynamic environments require hybrid or fallback paths.
• Semantic Drift and Non-Invariance: Judge-side KV cache reuse can substantially alter selection behavior while end-task accuracy remains stable, revealing a silent failure mode if cross-candidate attention structures are not preserved (Liang et al., 13 Jan 2026).
• Security and Trust: Transport-level integrity must be enforced to prevent subtle or catastrophic compromise of agent-shared latent KV states. Adaptive attacks can evade magnitude or anomaly checks; cryptographic manifest protocols offer robust defense but increase system complexity (Brito et al., 27 Jun 2026).
• Complexity and Engineering Overhead: Integrated, high-performance cache sharing—especially in quantized or disaggregated settings—requires custom kernel implementations (e.g., ResidualAttention, Flash-LoRA) and sophisticated memory management. Compression, quantization, and dynamic prediction introduce new engineering and tuning demands.
• Generality and Applicability: Some techniques assume homogeneous models or system stacks (e.g., NVLink, shared LoRA bases), while practical deployments may entail hybrid, device-heterogeneous or cross-cloud topologies. Open questions remain on dynamically extending to multi-adapter, non-LoRA, or general distributed inference.

Current research is actively exploring adaptive cross-modal cache, fine-grained trust and provenance protocols, and cooperative cache coordination in federated, privacy-preserving, and adversarial environments. Integration with future proof-of-stake or attestation mechanisms and systemically robust design is an area of continued importance.

---

References

• (Ruan et al., 22 Sep 2025) Asteria: Semantic-Aware Cross-Region Caching for Agentic LLM Tool Access
• (Wang et al., 7 Apr 2026) ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache
• (Jeon et al., 1 Feb 2026) LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
• (Patel et al., 27 Apr 2026) PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
• (Honavar et al., 5 May 2026) QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMs
• (Bian et al., 3 Apr 2026) TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
• (Giurgiu et al., 10 Dec 2025) Supporting Dynamic Agentic Workloads: How Data and Agents Interact
• (Hu et al., 15 Jun 2026) SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing
• (Ye et al., 14 Oct 2025) KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
• (Bian et al., 21 Oct 2025) Tokencake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
• (Kim et al., 27 Apr 2026) AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents
• (Gogineni et al., 2023) AccMER: Accelerating Multi-Agent Experience Replay with Cache Locality-aware Prioritization
• (Rezaei et al., 2018) Multi-agent Learning for Cooperative Large-scale Caching Networks
• (Pedersen et al., 2021) Dynamic Coded Caching in Wireless Networks Using Multi-Agent Reinforcement Learning
• (Zheng et al., 7 May 2026) Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
• (Liang et al., 13 Jan 2026) When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges
• (Brito et al., 27 Jun 2026) When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration
• (Tang et al., 6 Jan 2025) Enhancing Lifelong Multi-Agent Path Finding with Cache Mechanism