KVCOMM: Transformer KV-Cache Communication

• KVCOMM is a collection of methods that treat transformer key-value caches as a reusable communication substrate, enhancing efficiency in multi-agent language model systems.
• Key techniques include online cross-context cache reuse with RoPE alignment, selective layer KV sharing based on attention scores and Gaussian priors, and adaptive quantization for compression.
• Empirical results demonstrate up to 7.82x speedup, 5–6x reduction in computation, and significant bandwidth savings, highlighting the practical benefits of KV-cache communication.

Searching arXiv for papers on KVCOMM and closely related KV-cache communication/compression work. KVCOMM denotes a line of research that treats transformer key-value (KV) caches as a communication substrate for LLM systems, especially when repeated text-based exchange causes redundant transmission and redundant inference. In the 2025 literature, the name is used for multiple closely related proposals: an online cross-context KV-cache reuse framework for homogeneous multi-agent prefilling (Ye et al., 14 Oct 2025), a selective inter-LLM KV-sharing method for sender–receiver communication (Shi et al., 2 Oct 2025), and a later compressed protocol, Q-KVComm, that adds adaptive quantization, explicit fact extraction, and heterogeneous-model calibration (Kriuk et al., 27 Nov 2025). Across these formulations, the common premise is that internal attention memory can be transferred, aligned, or approximated more efficiently than re-encoding overlapping text from scratch.

1. Scope and named formulations

The label "KVCOMM" is not attached to a single algorithmic object. It refers to a cluster of methods that all move communication away from surface text and toward transformer KV state, but they do so under different assumptions and with different system goals (Ye et al., 14 Oct 2025).

Formulation	Primary mechanism	Reported evidence
KVCOMM	Online cross-context KV-cache reuse with anchor-based offset correction and RoPE alignment	over 70% reuse rate; up to 7.82x speedup; TTFT from about 428.6 ms to about 55 ms
KVComm	Selective sharing of top-scoring layers' KV pairs from a sender to a receiver	comparable to Skyline while transmitting as few as 30% of layers' KV pairs; 2.5x to 6x computation reduction
Q-KVComm	Direct transmission of compressed KV caches plus extracted facts, with adaptive quantization and calibration	5–6x compression ratios; coherence quality scores above 0.77; up to 6.93x compression

The first KVCOMM formulation models a multi-agent system as a directed graph $\mathcal{G}=(\mathcal{M},\mathcal{E})$, assumes agents are instances of the same RoPE-based decoder-only LLM checkpoint, and focuses on eliminating repeated prefilling when the same text appears under different prefixes (Ye et al., 14 Oct 2025). The second KVComm formulation studies a two-LLM sender–receiver setting in which the models are either two instances of the same LLM or two models fine-tuned from the same base LLM, and it frames communication as selective transmission of whole layers' KV pairs (Shi et al., 2 Oct 2025). Q-KVComm broadens the paradigm further by treating compressed, serialized KV tensors as communicated objects and combining them with a compact explicit fact summary (Kriuk et al., 27 Nov 2025).

2. Why KV-cache communication is needed

The central systems argument is that standard text communication discards already-computed semantic state. In Q-KVComm's formulation, an upstream agent reads long context, forms latent semantic structure internally, emits text, and forces the receiver to reconstruct nearly the same latent structure from scratch; the paper identifies both bandwidth inefficiency and compute inefficiency in that workflow (Kriuk et al., 27 Nov 2025). KVCOMM-style methods therefore treat keys and values as reusable internal state rather than ephemeral decoding by-products.

The inefficiency is particularly acute in multi-agent pipelines. In the online cross-context framework, every agent must run a prefill pass on its entire input before decoding, yet many prompts contain substantial overlap in user questions, retrieved passages, upstream agent outputs, or previous-turn content. Ordinary prefix caching does not solve this when the same shared text appears under different surrounding prefixes, because the resulting KV-cache changes with both position and local context; the paper names this obstacle KV-cache offset variance (Ye et al., 14 Oct 2025).

A related critique appears in the selective KV-sharing formulation. That paper argues that natural-language communication is lossy because information is lost in the sampling process as each token is produced, and it argues that hidden-state communication suffers from information concentration bias: later layers in decoder-only LLMs place disproportionate importance on the last token's hidden state, so replacing or averaging hidden states can corrupt the receiver's own representation (Shi et al., 2 Oct 2025). KV pairs are presented as preferable because the receiver can decide how much to use them through its own attention mechanism rather than having them overwrite internal hidden states.

A plausible synthesis is that KVCOMM is best understood not as a caching trick in the narrow sense, but as a representation-exchange abstraction for multi-agent inference. Prefix reuse, cross-context reuse, and compressed transmission are different instances of that broader abstraction.

3. Online cross-context KV-cache communication

The online KVCOMM framework is designed for prompts with alternating fixed prompt text and runtime-filled placeholders. Before serving requests, each agent precomputes and stores KV-caches for all fixed prefix segments in its prompt template. For each placeholder, the system checks whether a base KV-cache already exists in shared memory; if not, it computes and stores it. Reuse then depends on whether the placeholder is judged shareable, otherwise the system falls back to dense prefilling and records the true offset as a new anchor for future use (Ye et al., 14 Oct 2025).

Its core mechanism is an anchor pool. For each placeholder-specific pool, an anchor stores a base KV-cache, placeholder offset, neighboring prefix offset, embedding tensor, and usage statistics. For a new placeholder $\phi_{(m,i)}$, the approximated placeholder cache is

$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$

and the approximated neighboring prefix cache is

$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$

with interpolation weights given by

$$w_{\phi\rightarrow\psi}=\mathrm{softmax}(-\|\mathbf{h}_\phi-\mathbf{h}_\psi\|).$$

The framework also performs de-rotation and re-rotation for Keys to correct RoPE-induced positional mismatch, whereas Values are adjusted directly (Ye et al., 14 Oct 2025).

The paper provides two theoretical propositions motivating anchor retrieval. One bounds the KV distance between different tokens under the same prefix by their embedding gap, and the other bounds the difference between cache deviations under different prefixes, again by embedding distance after positional alignment. These propositions are used to justify embedding-nearest anchors for offset estimation rather than direct dense recomputation (Ye et al., 14 Oct 2025).

Experimentally, the method is evaluated on retrieval-augmented generation using MMLU, math reasoning using GSM8K, and collaborative coding using HumanEval, in fully connected systems with 2 to 5 agents. It reports over 70% reuse rate across workloads, around $6.7\times$ average prefill speedup in the three-agent setting, and up to $7.82\times$ speedup in a five-agent setting with 1K input tokens, 512 prefix tokens, and 512 output tokens, reducing TTFT from about 428.6 ms to about 55 ms. The ablations are structurally important: on MMLU with four agents, key rotation only yields 43.1% accuracy, placeholder offset only 58.8%, prefix offset only 60.1%, and using all three components yields 68.0%, indicating that positional alignment, placeholder correction, and prefix correction are jointly necessary (Ye et al., 14 Oct 2025).

The method is explicitly limited to homogeneous agents using the same architecture/checkpoint, structured prompt segmentation into placeholders, and prefill acceleration only. It does not accelerate decoding latency, and long-context deployment can be dominated by CPU offloading of anchor KVs; for 4K-token anchor matching on MMLU with Llama-3.1-8B, the appendix reports softmax latency of 104–122 ms, offloading latency per agent of 1260–1300 ms, and memory cost of 68.5–95.1 GB (Ye et al., 14 Oct 2025).

4. Selective KV sharing between sender and receiver LLMs

The second KVComm formulation considers a sender model $\mathcal{M}_s$ that reads context $C$ and a receiver model $\mathcal{M}_r$ that reads query $\phi_{(m,i)}$0. Communication is one-way: the sender computes per-layer KV pairs $\phi_{(m,i)}$1, selects a subset of layers, and the receiver concatenates those transmitted KV tensors with its own cache at matching layers,

$\phi_{(m,i)}$2

No training objective is introduced; the method is explicitly training-free (Shi et al., 2 Oct 2025).

Layer selection is based on an attention importance score and a Gaussian prior over depth. The unnormalized attention score is

$\phi_{(m,i)}$3

which is normalized to $\phi_{(m,i)}$4. The depth prior is

$\phi_{(m,i)}$5

and the final score is

$\phi_{(m,i)}$6

The implementation sets $\phi_{(m,i)}$7, $\phi_{(m,i)}$8, and $\phi_{(m,i)}$9 for Llama family models and $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$0 for Qwen and Falcon family models. Communication ratios of 0.3, 0.5, and 0.7 correspond to transmitting 30%, 50%, and 70% of layers' KV pairs (Shi et al., 2 Oct 2025).

The selective scheme is motivated by two hypotheses stated in the paper: intermediate layers contain the most transferable semantic knowledge, and layers with stronger attention distributions are more effective for communication. The paper contrasts this with hidden-state communication, including AC-style last-token exchange and all-token hidden-state transfer, and argues that KV sharing preserves contextual information without directly overwriting the receiver's hidden state (Shi et al., 2 Oct 2025).

The evaluation spans eight contextual reasoning datasets—Countries, Tipsheets, HotpotQA, QASPER, MuSiQue, MultiFieldQA-en, 2WikiMQA, and TMATH—and eight model pairs, including same-model pairs and fine-tuned-from-the-same-base pairs. The principal upper bound is Skyline, where the receiver directly processes concatenated context and query without any communication bottleneck. Across the reported tables, KVComm at 70% layer sharing is the most consistently near-Skyline regime, though the abstract states comparable performance while transmitting as few as 30% of layers' KV pairs. The method also reports up to $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$1 reduction in communication relative to transmitting the entire set of KV pairs and $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$2 to $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$3 reduction in computation cost compared to Skyline (Shi et al., 2 Oct 2025).

Several ablations clarify the method's structure. Random layer selection is markedly worse than the attention-plus-Gaussian score at 30% and 50% budgets. Non-contiguous top-$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$4 selection is more robust than selecting a single contiguous layer block. A calibration set of even one sample performs as well as up to 128 samples in the reported experiment. The paper also notes that gains are not substantial on TMATH, suggesting that communication benefits are task dependent even when architectures are aligned (Shi et al., 2 Oct 2025).

5. Compression, heterogeneous calibration, and adjacent KV methods

Q-KVComm extends the representation-exchange perspective by making compressed KV transport itself the communication protocol. Its pipeline has four stages—layer selection, hybrid information extraction, adaptive quantization, and cross-model calibration—and the sender transmits compressed key/value caches plus a compact explicit fact summary. The receiver de-serializes, dequantizes, optionally calibrates for a different receiver architecture, and integrates the result into attention by concatenation along the sequence dimension. The paper explicitly frames the system as a dual-channel communication protocol: implicit knowledge transfer via KV caches and explicit knowledge transfer via short textual facts (Kriuk et al., 27 Nov 2025).

Compression is not a single low-bit quantizer but a layered system. Q-KVComm first selects the top proportion of layers according to

$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$5

with a reported layer selection ratio of 0.7. It then profiles quantization sensitivity using expected squared reconstruction error and assigns precision by percentile: top 30% most sensitive layers to 8-bit, middle 40% to 6-bit, and bottom 30% to 4-bit. The protocol bit-packs the results, with 4-bit quantization storing two values per byte and 6-bit quantization storing four values per three bytes. It also provides a zero-shot statistical calibration,

$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$6

and uses scalar statistics averaged across dimensions when hidden dimensions differ (Kriuk et al., 27 Nov 2025).

Its reported results are a compression–quality tradeoff rather than a pure speed benchmark. On SQuAD, HotpotQA, and NarrativeQA, Q-KVComm reports $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$7 compression at 4-bit, $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$8 to $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\phi_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\phi_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\phi},$$9 at 6-bit, and $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$0 to $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$1 at 8-bit. Coherence remains above 0.77 across all scenarios, with SQuAD at 0.925–0.943, NarrativeQA at 0.937–0.957, and HotpotQA at 0.777–0.797. The runtime behavior is not monotone in bit-width: 8-bit is fastest, 6-bit is slower than 4-bit, and the paper recommends 8-bit for latency-sensitive applications and 4-bit for bandwidth-limited deployments. The abstract also claims less than 5% quality degradation in heterogeneous deployments, but the provided excerpt does not contain detailed numeric tables isolating that claim (Kriuk et al., 27 Nov 2025).

Two adjacent papers are particularly relevant because they address the representational and systems constraints that KVCOMM-style protocols inherit. "KVCompose" constructs composite tokens so that each compressed layer remains a dense tensor

$$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$2

which the paper explicitly argues is easier to store, page, serialize, and move than ragged per-head sparse structures; it emphasizes that the method is not explicitly a distributed communication protocol, but is directly relevant to communication-compatible layout (Akulov et al., 5 Sep 2025). "CommVQ" uses additive vector quantization and a RoPE-commutative key codebook, reports 87.5% FP16 KV-cache size reduction with 2-bit quantization, supports about 1-bit average precision, and provides a Triton implementation with per-layer per-token latency speedups of $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$3 at 8K context, $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$4 at 32K, and $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$5 at 128K (Li et al., 23 Jun 2025). These methods are not themselves KVCOMM protocols, but they define practical compression primitives that a KVCOMM deployment could exploit.

6. Misconceptions, assumptions, and open problems

A common misconception is that KVCOMM is merely ordinary prefix caching under another name. The online cross-context work explicitly addresses the opposite regime: the same shared text under different surrounding prefixes, where direct cache reuse fails because of offset variance across agents and RoPE position mismatch (Ye et al., 14 Oct 2025). Another misconception is that any internal representation would suffice. The selective-sharing paper explicitly argues that hidden-state communication suffers from information concentration bias and destructive interference with the receiver's own last-token state, whereas KV sharing preserves an external attention memory (Shi et al., 2 Oct 2025).

A further misconception is that KVCOMM already solves arbitrary heterogeneous inter-model communication. The selective-sharing framework assumes the sender and receiver are either the same model or fine-tuned from the same base LLM, with direct layer-index alignment and compatible KV tensor shapes (Shi et al., 2 Oct 2025). Q-KVComm does attempt cross-architecture transfer, but its calibration is limited to first- and second-order statistics, uses scalar statistics when dimensions mismatch, and the provided text explicitly notes the absence of richer alignment matrices, supervised alignment losses, or detailed large-model evidence beyond compact 1.1B–1.5B settings (Kriuk et al., 27 Nov 2025).

The empirical picture is therefore mixed in a productive way. The strongest support concerns prefill reuse and bandwidth reduction: over 70% reuse rate and up to $$(\hat{\mathbf{k}}/\hat{\mathbf{v}})_{\mathbf{p}_{(m,i)}}=(\mathbf{k}/\mathbf{v})_{\mathbf{p}_{(m,i)}}+\sum_{\psi\in\mathcal{A}_{\phi_{(m,i)}}} w_{\phi_{(m,i)}\rightarrow\psi}\cdot \Delta(\mathbf{k}/\mathbf{v})_{(m,\psi)}^{\mathbf{p}},$$6 TTFT speedup for online cross-context KVCOMM, and 5–6x compression with high coherence for Q-KVComm (Ye et al., 14 Oct 2025). The evidence is weaker for full heterogeneity, rigorous component-wise ablations in the compressed setting, and generalization to larger 7B+ transfer scenarios (Kriuk et al., 27 Nov 2025). Task sensitivity also remains visible: HotpotQA is materially harder than SQuAD or NarrativeQA under aggressive compression, and TMATH shows weaker gains for selective sharing (Shi et al., 2 Oct 2025).

Taken together, the literature indicates that KVCOMM is significant because it redefines communication in LLM systems from "send words" to "send reusable attention state." In its current forms, that redefinition is already sufficient to yield large prefill-speed and bandwidth gains under templated, overlap-heavy, mostly homogeneous deployments. The unresolved questions concern dynamic unstructured prompts, stronger heterogeneous alignment, long-context anchor movement costs, and the co-optimization of prefill reuse with decode-time efficiency.