Q-KVComm: Efficient KV Communication

• Q-KVComm is a protocol that transmits compressed transformer key-value caches to reduce redundant processing and bandwidth in multi-agent LLM systems.
• It employs adaptive layer selection and mixed-precision quantization to achieve 5–6× compression while preserving semantic fidelity.
• The protocol integrates explicit fact extraction and cross-model calibration to enable efficient and interoperable communication across heterogeneous architectures.

Q-KVComm is a communication protocol for multi-agent LLM systems in which agents exchange compressed key-value cache representations rather than raw text. The protocol is motivated by the observation that an upstream agent has already computed a transformer state that encodes the semantics of its context, whereas text-based handoff is verbose, lossy with respect to model-internal structure, and forces the downstream agent to reconstruct similar representations from scratch. Q-KVComm therefore treats inter-agent communication as representation-based communication: the sender computes KV caches, selects layers, extracts salient facts, quantizes the selected caches with mixed precision, calibrates them when sender and receiver differ, and transmits a compact payload that the receiver dequantizes and concatenates with its own cache during attention (Kriuk et al., 27 Nov 2025).

1. Problem setting and conceptual lineage

Q-KVComm addresses a bottleneck in collaborative LLM systems: redundant transmission of contextual information between agents consumes bandwidth and compute, especially when a sender has already processed long documents, conversations, or evidence sets into transformer KV caches (Kriuk et al., 27 Nov 2025). The protocol explicitly contrasts this with the standard “send text, re-encode everything” workflow, which discards internal semantic representations and forces the receiver to repeat upstream work.

This design sits in a broader line of KV-based inter-LLM communication. Earlier KVComm formalized communication between a sender model $\mathcal{M}_s$ that sees a context $C$ and a receiver model $\mathcal{M}_r$ that answers a query $Q$, with the goal of conveying the necessary information while minimizing transmitted data. That work argued that natural-language communication incurs high inference cost and information loss during sampling, while hidden-state communication suffers from information concentration bias and awkward efficiency-quality trade-offs. KVComm instead proposed selective sharing of layer KV pairs and reported that it can achieve comparable performance to the Skyline upper bound when selecting $70\%$ of layers’ KV pairs for communication, while transmitting as few as $30\%$ of layers’ KV pairs (Shi et al., 2 Oct 2025).

Q-KVComm retains the core premise that KV caches are a more suitable communication substrate than generated text, but extends it with three additional mechanisms: adaptive layer-wise quantization based on sensitivity profiling, hybrid information extraction to preserve critical facts across content domains, and heterogeneous model calibration for cross-architecture communication (Kriuk et al., 27 Nov 2025). This suggests a shift from selective sharing alone to a more general compressed and interoperable semantic-state transfer protocol.

2. Layer selection and adaptive mixed-precision quantization

The first stage of Q-KVComm is layer selection. Rather than transmitting all transformer layers uniformly, the protocol ranks layers using a hybrid score that combines attention-based salience with a Gaussian positional prior favoring middle layers, reflecting the paper’s statement that mid-layer representations often carry rich semantic content (Kriuk et al., 27 Nov 2025).

The attention importance for layer $l$ is defined as

$$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$

where $H$ is the number of heads, $T$ is sequence length, and $C$0 is the attention weight at layer $C$1, head $C$2, position $C$3. The positional prior is

$C$4

with $C$5 and $C$6, where $C$7 is the total number of layers. These are combined as

$C$8

Layers are ranked by $C$9, and the top proportion is selected for transmission; the reported experiments use a layer selection ratio of $\mathcal{M}_r$0 (Kriuk et al., 27 Nov 2025).

After selection, Q-KVComm performs sensitivity profiling to allocate bit-widths non-uniformly across layers. The reconstruction-error sensitivity metric is

$\mathcal{M}_r$1

Higher $\mathcal{M}_r$2 indicates greater sensitivity to quantization error. The practical mixed-precision policy reported in the paper assigns 8-bit quantization to the top $\mathcal{M}_r$3 most sensitive layers, 6-bit to the middle $\mathcal{M}_r$4, and 4-bit to the bottom $\mathcal{M}_r$5 least sensitive layers (Kriuk et al., 27 Nov 2025).

The protocol therefore does not treat the KV cache as a homogeneous tensor. Its compression policy is layer-adaptive, based on an explicit estimate of how much semantic fidelity is likely to degrade under quantization. Within the paper’s framing, this is the central mechanism that permits compression ratios around $\mathcal{M}_r$6–$\mathcal{M}_r$7 without uniformly degrading all layers to the same precision budget (Kriuk et al., 27 Nov 2025).

3. Quantization, bit packing, and payload formation

For each selected tensor $\mathcal{M}_r$8, Q-KVComm uses asymmetric quantization. Given bit-width $\mathcal{M}_r$9, the scale and zero-point are

$Q$0

$Q$1

Quantization maps values to integers via

$Q$2

and dequantization reconstructs

$Q$3

The paper emphasizes that this mapping places the minimum and maximum tensor values exactly at the quantization endpoints, reducing clipping error (Kriuk et al., 27 Nov 2025).

The quantized integers are then bit-packed rather than stored as full-width integers. The paper gives concrete examples: in 4-bit mode, two values are packed per byte; in 6-bit mode, four values are packed into three bytes. Compression is defined operationally as

$Q$4

The reported ratios are $Q$5 for 4-bit, $Q$6 for 6-bit, and $Q$7 for 8-bit quantization (Kriuk et al., 27 Nov 2025).

The reported results expose a nontrivial implementation trade-off. The 4-bit mode yields the greatest compression, but 8-bit is described as the best speed-quality compromise among the tested settings. The paper also explicitly recommends avoiding 6-bit in practice because it is slower than 4-bit while not providing enough quality gain to justify the trade-off. It further reports that 8-bit is about $Q$8 faster than 6-bit, and attributes the counterintuitive slowdown of 6-bit partly to quantization and dequantization overhead (Kriuk et al., 27 Nov 2025).

This behavior indicates that, in Q-KVComm, communication efficiency is not determined solely by nominal bit-width. Payload structure, packing overhead, and decode behavior materially affect end-to-end runtime, so the optimal operating point is not necessarily the most compressed representation.

4. Hybrid information extraction and heterogeneous calibration

Q-KVComm supplements compressed KV transfer with an explicit information-extraction channel. The rationale given in the paper is that implicit semantic state in KV caches and explicit symbolic facts are complementary: KV caches preserve processing state, while extracted facts provide a compact textual anchor for critical content that might be harder to preserve under aggressive compression (Kriuk et al., 27 Nov 2025).

The extraction module is designed to preserve “critical facts across content domains,” including keywords and key phrases, named entities, technical parameters, API endpoints, rate limits, version numbers, numeric values, and multi-word technical concepts. Each extracted item is represented as a tuple containing type, content, confidence score, and metadata (Kriuk et al., 27 Nov 2025).

For general text, Q-KVComm uses YAKE keyword extraction with score

$Q$9

For structured content such as API documentation or technical specifications, the system uses domain-specific pattern matching to preserve exact operational details such as endpoints and numeric configuration values. When named entities dominate, it applies SpaCy NER, and noun chunk extraction is used to capture technical phrases (Kriuk et al., 27 Nov 2025). The extracted facts are prepended as a short summary to the receiver’s prompt.

A separate and distinctive component is heterogeneous model calibration. Direct KV sharing across different model architectures is ordinarily problematic because hidden states may occupy different representation spaces and may also differ in dimensionality. Q-KVComm addresses this with zero-shot calibration based on distributional alignment. For sender layer $70\%$0, it computes

$70\%$1

$70\%$2

and analogously receiver statistics $70\%$3 and $70\%$4. The calibration transform is

$70\%$5

The paper describes this as a feature-wise affine normalization that preserves relative relationships within the feature space while aligning global statistics (Kriuk et al., 27 Nov 2025).

When sender and receiver dimensions do not match, the paper states that calibration uses scalar statistics averaged across dimensions, yielding a dimension-agnostic transformation that can operate across arbitrary architectures. Calibration is performed once during initialization and then reused for later transmissions (Kriuk et al., 27 Nov 2025).

5. Receiver-side integration and relation to adjacent KV research

The full Q-KVComm pipeline consists of nine stages: the sender processes input and forms KV caches; layer selection chooses important layers using $70\%$6; hybrid extraction identifies salient facts; selected caches are quantized according to adaptive bit allocation; values are bit-packed and transmitted with metadata; the receiver unpacks and dequantizes; cross-model calibration is applied if needed; sender KV caches are concatenated with receiver caches during attention; and the extracted facts are prepended as a compact prompt summary (Kriuk et al., 27 Nov 2025).

The receiver-side concatenation step inherits the communication semantics established by KVComm. In that precursor framework, if sender layer $70\%$7 is selected, receiver integration is written as

$70\%$8

so that the receiver’s attention mechanism can attend jointly over its own context and the sender’s transmitted semantic state. KVComm restricted this to the same model or to fine-tuned versions of the same base LLM, with 1-to-1 layer matching (Shi et al., 2 Oct 2025). Q-KVComm preserves the same concatenative attention interface, but adds a calibration step intended to support heterogeneous-model communication (Kriuk et al., 27 Nov 2025).

Within the broader KV-cache literature, Q-KVComm occupies a distinct position. VQKV is a training-free KV-cache compression method that replaces each high-dimensional key/value vector with residual vector-quantization code indices and reconstructs only the needed cache region during decoding, reporting an $70\%$9 compression ratio on LLaMA3.1-8B while retaining $30\%$0 of baseline LongBench performance (Wang et al., 17 Mar 2026). CommVQ instead uses additive vector quantization and a RoPE-commutative codebook so that decoding can be fused into attention, reporting an $30\%$1 KV-cache reduction at 2-bit and strong 1-bit results (Li et al., 23 Jun 2025). These works target single-model long-context inference, whereas Q-KVComm targets inter-agent semantic-state transmission.

A separate line of analysis studies which parts of the KV cache are most fragile under quantization. Under a fair bit budget, the study of KV, KQV, and QKQV reports that at $30\%$2, KQV wins on every measure—KL divergence, geometric $30\%$3 error, and 6D distance—and argues that applying QJL to $30\%$4 is structurally harmful because softmax amplifies key-side variance via Jensen’s inequality (D'Alberto, 27 Apr 2026). This is relevant to Q-KVComm because the protocol compresses KV caches used as communication payloads; the result suggests that preserving key-side routing semantics may matter as much as raw reconstruction error.

At the systems level, KVServe addresses communication-efficient KV movement in disaggregated LLM serving rather than agent-to-agent reasoning. It treats KV transfer as a service-aware control problem over workload, bandwidth, SLO, and quality constraints, and reports up to $30\%$5 JCT speedup in PD-separated serving and up to $30\%$6 TTFT reduction in KV-disaggregated serving (Liu et al., 13 May 2026). The contrast is instructive: KVServe optimizes runtime transfer of KV states across serving components, whereas Q-KVComm optimizes semantic communication between collaborating agents.

6. Empirical results, trade-offs, and limitations

Q-KVComm is evaluated on two compact instruction-tuned models—TinyLlama-1.1B-Chat-v1.0 and Qwen2.5-1.5B-Instruct—and on three QA datasets chosen to reflect distinct communication regimes: SQuAD for extractive QA over Wikipedia paragraphs, HotpotQA for multi-hop reasoning over multiple documents, and NarrativeQA for long-form narrative understanding (Kriuk et al., 27 Nov 2025). The baselines are Full text, Uncompressed KV, and Uniform quantization.

The paper’s headline result is that Q-KVComm achieves about $30\%$7–$30\%$8 compression while maintaining strong semantic quality, with coherence quality scores above $30\%$9 across all scenarios (Kriuk et al., 27 Nov 2025). At 4-bit, the reported results are: SQuAD relevance $l$0, coherence $l$1, compression $l$2, and $l$3 MB saved; HotpotQA relevance $l$4, coherence $l$5, compression $l$6, and $l$7 MB saved; NarrativeQA relevance $l$8, coherence $l$9, compression $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$0, and $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$1 MB saved. At 6-bit, the reported results are: SQuAD relevance $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$2, coherence $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$3, compression $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$4, and $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$5 MB saved; HotpotQA relevance $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$6, coherence $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$7, compression $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$8, and $$S_l^a = \frac{1}{HT} \sum_{h=1}^{H} \sum_{t=1}^{T} A_{l,h,t},$$9 MB saved; NarrativeQA relevance $H$0, coherence $H$1, compression $H$2, and $H$3 MB saved. At 8-bit, the reported results are: SQuAD relevance $H$4, coherence $H$5, compression $H$6, and $H$7 MB saved; HotpotQA relevance $H$8, coherence $H$9, compression $T$0, and $T$1 MB saved; NarrativeQA relevance $T$2, coherence $T$3, compression $T$4, and $T$5 MB saved (Kriuk et al., 27 Nov 2025).

The reported latency figures reinforce the uneven behavior of different bit-widths. SQuAD inference times are $T$6s, $T$7s, and $T$8s for 4/6/8-bit; HotpotQA times are $T$9s, $C$00s, and $C$01s; NarrativeQA times are $C$02s, $C$03s, and $C$04s (Kriuk et al., 27 Nov 2025). SQuAD is fastest, NarrativeQA is intermediate, and HotpotQA is slowest because of multi-hop and multi-document reasoning. The paper treats HotpotQA’s lower relevance and coherence as evidence that multi-hop reasoning is more fragile under compression, which in turn motivates the combination of selective layer preservation and explicit fact extraction.

Several limitations are either explicit or implied by the reported setup. The experiments are conducted on compact models in the $C$05B–$C$06B range rather than on very large LLMs (Kriuk et al., 27 Nov 2025). The paper does not present a fully separate ablation table, although it does provide sensitivity and trade-off analyses. The quality gains from higher precision are modest but consistent, which supports the adaptive quantization design; however, the operating point is task-dependent, and the 6-bit regime is reported as an unattractive compromise. More broadly, the protocol assumes that meaningful semantic state can be exposed through selected layers and stabilized by hybrid extraction and calibration. The reported results support that assumption for conversational QA, multi-hop reasoning, and long-document understanding, but also show that reasoning-intensive tasks remain the hardest regime (Kriuk et al., 27 Nov 2025).

In aggregate, Q-KVComm reframes KV caches from an internal inference artifact into an explicit communication medium between agents. Its distinctive contribution is not only compressing KV payloads, but doing so in a way that combines semantic layer selection, mixed-precision allocation, explicit fact preservation, and cross-model alignment, thereby making representation-level multi-agent communication practically evaluable rather than purely conceptual (Kriuk et al., 27 Nov 2025).