KVComm: Selective KV Sharing Framework

• KVComm is a communication framework that selectively shares key-value pairs between LLM agents to efficiently preserve contextual information.
• It employs a layer-wise selection strategy that combines attention importance with a Gaussian prior to optimize KV transmission and reduce overhead.
• Empirical results show KVComm attains near Skyline performance using only 30%-70% of layers, achieving significant computational savings.

KVComm is a communication framework for multi-agent LLMs in which a sender model transmits selected key-value (KV) pairs, rather than natural-language messages or hidden states, to a receiver model. In the formulation introduced in "KVComm: Enabling Efficient LLM Communication through Selective KV Sharing" (Shi et al., 2 Oct 2025), a sender $\mathcal{M}_s$ processes a context $C$, a receiver $\mathcal{M}_r$ answers a query $Q$, and communication is effected by sharing only a subset of the sender’s layerwise KV caches. The framework is designed to preserve rich contextual information while avoiding the repeated decoding cost and information loss of natural-language communication, and it is reported to achieve comparable performance to the Skyline upper bound while transmitting as few as $30\%$ of layers’ KV pairs (Shi et al., 2 Oct 2025).

1. Conceptual basis and motivation

KVComm is motivated by a specific tension in multi-agent LLM systems: agents must convey relevant information $I_C$, but common communication channels are either expensive or lossy. The paper positions natural-language debate and summarization as the dominant existing mechanism, but one that requires multiple decoding rounds and may lose information through token generation. It also contrasts KV sharing with activation-based approaches such as CIPHER and AC, arguing that embedding-space communication is indirect and that hidden-state communication suffers from “information concentration bias,” where the last token’s hidden state dominates later-layer computation (Shi et al., 2 Oct 2025).

The framework’s central claim is that KV pairs constitute a more suitable communication medium than a single hidden state. In decoder-only LLMs, keys and values are the operative memory representation used by attention. KVComm therefore treats them not as an implementation detail of single-model inference, but as a transferable representation between agents. Because the receiver does not have its hidden state overwritten, but instead receives additional memory entries in attention, it can “attend to both its own context and the information provided by $\mathcal{M}_s$.” This design is presented as richer than hidden-state transfer and more efficient than text generation (Shi et al., 2 Oct 2025).

A common misunderstanding is that cheaper internal activations should necessarily dominate text-based communication. KVComm rejects that conclusion in its specific setting: the paper’s argument is not that all internal states are equally useful, but that KV pairs retain contextual structure without inducing the overwrite problem associated with hidden-state replacement or averaging.

2. Communication model and runtime mechanism

The framework assumes a sender-receiver configuration. The sender $\mathcal{M}_s$ performs one prefill pass over the context $C$, producing layerwise KV pairs $\{(\mathbf{k}_s^l, \mathbf{v}_s^l)\}$ for $C$0. Rather than transmitting all layers, KVComm selects a subset $C$1 and sends only those KV pairs to the receiver. When the receiver processes $C$2, the sender cache is inserted by concatenation at the selected layers: $C$3 The sender and receiver are either the same model or fine-tuned versions of the same base LLM, so layer indices align one-to-one (Shi et al., 2 Oct 2025).

Operationally, the pipeline is lightweight. The sender prefills on $C$4, informative layers are selected, only those KV pairs are transmitted, the receiver uses them during prefill and decoding on $C$5, and then generates the final answer. The method therefore avoids an additional natural-language generation stage and does not require the receiver to recompute the sender’s semantic representation from text alone (Shi et al., 2 Oct 2025).

This mechanism also clarifies why KVComm is selective rather than exhaustive. Sending every layer would maximize transferred internal state but would reduce the efficiency advantage. Sending only selected layers imposes an information bottleneck at the level of layerwise attention memory rather than at the level of generated tokens.

3. Layer selection and scoring rule

The distinctive component of KVComm is its layer-wise selection strategy. The ideal objective is stated as choosing a subset $C$6 of size $C$7 that maximizes the informativeness of the receiver’s output: $C$8 where $C$9 is the task metric. Because direct optimization is intractable, the paper introduces two hypotheses: H1, that intermediate layers contain the most transferable semantic knowledge, and H2, that layers with stronger attention distributions are more effective for communication (Shi et al., 2 Oct 2025).

To operationalize these hypotheses, KVComm defines an attention importance score

$\mathcal{M}_r$0

normalizes it to $\mathcal{M}_r$1, and combines it with a Gaussian prior over depth,

$\mathcal{M}_r$2

The final layer score is

$\mathcal{M}_r$3

and the top $\mathcal{M}_r$4 layers by $\mathcal{M}_r$5 are selected (Shi et al., 2 Oct 2025).

The Gaussian prior centers the selection around intermediate layers, consistent with the paper’s interpretation that early layers emphasize surface form and late layers are more specialized for output prediction. The attention score then refines this prior by favoring layers whose KV memories are heavily used for context attention. The method is therefore not simply a middle-layer heuristic; it is a hybrid rule that biases toward intermediate depth while allowing non-contiguous selection of high-importance layers. In the reported experiments, $\mathcal{M}_r$6, $\mathcal{M}_r$7, and $\mathcal{M}_r$8 is tuned per model family, with $\mathcal{M}_r$9 for Llama and $Q$0 for Qwen and Falcon. The paper further states that the selection can be calibrated with only one example and still generalize well to the test set (Shi et al., 2 Oct 2025).

4. Evaluation protocol and empirical findings

The evaluation spans eight model pairs: four same-model pairs and four pairs of fine-tuned versions of the same base LLM. The model families are LLaMA, Qwen, and Falcon. The eight pairs are Llama-3.1-8B-Instruct with itself, Llama-3.2-3B-Instruct with itself, Qwen2.5-7B-Instruct with itself, Falcon3-7B-Instruct with itself, EvolCodeLlama-3.1-8B-Instruct $Q$1 ToolACE-2-Llama-3.1-8B, Llama-3.2-3B-Instruct-abliterated $Q$2 DeepSeek-R1-Distill-Llama-3B, Qwen2.5-7B-Instruct-Uncensored $Q$3 Bespoke-Stratos-7B, and falcon3-ultraset $Q$4 Falcon3-7B-Instruct-abliterated. Tasks include Countries, Tipsheets, HotpotQA, QASPER, MuSiQue, MultiFieldQA-en, 2WikiMQA, and TMATH. F1 is used for most tasks, while ROUGE-L Recall is used for TMATH. Baselines are Baseline, Skyline, Natural Language Debate, CIPHER, and AC with mean, replace, and sum variants (Shi et al., 2 Oct 2025).

Across datasets and model pairs, KVComm is reported to consistently outperform the communication baselines and approach the Skyline upper bound. With $Q$5 of layers’ KV pairs transmitted, it can match Skyline-like performance on many tasks; with only $Q$6 of layers selected, it often beats most baselines. The paper explicitly states that communicating as few as $Q$7 of layers’ KV pairs yields up to a $Q$8 reduction in communication relative to transmitting all layers’ KV pairs, and reports a $Q$9 to $30\%$0 reduction in computation compared with Skyline (Shi et al., 2 Oct 2025).

The evaluation also reports cases in which KVComm exceeds Skyline. The authors interpret this as evidence that selective KV sharing can help the receiver focus on the most relevant information and can complement the receiver’s own strengths. Gains are smaller on TMATH, where the paper argues that mathematical reasoning may already be strongly supported by pretraining; AC also performs relatively well there because the hints are easy to exploit even when hidden-state communication is imperfect (Shi et al., 2 Oct 2025).

These results situate KVComm as a middle point between two extremes: full input merging, represented by Skyline, and minimal or lossy communication through text or single-state activations. Its empirical contribution is not merely accuracy at equal transmission cost, but the claim that selective KV transfer approximates upper-bound performance without requiring the receiver to solve the original concatenated-input problem directly.

5. Analyses, misconceptions, and theoretical framing

A major analytical theme in the paper is the inadequacy of hidden states as a default communication channel. KVComm studies hidden states directly and argues that the last token becomes most critical in later layers, demonstrating information concentration bias. Under this view, preserving the most important hidden-state signal implies preserving the last-token state; however, AC-style methods that replace or average the receiver’s last-token hidden state inevitably corrupt the receiver’s own information. The paper also reports that prepending all tokens’ hidden states works only under restrictive conditions: early-layer sender states must be combined with early-layer receiver states, the efficiency benefit is small if the sender states come from early layers, and performance drops if those states are injected into later receiver layers (Shi et al., 2 Oct 2025).

The layer-selection analyses are intended to validate H1 and H2. KVComm is compared against a contiguous chunk of layers, analogous to a Droidspeak-like strategy, and the contiguous approach is described as working only in a narrow hyperparameter region. Non-contiguous selection is more robust and often better for the same number of layers. Random selection is also consistently worse, especially at low transmission ratios such as $30\%$1, where careful selection matters most. An attention-distribution analysis further supports H2 by showing that higher attention-importance layers produce better performance, whereas low-score layers degrade performance (Shi et al., 2 Oct 2025).

On efficiency, the paper derives

$30\%$2

with the interpretation that savings grow as fewer layers $30\%$3 are selected. FLOPs measurements on Tipsheets and MultiFieldQA-en are reported to confirm the theoretical claim. This formulation also clarifies the framework’s systems-level objective: KVComm avoids the repeated decoding overhead of debate-based methods and reduces the recomputation inherent in Skyline because the receiver does not attend over the full concatenated context at every layer (Shi et al., 2 Oct 2025).

6. Scope, limitations, and related usages of the name

KVComm, as introduced in 2025, is restricted to sender-receiver pairs that share architecture or derive from the same base LLM, because it relies on one-to-one layer alignment. The selection strategy is calibrated on a small set and uses fixed hyperparameters $30\%$4, $30\%$5, and $30\%$6; the paper identifies dynamic or per-input selection as future work. It also studies only two-agent communication, even though the motivating setting is broader multi-agent systems. Finally, the method is described as orthogonal to standard inference accelerators and KV-cache compression methods, suggesting future composition with those techniques (Shi et al., 2 Oct 2025).

The term itself later became ambiguous. "KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems" uses a similar name for a distinct training-free framework that reuses and aligns KV caches across agents under diverse prefix contexts, with an anchor pool that estimates context-induced KV offsets and enables efficient prefilling rather than selective layer sharing (Ye et al., 14 Oct 2025). "Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression" extends the communication theme in another direction, emphasizing direct transmission of compressed KV caches, adaptive layer-wise quantization, hybrid information extraction, and heterogeneous model calibration (Kriuk et al., 27 Nov 2025).

This suggests that “KVComm” has come to denote a broader class of representation-based inter-LLM communication methods. In the stricter historical sense, however, KVComm refers to the selective KV-sharing framework of 2025: a method in which a sender transmits only the most informative layers’ KV pairs so that a receiver can incorporate them through attention and answer a downstream query with substantially less communication than full-context transfer (Shi et al., 2 Oct 2025).