Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

Abstract: Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source's KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B->32B, one source layer explains 56% of variance in the target's keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 73-98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to +37 pp HellaSwag retention on the failures. The mapper runs 2.7-25x faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.

• The paper introduces a per-head, closed-form ridge mapper that combines selected source layers, removes and reapplies RoPE, and transfers KV caches without target-side prefill recomputation.
• Across six matched-KV model pairs, four retain 73%–98% of standalone accuracy, while Qwen3 14B→32B achieves 97.6% retention and up to 25× lower prefill latency at 32K tokens.
• The results show that attention-output similarity predicts transfer quality better than global reconstruction error, and nonlinear MLP mappers recover up to 36.8 percentage points on difficult Ministral pairs.

Cross-Model KV Cache Transfer in LLM Families

Problem formulation and motivation

“Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse” (2608.03893) studies a systems problem created by multi-model LLM serving. Cost-quality cascades, routing policies, and mid-conversation model switching frequently move an active session between differently sized models in the same family. Although prefix caching avoids repeated prefill for requests served by one model, it does not solve the inter-model case: the receiving model normally recomputes the entire prefix to construct its own key-value cache.

The paper formulates model switching as a cache-representation transformation problem. Given a source model $\mathcal{S}$ and target model $\mathcal{T}$, the objective is to construct an approximate target cache $\hat{\mathcal{C}}_{\mathcal{T}}$ from the source cache $\mathcal{C}_{\mathcal{S}}$, such that decoding from the transformed cache preserves the target model’s downstream behavior. This framing is important because it evaluates transfer by task accuracy and perplexity rather than by cache reconstruction error alone. A cache can have favorable average $R^2$ while introducing errors in the particular subspaces that determine attention scores and attention outputs.

The method is restricted experimentally to within-family pairs with matched KV head counts and matched per-head dimensions. The models may nevertheless differ substantially in parameter count, hidden dimension, and depth. The serving pipeline consists of precomputing the source cache, applying a learned-but-closed-form transformation to each source layer’s keys and values, and passing the resulting target-format cache directly to the target decoder.

Figure 1: A per-head linear map converts the source model’s prefetched KV cache into the target model’s expected representation, eliminating target-side re-prefill.

Empirical linear structure of cross-model KV representations

The central empirical claim is that matched-KV models exhibit substantial linear structure in their cache representations. The authors perform token-level regressions between individual source and target layers, separately for keys, RoPE-stripped keys, and values. On Qwen3 14B $\rightarrow$ 32B, a single source layer explains approximately $56\%$ of the target-key variance and $32\%$ of the target-value variance under the production-oriented analysis. Combining multiple source layers raises these figures to approximately $79\%$ and $65\%$, respectively.

The layerwise structure is nonuniform. Source layers near particular target layers are considerably more predictive than others, producing sharp or diffuse alignment patterns depending on the architectural and depth gap. The best individual layer pair reaches a RoPE-stripped-key $\mathcal{T}$0 of $\mathcal{T}$1 for Qwen3 14B $\mathcal{T}$2 32B, compared with $\mathcal{T}$3 for the more distant Qwen3 8B $\mathcal{T}$4 32B pair. This supports the paper’s claim that model-family membership is insufficient by itself: architectural proximity and training compatibility affect the quality of cache transfer.

The analysis also identifies two structural asymmetries. First, keys are generally more linearly predictable than values, with an approximate $\mathcal{T}$5 gap in head-averaged $\mathcal{T}$6. Second, RoPE obscures the content-level correspondence. Applying the inverse source rotation before regression and the target rotation after mapping separates position-dependent geometry from the underlying content representation. This makes the fitted map independent of the calibration sequence positions and, in principle, reusable across context lengths and RoPE configurations.

Figure 2: Layerwise regression reveals strong but nonuniform cross-model correspondence, with RoPE-stripped keys generally exhibiting sharper and higher-$\mathcal{T}$7 alignments than rotated keys or values.

The number of source layers is also consequential. On Qwen3 14B $\mathcal{T}$8 32B, a single source layer captures only $\mathcal{T}$9 of the all-layer $\hat{\mathcal{C}}_{\mathcal{T}}$0 for stripped keys and $\hat{\mathcal{C}}_{\mathcal{T}}$1 for values. The largest improvement occurs between one and four source layers, while performance approaches its all-layer value around six selected layers. Thus, the cache relationship is not adequately modeled as a one-to-one layer correspondence; complementary information is distributed across source depth.

Closed-form per-head ridge mapper

The proposed mapper is constructed independently for every target layer, target KV head, and cache type. For each target layer, the method selects the top-$\hat{\mathcal{C}}_{\mathcal{T}}$2 source layers according to head-averaged predictive $\hat{\mathcal{C}}_{\mathcal{T}}$3. The KV features from those layers are concatenated across source heads, and separate ridge regressions map the resulting feature vectors to the target key and value dimensions.

The transformation is therefore a collection of ordinary matrix multiplications at inference time. Given centered design matrix $\hat{\mathcal{C}}_{\mathcal{T}}$4 and target matrix $\hat{\mathcal{C}}_{\mathcal{T}}$5, the slope is obtained through the closed-form ridge solution

$\hat{\mathcal{C}}_{\mathcal{T}}$6

with $\hat{\mathcal{C}}_{\mathcal{T}}$7 in the production configuration. Biases are recovered from the feature and response means. The procedure requires no backpropagation, pair-specific end-to-end fine-tuning, or decoder-level optimization.

Figure 3: Each target layer and head receives a separate key and value projection from the top-$\hat{\mathcal{C}}_{\mathcal{T}}$8 predictive source layers selected by cross-layer regression.

Calibration uses 500 FineWeb-Edu sequences of 1,024 tokens, with stride-4 token subsampling producing approximately 128,000 observations per target head. The resulting mapper requires between 1.01 and 3.36 billion parameters across the evaluated pairs, corresponding to approximately 4--12 GB of storage. Fitting takes approximately 47--87 minutes on a single node equipped with eight H100 GPUs. These costs are substantial for a pairwise deployment artifact, but they are paid offline and can be amortized across many inference requests.

The factorization of RoPE is one of the method’s more technically important design choices. Source keys are inverse-rotated into content space, mapped by the regression, and then re-rotated using the target model’s positional encoding. Ablations indicate that fitting and evaluating entirely in RoPE-coupled space performs similarly at the 1,024-token calibration length, but that omission of inference-time target RoPE handling creates severe task-specific failures. The content-space formulation is consequently justified primarily by positional modularity and expected long-context generalization rather than by an immediate short-context accuracy advantage.

Main transfer results

The main evaluation covers six small-to-large pairs from Qwen3, Llama 3.1, and Ministral 3. The paper reports a pronounced two-tier outcome.

Four pairs retain between $\hat{\mathcal{C}}_{\mathcal{T}}$9 and $\mathcal{C}_{\mathcal{S}}$0 of target standalone accuracy averaged over ARC-Challenge, HellaSwag, WinoGrande, MMLU, and GSM8K:

• Qwen3 14B $\mathcal{C}_{\mathcal{S}}$1 32B: $\mathcal{C}_{\mathcal{S}}$2 average retention.
• Qwen3 8B $\mathcal{C}_{\mathcal{S}}$3 32B: $\mathcal{C}_{\mathcal{S}}$4.
• Llama 3.1 8B $\mathcal{C}_{\mathcal{S}}$5 70B: $\mathcal{C}_{\mathcal{S}}$6.
• Ministral 3B $\mathcal{C}_{\mathcal{S}}$7 8B: $\mathcal{C}_{\mathcal{S}}$8.

The strongest result is Qwen3 14B $\mathcal{C}_{\mathcal{S}}$9 32B, where transfer slightly exceeds standalone performance on ARC-Challenge in the reported evaluation, while retaining $R^2$0 of GSM8K accuracy. The Llama 3.1 8B $R^2$1 70B result is particularly notable because the parameter ratio is approximately $R^2$2, yet HellaSwag retention reaches $R^2$3. These results support the paper’s claim that cache transfer can remain effective across large scale differences when the models share compatible internal representations.

The contradictory result is within Ministral 3. The 3B $R^2$4 8B pair retains $R^2$5 on average, but the 3B $R^2$6 14B and 8B $R^2$7 14B pairs collapse to $R^2$8 and $R^2$9, respectively. Floor-normalized retention falls further, to $\rightarrow$0 and $\rightarrow$1. The failures are not attributable simply to unmatched KV geometry, because all evaluated pairs satisfy the matched-head and matched-head-dimension condition. Matched KV configuration is therefore a useful prerequisite but not a sufficient predictor of transferability.

Figure 4: Transfer quality varies with the number of selected source layers, with closer model pairs saturating earlier and more difficult pairs requiring larger cross-layer feature sets.

The source-layer sweep reinforces the importance of cross-layer aggregation. Setting $\rightarrow$2 is consistently inadequate, while the optimal $\rightarrow$3 varies by pair. Qwen3 14B $\rightarrow$4 32B reaches near-peak HellaSwag performance around $\rightarrow$5, whereas the Llama 3.1 8B $\rightarrow$6 70B pair continues improving at larger values, reaching its selected configuration at $\rightarrow$7. This dependence suggests that $\rightarrow$8 is not merely a computational parameter; it reflects the degree to which a target layer’s representation is distributed across source depth.

Ablation and the role of attention-sensitive error

Component ablations on Qwen3 14B $\rightarrow$9 32B identify cross-layer source selection as the largest contributor to performance. Reducing the mapper from $56\%$0 to $56\%$1 reduces stripped-key $56\%$2 from approximately $56\%$3 to $56\%$4 and causes major downstream degradation. Removing ridge regularization is less damaging than removing cross-layer selection, although the unregularized high-dimensional systems are more vulnerable to conditioning problems.

Figure 5: Sequential component removal shows that cross-layer source selection contributes more to transfer quality than either ridge regularization or the RoPE factorization under the tested configuration.

The paper’s most consequential theoretical observation is that average reconstruction quality does not determine downstream retention. Across 12 directional pair evaluations, calibration-domain key $56\%$5 correlates weakly and negatively with HellaSwag retention, at Pearson $56\%$6. By contrast, cosine similarity between attention outputs computed from mapped and ground-truth KV caches correlates positively with HellaSwag retention at $56\%$7.

This distinction follows from the structure of attention. Key errors matter primarily when they alter query-key compatibility, and value errors matter primarily when they occur at positions receiving substantial attention mass. An isotropic reconstruction metric weights all feature directions equally, whereas attention selectively amplifies particular query-aligned key directions and attended value positions. The paper operationalizes this distinction through error-concentration measures in query-sensitive singular-vector subspaces for keys and attention-weighted positions for values.

The result changes how mapper quality should be diagnosed. A low global $56\%$8 need not imply poor generation if the residual lies in attention-irrelevant directions. Conversely, a mapper with favorable average reconstruction statistics can fail if its errors concentrate in attention-critical subspaces. This explains why the two Ministral failure pairs degrade sharply despite satisfying the same basic KV compatibility conditions as the successful pairs.

Nonlinear extension

To test whether the failures reflect a limitation of linearity rather than an absence of transferable information, the authors replace ridge regression with per-head two-hidden-layer MLPs. The MLPs use 1,024-unit ReLU layers, Adam optimization, and the same MSE reconstruction objective and calibration data.

The nonlinear model does not uniformly dominate the linear mapper. On Qwen3 14B $56\%$9 32B, HellaSwag retention changes from $32\%$0 to $32\%$1; on Ministral 3B $32\%$2 8B, it falls from $32\%$3 to $32\%$4. However, on the two ridge failure pairs, the gains are substantial:

• Ministral 3B $32\%$5 14B: $32\%$6 to $32\%$7 HellaSwag retention, a gain of $32\%$8 percentage points.
• Ministral 8B $32\%$9 14B: $79\%$0 to $79\%$1, a gain of $79\%$2 percentage points.

  Figure 6: Accuracy retention demonstrates a two-tier pattern for ridge transfer, with nonlinear mapping recovering much of the loss on the difficult Ministral pairs.

On failure pairs, the ridge mapper exhibits deeply negative evaluation-domain key $79\%$3, indicating poor extrapolation from the FineWeb-Edu calibration distribution. The MLP moves this quantity substantially toward zero, lowers key-error concentration by approximately $79\%$4 on average, and increases attention-output cosine by approximately $79\%$5. The paper interprets this as error redistribution rather than simply lower Euclidean error: the MLP moves residual error away from directions that control attention.

This result is technically significant but should be interpreted cautiously. The MLP is trained with an MSE objective, not directly with an attention-output or downstream loss. Its improvement therefore does not establish that generic nonlinear regression is optimal. It does, however, show that the linear failures are at least partly recoverable and motivate attention-aligned objectives, mixture-of-experts mappers, and task- or domain-conditioned transformations.

Multi-turn stability

The paper evaluates repeated switching on 100 CoQA conversations of approximately 15 turns each, alternating between Qwen3 14B and 32B. The mapped cache remains stable over ten turns. In the small-to-large direction, the gap relative to the target standalone baseline widens by only $79\%$6 percentage points from turn one to turn ten. In the large-to-small direction, drift increases approximately linearly at $79\%$7 percentage points per turn.

Figure 7: Repeated Qwen3 14B $79\%$8 32B handoffs exhibit limited accuracy drift over ten CoQA turns, although the large-to-small direction accumulates error more steadily.

These measurements support the feasibility of mid-conversation handoff but do not establish indefinite stability. Linear drift in the large-to-small direction could become material over substantially longer agentic sessions. Moreover, the experiment uses one model family, one task, and a fixed handoff protocol. Recursive application of approximate caches, heterogeneous routing schedules, and context distributions outside the calibration domain remain open concerns.

Latency and deployment implications

The strongest practical result concerns prefill latency. On eight H100 GPUs, the mapper is substantially faster than target re-prefill across sequence lengths from 64 to 32,768 tokens. For Qwen3 14B $79\%$9 32B, transfer takes 14.0 ms at 64 tokens and 277.6 ms at 32K, compared with 61.7 ms and 6,975.3 ms for re-prefill. The corresponding speedups are approximately $65\%$0 and $65\%$1. For the reverse 32B $65\%$2 14B direction, the speedup ranges from approximately $65\%$3 to $65\%$4.

Figure 8: Mapper application remains comparatively inexpensive as sequence length grows, while target re-prefill scales more steeply at every evaluated length.

The asymmetry between directions is operationally relevant. Small-to-large transfer is useful for quality escalation, while large-to-small transfer reduces serving cost after a high-capability model has processed a difficult turn. The mapper’s execution is dominated by per-layer batched matrix multiplications and does not require running the target transformer body over the prefix. At short sequences, fixed dispatch and cache-transfer overheads dominate; at long sequences, the mapper’s linear growth is much smaller than the target’s re-prefill cost.

There are deployment tradeoffs. A directional mapper is required for each ordered model pair, so a fleet of $65\%$5 models can require up to $65\%$6 artifacts. The reported mapper sizes of 4--12 GB are manageable in host memory or storage but create nontrivial artifact-management and loading requirements. The study also does not measure complete end-to-end routing overhead, cache movement across processes, network transfer, or concurrency effects. Its latency results therefore establish a compute advantage rather than a complete serving-cost model.

Limitations and future research

The empirical scope is deliberately constrained. All six primary pairs are within-family and matched-KV; mismatched head counts and per-head dimensions are not tested. The models use dense full attention, excluding sliding-window, hybrid attention, and attention-recurrent architectures that carry additional state. Cross-family transfer, such as Qwen3 $65\%$7 Llama 3.1, is also unexamined.

Calibration uses only FineWeb-Edu. The domain sweep indicates that Wikipedia produces relatively small average degradation, whereas CodeAlpaca can reduce HellaSwag by $65\%$8 percentage points and produces more variable benchmark effects. These findings suggest that calibration distribution and register matter, but they do not characterize specialized domains such as medicine, law, code generation, or long-horizon tool use.

The selection of $65\%$9 is partly optimized on benchmarks also used for reporting, although leave-one-benchmark-out analyses limit the measured effect to at most $\mathcal{T}$00 percentage points. More important is the absence of a reliable pre-fit transferability predictor. Attention-output cosine is a useful post-fit diagnostic, but it cannot screen a pair before calibration and mapper construction. Future work should identify architecture- and training-derived indicators of representational compatibility, possibly using activation statistics, inter-model CKA, tokenizer-conditioned probes, or low-cost attention-output estimates.

The nonlinear results point toward a broader design space. Mappers could be trained directly against attention-output fidelity, target logit preservation, or downstream task loss rather than per-channel MSE. Conditional or low-rank adapters could reduce the 1--3 billion parameter footprint. Cross-layer sparsity, shared bases across heads, and low-rank factorization could make pairwise artifacts cheaper to store. For large model fleets, a shared latent cache space might replace the quadratic collection of directional pairwise maps, although such a design would need to preserve target-specific attention geometry.

Conclusion

The paper demonstrates that cross-model KV cache transfer is feasible within several LLM families without gradient-based mapper training. Its closed-form per-head ridge mapper exploits cross-layer linear structure, strips and reapplies RoPE to isolate content representations, and uses a small calibration corpus to construct target-format caches. Across six matched-KV pairs, four retain $\mathcal{T}$01--$\mathcal{T}$02 of standalone accuracy and achieve $\mathcal{T}$03--$\mathcal{T}$04 lower prefill latency than re-prefill.

The results also establish clear boundaries. Matched KV geometry does not guarantee successful transfer, and average cache reconstruction metrics are insufficient to predict downstream behavior. The location of residual error relative to attention-sensitive subspaces is more informative, with attention-output cosine correlating with HellaSwag retention at $\mathcal{T}$05. Nonlinear mappers recover up to $\mathcal{T}$06 percentage points on difficult pairs, indicating that future systems should optimize attention-relevant fidelity rather than generic reconstruction error. The practical trajectory is therefore toward transferability diagnostics, attention-aligned mapping objectives, compact shared adapters, and evaluation across cross-family, long-context, heterogeneous, and recurrent serving architectures.