---
title: Knowledge Grafting in AI
url: https://www.emergentmind.com/topics/knowledge-grafting
type: topic
---

# Knowledge Grafting in AI

Knowledge grafting denotes a family of transfer procedures in which a model, module, or inference process acquires selected capabilities from an external source by inserting a restricted knowledge carrier—such as a wrapped network block, an intermediate feature map, a compressed parameter update, a LoRA adapter block, a latent intervention, or a key–value cache state—rather than retraining the entire system end to end. Across recent arXiv usage, the term covers progressive network replacement for few-shot knowledge distillation, scion/rootstock feature transfer for edge deployment, SkillPack-based capability storage for large language models, compatibility-aware adapter splicing for multimodal large language models, intervention-layer knowledge infusion in iterative generators, and byte-exact cache restoration during inference [2012.04915] [2507.19261] [2505.18502] [2506.23940] [2606.06356] [2607.14431].

## 1. Taxonomic scope of the term

Taken together, the literature suggests that “knowledge grafting” is not a single algorithm but a transfer paradigm defined by what is grafted and where it enters the computational pipeline. The grafted object may be an architectural block, a feature representation, a parameter delta, a low-rank adapter, an intermediate latent edit, or an inference-time state artifact.

| Form of grafting | Grafted object | Representative source |
|---|---|---|
| Progressive block grafting | Student blocks wrapped by $1\times1$ adaptors | [2012.04915] |
| Feature-map grafting | Pooled donor feature maps (“scions”) into a small “rootstock” | [2507.19261] |
| Capability grafting in LLMs | Compressed module-wise parameter updates (“SkillPack”) | [2505.18502] |
| Parameter splicing in MLLMs | Selectively fused LoRA blocks via CAPS | [2506.23940] |
| Layered knowledge infusion | Surface, trajectory, latent, and parametric interventions | [2606.06356] |
| Cache-state grafting | Byte-exact KV-cache state artifacts | [2607.14431] |

This breadth matters because the operational meaning of “knowledge” changes across works. In some settings it is a discriminative function that preserves teacher logits; in others it is a subset of donor features, a task-specific update $\Delta\theta$, a domain-specialized low-rank subspace, or a verified inference trace. A plausible implication is that the unifying property is not representational format but modular transfer under constrained data, memory, compute, or forgetting budgets.

## 2. Architectural and feature-level grafting

An early and technically precise precursor is “Progressive Network Grafting for Few-Shot Knowledge Distillation” [2012.04915]. The teacher is defined as
$f_T(x)=f_L\circ f_{L-1}\circ\cdots\circ f_1(x)$
and the student as
$f_S(x)=h_L\circ h_{L-1}\circ\cdots\circ h_1(x)$.
The few-shot setting assumes an $N$-way $K$-shot unlabeled dataset
$D=\cup_{c=1}^N D_c,\; D_c=\{x_{c,1},\dots,x_{c,K}\}$.
The method uses a dual-stage distillation scheme. In the block-wise grafting stage, a student block $h_l$ is wrapped by two $1\times1$ adaptation convolutions,
$H_l(u)=a_l^{s\to t}\circ h_l\circ a_{l-1}^{t\to s}(u)$,
and inserted into an otherwise fixed teacher:
$T_l^B(x)=f_L\circ\cdots\circ f_{l+1}\circ H_l\circ f_{l-1}\circ\cdots\circ f_1(x)$.
Only the parameters of $H_l$ are updated using
$L_l^B = E_{x\in D}\;\|\tilde T_l^B(x)-\tilde T(x)\|_2^2$,
where $\tilde T(\cdot)=T(\cdot)/\|T(\cdot)\|_2$.
In the progressive grafting stage, trained wrapped blocks are connected one by one,
$T_l^N(x)=f_L\circ\cdots\circ f_{l+1}\circ H_l\circ H_{l-1}\circ\cdots\circ H_1(x)$,
and jointly optimized with
$L_l^N=E_{x\in D}\;\|\tilde T_l^N(x)-\tilde T(x)\|_2^2$.
When $l=L$, the teacher has been fully replaced; the linear $1\times1$ adaptors are then fused into neighboring student convolutions to recover the original student architecture.

The reported results establish the few-shot regime as a central motivation for grafting. With teacher VGG16 and student VGG16-half, the method reaches $92.88\%\pm0.07$ on CIFAR-10 and $68.16\%\pm0.20$ on CIFAR-100 in the 5-shot setting, compared with teacher accuracies of $92.83\%$ and $69.82\%$, and with full-data KD at $92.06\%\pm0.17$ and $68.31\%\pm0.15$. On ILSVRC-2012 with ResNet34 to ResNet18 and 10 shots per class, it attains $68.15\%$ top-1 and $88.83\%$ top-5, while the teacher reaches $73.31\%$ and $91.42\%$, and a fully trained ResNet18 reaches $69.76\%$ and $89.08\%$ [2012.04915]. The paper further reports that single-block grafting converges $\sim3\times$ faster and generalizes better under 1-shot, and that partial grafting of only deep blocks can retain $>68\%$ top-1 on ImageNet.

A more literal use of the term appears in “Knowledge Grafting: A Mechanism for Optimizing AI Model Deployment in Resource-Constrained Environments” [2507.19261]. Here, knowledge grafting is defined as selectively transferring intermediate feature maps learned by a large donor model into a smaller rootstock model. In the reported setup, the donor is VGG16 pretrained on ImageNet without its top classification block; the donor is trained on DeepWeeds with 17,509 images and 8 classes. Candidate scions are donor layers 8, 9, and 10; each feature map $f_i(x)$ is reduced by Global Average Pooling, $g_i(x)=T(f_i(x))$, concatenated as
$h(x)=C(\{g_i(x)\mid i\in S\})$,
and mapped to predictions by
$y(x)=M(h(x))$.
The resulting grafted rootstock has 1,934,665 parameters and size 7.38 MB, compared with 16,880,201 parameters and 64.39 MB for the donor. Validation accuracy rises from $87.47\%$ to $89.97\%$, validation loss falls from $0.5068$ to $0.2976$, and test accuracy reaches $90.45\%$ with test loss $0.2959$ [2507.19261].

The significance of these two lines of work is different but complementary. Progressive network grafting keeps the teacher context fixed while student blocks learn to preserve end-to-end logits under extreme data scarcity. Feature-map grafting, by contrast, treats selected donor representations as reusable “scions” for a lightweight architecture targeted at edge deployment. Both attack the size–performance trade-off, but one is organized around functional imitation and the other around selective representational reuse.

## 3. Parameter-space grafting in language and multimodal models

In large language models, grafting is formulated explicitly as storage and reapplication of task-induced parameter updates. “Knowledge Grafting of Large Language Models” introduces GraftLLM, where cross-capability transfer begins from target parameters $\theta_{\mathrm{tgt}}$, proceeds through SFT and DPO to updated parameters $\theta^\star_{\mathrm{tgt}}$, and extracts
$\Delta\theta=\theta^\star_{\mathrm{tgt}}-\theta_{\mathrm{tgt}}$ [2505.18502].
This update is compressed module-wise into a SkillPack,
$\widehat{\Delta\theta}=\{\widehat{\Delta\theta}_m\}_{m\in\mathcal M}$,
with
$\widehat{\Delta\theta}_m=C_m(\Delta\theta_m;\phi_m)$,
and later grafted back by
$\theta'_{\mathrm{tgt}}=\theta_{\mathrm{tgt}}+\sum_{m\in\mathcal M}D_m(\widehat{\Delta\theta}_m;\phi_m)$.
The compression is module-aware: embeddings and the output head use magnitude pruning with keep-ratio $\alpha_m$; MLP weights use full SVD followed by mixed-precision quantization; attention weights use low-rank SVD of rank $r_m$ followed by group-wise quantization. The paper states that the overall storage cost is $\approx5$–$10\%$ of $\Delta\theta$. Empirically, GraftLLM matches within $1$–$2\%$ of full fine-tuning while adding only $5\%$ of $\Delta\theta$ parameters, achieves a $16.56\%$ win-rate on AlpacaEval2.0 with only $+28\%$ parameter overhead, reaches 7.70 on MT-Bench, and in continual learning attains an overall average of 64.3 with just $10\%$ $\Delta\theta$ storage, compared with 62.2 for Model Tailor [2505.18502].

For multimodal large language models, “Graft: Integrating the Domain Knowledge via Efficient Parameter Synergy for MLLMs” shifts the carrier from full parameter deltas to LoRA blocks [2506.23940]. A frozen pretrained MLLM is augmented by a set of domain experts $\{\Delta W_e\}$, one adapter per domain. Compatibility-Aware Parameter Splicing (CAPS) computes a local functional attribution score
$A_{e,i,j}$,
a global information-theoretic signal
$I_{e,i}=I(\mathbf h_{e,i};y)$,
and combines them through
$$
w_{e,i,j}=
\frac{\exp(\alpha A_{e,i,j}+\beta I_{e,i,j})}
{\sum_{e'}\exp(\alpha A_{e',i,j}+\beta I_{e',i,j})}.
$$
Each fused LoRA block is then
$\Delta W^{\mathrm{graft}}_{i,j}=\sum_e w_{e,i,j}\Delta W_{e,i,j}$.
Before fusion, the method computes activation statistics $\mu_{e,i}$, $s_{e,i}$, and $v_{e,i}$, forms a sensitivity score $\rho_{e,i}$, and aggregates a domain compatibility score
$C_{e\to t}=\frac{1}{M}\sum_{i=1}^M \rho'_{e,i}$.
The reported Spearman correlation is $\rho\approx0.86$ between $C_{e\to t}$ and both solo performance and downstream fused gain. On benchmarks including MathVista, HumanEval, MMMU, and MME, LoRA-based Graft reaches $52.2\%$ on MathVista, $15.9\%$ pass@1 on HumanEval, and 1488.4 on MME; the Math+Code ablation gives 52.0/15.9 for local-only, 51.7/12.2 for global-only, and 52.2/15.9 for dual local+global fusion [2506.23940].

These two approaches share a core commitment: the base model remains structurally intact, while specialized capability is externalized into a modular carrier that can be fused, selected, or rolled back. This is the mechanism behind the “forget-free continual learning” claim in GraftLLM and the “zero inference overhead” claim in Graft after fused adapters are collapsed into the base model.

## 4. Inference-state grafting and byte-exact restoration

A different interpretation appears in “Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel” [2607.14431]. Here the grafted object is not architecture or weights but the transformer’s internal KV cache state
$S=\{K_s^{(\ell)},V_s^{(\ell)}\}_{\ell=1..L}$.
A stored state captured from tokens $x_1,\dots,x_N$ is restored into a fresh process by allocating empty KV buffers, copying the stored keys and values into those buffers, and continuing inference on new tokens $x_{N+1},x_{N+2},\dots$.
Formally, with
$F(\theta;z)\to(\ell,S_z)$
and graft operator
$G(\theta;S_x,y)$,
the comparison is between
$\ell_{\mathrm{fresh}}=F(\theta;x\mathbin{\unicode{x29FA}} y).\mathrm{logits}$
and
$\ell_{\mathrm{graft}}=G(\theta;S_x,y).\mathrm{logits}$.
Byte-exactness is defined by
$\mathrm{SHA256}(\mathrm{bytes}(\ell_{\mathrm{graft}}))=\mathrm{SHA256}(\mathrm{bytes}(\ell_{\mathrm{fresh}}))$,
which implies zero KL divergence and identical argmax.

The exactness claim is highly conditional. Under rotary position embeddings, own-position graft is stated to be the unique numerically exact operating point: the cache must be restored at the same absolute positions, under a deterministic runtime and the same hardware architecture and compiler/runtime versions. Any positional relocation produces a small floating-point discrepancy because
$\cos a\cos b+\sin a\sin b\neq\cos(a-b)$
bit-for-bit after rounding in fp32 arithmetic. The paper reports byte-for-byte identical logits under a pinned deterministic configuration, with zero KL divergence and 100% argmax agreement over fifty samples, and states that byte-identical stores move between machines of the same architecture [2607.14431].

The empirical claims are correspondingly narrow and concrete. On AIME 2025, a frozen Gemma-4-12B moves from 80.0% to 93.3% once a verified solution library is grafted; the eight unsolved problems are stored as one 441 MB cache block, and 6 of 8 are recovered for a total of 28/30. On the recurring case, base best-of-5 sampling uses 401,026 tokens and solves 0/8, whereas grafted recurrence uses 61 tokens total and solves 8/8, a factor of 6,574 fewer decode tokens, approximately 1,700× faster wall time, and 3,000×–8,700× less energy. The same store extends usable context from 32,768 to 2,854,766 tokens at zero extra GPU memory, with 88 separate blocks and per-access cost of approximately 0.29 s plus approximately 35 token warm-up [2607.14431].

This form of grafting departs sharply from conventional fine-tuning or adapter fusion. The knowledge carrier is an inference artifact, not a parameter update, and the model weights remain frozen throughout.

## 5. Layered intervention as a general framework

“Where Should Knowledge Enter? A Layered Framework for Knowledge Infusion in Multimodal Iterative Generative Models” generalizes the problem by treating knowledge infusion as an intervention-layer problem [2606.06356]. The formal setup is an iterative trajectory
$h_0\to h_1\to\cdots\to h_T=x$,
with
$h_0=\mathrm{Init}(p,z)$
and
$h_{t+1}=f_\theta(h_t,c_t)$,
together with an external knowledge source $\mathcal K$ and a consistency predicate $\mathcal C(x;\mathcal K)\in\{0,1\}$.
The framework identifies four intervention layers: surface infusion at the input/output boundary, trajectory infusion that modifies the transition function at inference time, latent infusion that edits the intermediate state, and parametric infusion that alters $\theta$ before inference. The composed operator is written as
$$
x' = r_{\mathcal K}\circ \mathrm{Gen}_{\theta+\Delta\theta_{\mathcal K}}\circ \ell_{\mathcal K}\circ \tilde f_{\theta,\mathcal K}\circ g_{\mathcal K}(p).
$$

The paper instantiates the framework in diffusion models and argues for three design principles: match layers to failure classes, compose for complementary coverage, and manage inter-layer interference by using a single shared $\mathcal K$. In a controlled safety-alignment experiment on the Detonate benchmark with 25 K hateful-prompt variants and 4 samples per prompt, using frozen SDXL and SD-v1.5, three layers were deployed cumulatively. On SDXL, toxicity falls from .31 to .17 after surface input infusion, to .11 after trajectory+latent infusion, and to .09 after surface output infusion; on SD-v1.5, it falls from .28 to .16, then to .10, then to .08. The overall SDXL reduction from .31 to .09 is reported as 70.97% versus vanilla generation. CLIP alignment and aesthetic quality rise monotonically; no p-values are reported, but improvements are described as consistent across two architectures, and SAFREE and SLD are outperformed at each layer [2606.06356].

Within the broader grafting literature, this framework is important because it relocates the concept from a specific implementation trick to a structural question: whether knowledge enters at the boundary, the transition, the latent state, or the parameters. This suggests a unifying vocabulary for otherwise heterogeneous methods.

## 6. Limits, misconceptions, and research directions

Several limitations recur across the literature. In progressive network grafting, highly heterogeneous architectures such as VGG to ResNet can degrade more, and excessively small blocks may under-parameterize or become slow [2012.04915]. In feature-map grafting for edge models, scion layer indices are chosen manually, other donor/rootstock pairs remain untested, no dedicated mechanism is provided for merging or fine-tuning grafted weights beyond concatenation and retraining, and raw accuracy remains below larger state-of-the-art models by $\Delta$ 5–7 pp despite the size gain [2507.19261]. In GraftLLM, decompression adds latency compared to LoRA, capability transfer depends on high-quality SFT+DPO, and compression hyperparameters are hand-tuned [2505.18502]. In CAPS-based MLLM fusion, the method presupposes validation-time attribution and compatibility estimation rather than naive arithmetic averaging [2506.23940]. In KV-cache grafting, byte-exactness is guaranteed only within the same architecture and pinned deterministic configuration, own-position graft cannot be relocated, transfer succeeds only when the cached program is parametric in changed values, and recurrence degrades on multi-example archives such as LiveBench [2607.14431].

These limitations also clarify common misconceptions. Knowledge grafting is not equivalent to full-model merging: some methods keep the base frozen and attach only compressed updates or low-rank blocks, while KV-cache grafting changes no weights at all. It is not inherently exact: byte-exact restoration is a special property of own-position cache graft under deterministic conditions, not a generic feature of grafting. Nor is grafting necessarily a one-time compression method: in GraftLLM it is a carrier for continual learning and model fusion, whereas in the layered framework it is a compositional strategy for intervention across a generation trajectory.

The proposed extensions are correspondingly diverse. The few-shot distillation work proposes combining grafting with data augmentation or synthetic data, searching for blocks with NAS, and extending block grafting to segmentation or detection by grafting feature-pyramid blocks [2012.04915]. The edge-deployment work proposes automated scion selection via evolutionary or genetic algorithms, architecture-aware grafting for ResNets, Transformers, and LLMs, dynamic input-dependent feature routing, advanced weight interpolation such as Slerp or cross-attention based fusion of multiple donors, and real-hardware measurements of latency, power, and memory [2507.19261]. GraftLLM points to more efficient decompression, learned compression pipelines, and integration with MoE layers [2505.18502]. The layered framework emphasizes multi-layer composition under a shared knowledge source [2606.06356]. A plausible implication is that future “knowledge grafting” research will be organized less by model family than by the interface at which reusable knowledge is extracted, stored, and reinserted.

Source: https://www.emergentmind.com/topics/knowledge-grafting