KG2KV: Converting KGs to KV Memory

• KG2KV is a pipeline that converts knowledge graphs of textual triples into attention-compatible key-value memory, enabling direct parametric knowledge integration into LLMs.
• It transforms each triple into natural language query, key, and value strings encoded as sentence embeddings, allowing efficient attention integration with minimal retraining.
• Utilizing a hierarchical pruning scheme with UMAP and GMM, KG2KV scales sub-linearly in memory and time while supporting training-free adaptation to newly inserted knowledge.

KG2KV, short for “KG to KV,” denotes the pipeline introduced in AtlasKV for converting a knowledge graph of textual triples into attention-consumable key–value memory inside a LLM. Its purpose is to transform each triple $(h,r,t)$ into query–key–value strings and sentence embeddings so that KG-derived knowledge can be integrated parametrically into the attention layers of an LLM, without external retrievers, long textual context priors, or retraining when adapting to new knowledge (Huang et al., 20 Oct 2025).

1. Definition and representational objective

Formally, the knowledge graph is a set of triples

$$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$

where $h$ and $t$ can be named entities, events, concepts, and related objects. KG2KV converts each triple into natural-language strings that mirror self-attention’s Q/K/V structure. The masked entity is treated as the value, while the unmasked entity together with a nounified relation becomes the key. The operational rule is stated explicitly: “we consider the masked entity as the value data, and the other entity as well as the relation as the key data, which complete KGKV data” (Huang et al., 20 Oct 2025).

This construction is designed for parametric knowledge integration rather than non-parametric retrieval. A plausible implication is that the triple is rewritten into the representational format already expected by the model’s attention mechanism, rather than being appended as raw context. In AtlasKV, this is the data-centric component that precedes the attention-side mechanism and the hierarchical pruning system.

The resulting knowledge base memory is written as

$$\mathcal{M} = \{(\bm{k}^{(l)m}, \bm{v}^{(l)m})\}_{m=1}^{M},$$

where $M$ is the number of triples and $D_E$ is the sentence encoder output dimension. The key and value embeddings are sentence-level representations computed offline and later projected into layer-specific KB attention spaces.

2. Triple-to-string transformation

KG2KV begins from a textual triple $(h,r,t)$ and chooses a masked position, either head or tail. The masked entity becomes the value string. The relation is rewritten into a noun phrase conditioned on the masked side. If the tail $t$ is masked, the relation is rewritten to its noun form, such as “because” $\rightarrow$ “cause,” and the key string is built from the unmasked entity plus the relational clause. If the head $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$0 is masked, the relation is rewritten into its reversed noun, such as “result,” and used analogously (Huang et al., 20 Oct 2025).

The key composition rule is summarized as follows.

• If tail is masked, the key string has the form “the [noun] of [unmasked entity + relational clause]”.
• If head is masked, the key string is built from the reversed noun and the remaining entity.
• The value string is the masked entity, or a short proposition containing the masked entity value.

Query strings are then formed by prepending variable questioning prefixes to the key strings, including “What is …”, “Tell me …”, and “Provide details on …”. The stated purpose is to increase diversity and avoid overfitting to fixed query templates. This makes KG2KV not only a memory-construction procedure but also a synthetic supervision format for tuning the KG-specific attention heads.

After string construction, keys and values are encoded by a sentence encoder into base embeddings

$$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$1

The paper reports two practical encoder choices: all-MiniLM-L6-v2 with $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$2, and text-embedding-3-large with $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$3. Embeddings are computed offline and stored for inference-time loading (Huang et al., 20 Oct 2025).

3. Integration into LLM attention

AtlasKV places KG2KV outputs directly into a KB-augmented attention mechanism. The core architectural claim is that the model uses the LLM’s inherent attention mechanism to maintain knowledge grounding and generalization performance, while requiring no external retrievers or long context priors. AtlasKV only trains KG-specific query heads and KG projection heads, leaving the rest of the LLM frozen (Huang et al., 20 Oct 2025).

The background attention layer is defined over token embeddings $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$4, with sequence-side projections $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$5. The KB component introduces separate projections

$$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$6

which map the sentence-embedding keys and values into the attention space.

AtlasKV separates sequence and KG contributions, computes softmax distributions for each, and mixes them with data-dependent weights: $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$7 The coefficients $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$8 and $$\mathcal{G} = \{(h, r, t) \mid h, t \in \mathcal{E}, r \in \mathcal{R}\},$$9 are normalized from the exponentiated KG and sequence logits, so the final output interpolates between external KG memory and in-context sequence information. The appendix states that this formulation is equivalent to rectangular attention.

This design makes KG2KV a front end to a parametric memory system rather than a standalone representation. The strings and sentence embeddings are useful because they can be projected into the same attention pipeline that already governs token-token interactions.

4. Hierarchical organization and sub-linear scaling

KG2KV alone does not solve large-scale memory access; AtlasKV therefore couples it with Hierarchical Key-Value Pruning (HiKVP). To scale to billion-scale knowledge graphs, key embeddings are organized in a three-layer hierarchy using UMAP for dimensionality reduction and Gaussian Mixture Models for clustering. Each layer has

$h$0

clusters, and only the necessary subsets at each stage are moved to GPU (Huang et al., 20 Oct 2025).

The inference pipeline proceeds in three pruning stages.

First, only root-layer projected keys are staged on GPU. Root logits are computed and pruned by top-$h$1, and the selected root clusters determine the inter-layer subset.

Second, the selected inter-layer keys are uploaded to GPU. Inter-layer logits are computed and pruned by top-$h$2, which determines the leaf-layer subset.

Third, the selected leaf-layer keys are staged, leaf logits are computed, and the system keeps the top-$h$3 logits and their corresponding values. Only at this final stage are the required values loaded for the mixed attention computation.

The complexity claims are central. Standard self-attention has time complexity

$h$4

and memory

$h$5

KBLaM’s rectangular attention scales linearly with KB size $h$6: $h$7 in time and

$h$8

in memory. By contrast, AtlasKV with HiKVP achieves

$h$9

time and

$t$0

memory, with

$t$1

The paper’s stated memory bound is that “less than 20GB VRAM is required to augment LLMs with 1B triples.” The reported mechanism is CPU-GPU offloading combined with three-layer pruning and top-$t$2 selection, which keeps only $t$3 items on GPU per layer.

5. Training, adaptation, and empirical behavior

AtlasKV requires lightweight training, but adaptation to new knowledge after initial tuning is described as training-free. The learned components are the KG-specific query heads $t$4, initialized from the LLM’s $t$5, and the KG projection heads $t$6, initialized randomly. The backbone reported in the appendix is LLaMA3.1-8B-Instruct, with KGKVs integrated every 3 layers. Optimization uses AdamW with cosine decay from $t$7 to $t$8 over 3K iterations, or 10K for the larger encoder, with batch size 10 (Huang et al., 20 Oct 2025).

The training data consists of 20K KGKV samples from ATLAS-Wiki, whose source KG is described as containing 1.492B triples. The paper emphasizes that only small $t$9 values are seen during training, for example max $$\mathcal{M} = \{(\bm{k}^{(l)m}, \bm{v}^{(l)m})\}_{m=1}^{M},$$0, yet the trained model generalizes to much larger KGs at inference time. For updates, new triples are processed by KG2KV, encoded with the sentence encoder, inserted into the UMAP+GMM hierarchy, and indexed; no retraining of the LLM is required.

The paper also reports a data-quality effect from KG2KV itself. In the comparison of data diversity ratio and average token cost, KG2KV yields Diversity Ratio 7.864% versus Synthetic 0.003%, and Avg. Token Cost 165.7 versus 349.9. This is attributed to relation diversity and the focus on masked positions plus nounified relation phrases (Huang et al., 20 Oct 2025).

For knowledge grounding, the paper reports the following Top-1 and Top-5 results with AtlasKV $$\mathcal{M} = \{(\bm{k}^{(l)m}, \bm{v}^{(l)m})\}_{m=1}^{M},$$1 at $$\mathcal{M} = \{(\bm{k}^{(l)m}, \bm{v}^{(l)m})\}_{m=1}^{M},$$2 triples.

Dataset	AtlasKV ACC@1 / ACC@5	KBLaM ACC@1 / ACC@5
ATLAS-Pes2o-QA	87.3 / 92.7	16.4 / 45.5
ATLAS-CC-QA	89.1 / 90.9	21.8 / 38.2
Enron	67.3 / 90.9	50.9 / 83.6

The paper further states that AtlasKV’s VRAM usage remains below 20GB for 1B triples, whereas KBLaM exceeds 40GB at 100K triples. It also reports that AtlasKV consistently achieves higher GPT-4o relevance scores than KBLaM on OOD datasets, approaching ICL while using substantially less latency and memory.

6. Positioning, constraints, and scope

KG2KV is positioned against RAG, ICL, and weight-distillation approaches. Relative to RAG, the claim is that KG2KV avoids external nearest-neighbor retrieval, expensive searches, long context concatenation, and the “lost in the middle” failure mode by converting the KG into parametric KV memory inside attention. Relative to distillation or adapter-based methods, the distinction is that AtlasKV does not require retraining for new knowledge once the KG-specific heads have been tuned (Huang et al., 20 Oct 2025).

Several limitations are identified. There is an explicit performance–scalability trade-off governed by $$\mathcal{M} = \{(\bm{k}^{(l)m}, \bm{v}^{(l)m})\}_{m=1}^{M},$$3: larger values improve grounding but increase memory and time, while smaller values preserve scalability but may prune too aggressively. Hierarchical clustering quality matters; UMAP+GMM errors can route queries to suboptimal leaves, and large incremental updates may require reclustering. Relation rewriting is another source of error, since nounification uses a small LLM and ambiguous or incorrect rewrites can degrade key quality.

A further limitation concerns training data composition. The paper reports that using only event entities or only named entities degrades performance. Event-only data is described as challenging because of complex semantics, while named-only data is simpler but lacks the richness needed for generalization. This suggests that KG2KV’s utility depends not only on the attention mechanism but also on the semantic heterogeneity of the triples that are transformed into keys and values.

In this sense, KG2KV is best understood as the representational interface of AtlasKV: it translates graph triples into sentence-level keys and values aligned with attention, while HiKVP provides the scalable access path. Its distinctive claim is not merely that knowledge can be encoded as embeddings, but that billion-scale textual triples can be re-expressed as attention-native memory with sub-linear time and memory complexity, strong OOD grounding, and training-free adaptation to newly inserted knowledge after the initial tuning stage.