GS-Quant: Discrete Semantics in KGC

• GS-Quant is a framework for knowledge graph completion that converts entity representations into semantically ordered discrete code sequences, bridging the gap between continuous embeddings and discrete LLM tokens.
• It integrates residual quantization with Granular Semantic Enhancement and Generative Structural Reconstruction to impose a coarse-to-fine hierarchical structure on entity codes.
• Empirical results on WN18RR and FB15k-237 demonstrate GS-Quant’s superior performance with notable improvements in MRR and Hits@1 compared to prior LLM baselines.

GS-Quant is a framework for knowledge graph completion (KGC) that quantizes entity representations into semantically organized discrete code sequences for use by LLMs. It is designed to address the modality gap between continuous knowledge-graph embeddings and discrete LLM tokens by replacing flat numerical compression with a coarse-to-fine code structure. The method combines residual quantization with a Granular Semantic Enhancement module and a Generative Structural Reconstruction module, then inserts the learned code tokens into an LLM vocabulary for KGC prompting and reranking. On WN18RR and FB15k-237, GS-Quant reports improvements over prior LLM baselines in both MRR and Hits@1, with the strongest gains concentrated in top-rank accuracy (Xie et al., 23 Apr 2026).

1. Problem formulation and conceptual basis

GS-Quant is formulated in the standard KGC setting. A knowledge graph is defined as

$$\mathcal{G} = \{\mathcal{E}, \mathcal{R}, \mathcal{T}\},$$

where $\mathcal{E}$ is the entity set, $\mathcal{R}$ is the relation set, and $\mathcal{T}$ is the set of observed triples. Each triple has the form

$(h,r,t)\in\mathcal{T},$

with $h,t\in\mathcal{E}$ and $r\in\mathcal{R}$. The downstream task is mainly tail prediction, $(h,r,?)$, although head prediction is also possible. The paper’s central premise is that existing LLM-based KGC methods either linearize graph structure into text or inject dense embeddings into the model, and that both choices are mismatched to autoregressive token reasoning (Xie et al., 23 Apr 2026).

The framework is motivated by four contrasts between KG embeddings and LLM tokens: continuous versus discrete representation, holistic versus sequential structure, lookup-style versus autoregressive reasoning dynamics, and entangled latent semantics versus token-like compositional semantics. GS-Quant treats these contrasts not as an incidental implementation detail but as the main systems problem. Its argument is that prior quantization approaches such as SSQR produce code tuples that act as compressed numeric surrogates rather than as semantically ordered symbolic descriptors.

This motivates the method’s core design principle: entity codes should follow a linguistic coarse-to-fine logic. Earlier code positions are intended to encode broad semantic categories, while later positions refine specific attributes or instance-level distinctions. In the paper’s formulation, that principle is enforced by quantization-time supervision rather than being left to emerge implicitly.

2. Entity representation and residual quantization backbone

Before quantization, GS-Quant constructs an entity representation by combining structural and textual signals. Structural information is produced by a KG embedding backbone, exemplified by RotatE, yielding

$\mathbf{s}_x^{\mathcal{G}} \in \mathbb{R}^d.$

Textual information is produced by a pretrained LLM over entity names and descriptions, yielding

$\mathbf{s}_x^T \in \mathbb{R}^d.$

The fused entity representation is

$\mathcal{E}$0

with $\mathcal{E}$1 controlling the balance between relational and textual content (Xie et al., 23 Apr 2026).

Quantization is built on residual quantization. An entity representation $\mathcal{E}$2 is projected by an MLP,

$\mathcal{E}$3

and the initial residual is

$\mathcal{E}$4

GS-Quant uses $\mathcal{E}$5 residual quantization levels, with codebooks

$\mathcal{E}$6

In the reported implementation, the framework uses 4 codebook layers and codebook size 1024, so each entity receives a 4-token code tuple. At level $\mathcal{E}$7, the assigned code is

$\mathcal{E}$8

followed by the residual update

$\mathcal{E}$9

The final discrete representation is

$\mathcal{R}$0

The residual quantization loss is

$\mathcal{R}$1

where $\mathcal{R}$2 is stop-gradient and $\mathcal{R}$3 is the commitment weight. The paper is explicit that $\mathcal{R}$4 alone is insufficient for semantically meaningful codebooks; it creates a numerical hierarchy, but not necessarily a semantic one.

3. Granular Semantic Enhancement

Granular Semantic Enhancement (GSE) is the component that aligns quantization depth with semantic granularity. The framework first constructs a hierarchy tree $\mathcal{R}$5 by hierarchical clustering over entity semantic representations. For a batch $\mathcal{R}$6 of entities, each entity $\mathcal{R}$7 is associated with a cluster centroid $\mathcal{R}$8. To allow gradients to pass through discrete assignments, GS-Quant defines a differentiable surrogate sequence

$\mathcal{R}$9

This makes the forward pass use the selected code vector while the backward pass follows the residual path (Xie et al., 23 Apr 2026).

GSE contains two losses. The first is a coarse-to-fine alignment objective,

$\mathcal{T}$0

Because the layer weight decays with depth, earlier quantization levels receive stronger pressure to align with broad semantics. The second is a hierarchical separability objective,

$\mathcal{T}$1

Here $\mathcal{T}$2 is the set of neighbor centroids of entity $\mathcal{T}$3 in the hierarchy tree. The reverse decay in $\mathcal{T}$4 pushes deeper layers toward finer discrimination.

The total GSE objective is

$\mathcal{T}$5

Operationally, this makes low-index codes coarse and high-index codes discriminative. The paper’s qualitative examples, including containment hierarchy and fine-grained degree distinctions, are consistent with this intended role separation.

4. Generative Structural Reconstruction and LLM integration

Even semantically stratified codebooks do not guarantee that the code tuple behaves like an ordered sequence. GS-Quant therefore adds Generative Structural Reconstruction (GSR), which uses a lightweight GPT-style Transformer decoder to impose causal dependencies over the code sequence. The decoder takes learnable queries

$\mathcal{T}$6

concatenated with the surrogate code sequence $\mathcal{T}$7, and produces outputs $\mathcal{T}$8 through causal self-attention. These outputs are aligned with multi-granular targets: $\mathcal{T}$9 reconstructs the entity embedding $(h,r,t)\in\mathcal{T},$0, while later query positions reconstruct hierarchy ancestors $(h,r,t)\in\mathcal{T},$1 (Xie et al., 23 Apr 2026).

The GSR loss is

$(h,r,t)\in\mathcal{T},$2

In the reported settings, reconstruction layers are 2, reconstruction heads are 4, and parent recon count is 5. For FB15k-237, $(h,r,t)\in\mathcal{T},$3 and $(h,r,t)\in\mathcal{T},$4; for WN18RR, $(h,r,t)\in\mathcal{T},$5 and $(h,r,t)\in\mathcal{T},$6. The full pretraining objective is

$(h,r,t)\in\mathcal{T},$7

After pretraining, codebook entries are added as new tokens in the LLM vocabulary. The original LLM parameters are frozen, and only the new token embeddings and LoRA adapter matrices in attention and feed-forward layers are updated. The reported fine-tuning setup uses Llama3-8B as the unified LLM backbone, with LoRA rank 32, LoRA $(h,r,t)\in\mathcal{T},$8, learning rate $(h,r,t)\in\mathcal{T},$9, optimizer AdamW, bf16 precision, max training steps 3800, and DeepSpeed Zero-3. The prompt includes the query triple, textual description of the unknown entity, relational context triplets, and a candidate list in which each candidate is annotated with its 4-token code sequence. Inference is performed by having the fine-tuned LLM select the most plausible entity from the candidate set.

5. Empirical performance and ablation evidence

GS-Quant is evaluated on FB15k-237 and WN18RR, using Hits@1, Hits@3, Hits@10, and MRR. The paper emphasizes that Hits@1 and MRR are the most practically informative because they measure whether the correct entity is ranked at the top (Xie et al., 23 Apr 2026).

Dataset	GS-Quant	Best prior LLM baseline
WN18RR MRR	0.635	0.617
WN18RR Hits@1	0.594	0.569
FB15k-237 MRR	0.455	0.439
FB15k-237 Hits@1	0.386	0.364

On WN18RR, GS-Quant reports MRR 0.635, Hits@1 0.594, Hits@3 0.649, and Hits@10 0.712. The strongest prior LLM baseline reported in the paper, DIFT, reaches MRR 0.617 and Hits@1 0.569. On FB15k-237, GS-Quant reports MRR 0.455, Hits@1 0.386, Hits@3 0.479, and Hits@10 0.592, compared with DIFT at MRR 0.439 and Hits@1 0.364. The gains are therefore +0.018 MRR and +0.025 Hits@1 on WN18RR, and +0.016 MRR and +0.022 Hits@1 on FB15k-237.

The ablations are consistent with the architecture claim. On FB15k-237, removing $h,t\in\mathcal{E}$0 drops MRR from 0.455 to 0.450, removing $h,t\in\mathcal{E}$1 also yields 0.450, removing $h,t\in\mathcal{E}$2 yields 0.448, and removing the code tokens entirely yields 0.404. On WN18RR, the corresponding MRR values are 0.629, 0.625, 0.627, and 0.607, versus 0.635 for the full model. The largest degradation comes from removing code tokens, indicating that explicit discrete semantic units are the most consequential component.

The paper also reports codebook analyses and t-SNE visualizations indicating that coarse layers are sparse and globally uniform while deeper layers are more discriminative, and that GS-Quant tokens occupy a compact but integrated region of the LLM token space. This suggests that the framework is not only compressing entity representations but also reformatting them into a vocabulary structure more compatible with LLM sequence processing.

6. Implementation profile, limitations, and related usage context

GS-Quant’s quantization pretraining uses embedding dimension 512, encoder layers [512, 512, 512], codebook layers 4, codebook size 1024, learning rate $h,t\in\mathcal{E}$3, commitment loss weight 0.25, $h,t\in\mathcal{E}$4, and $h,t\in\mathcal{E}$5. For FB15k-237, maximum pretraining steps are 500 with batch size 14,541; for WN18RR, maximum steps are 1000 with batch size 16,348. The framework also reports a checkpoint-selection criterion based on codebook entropy,

$h,t\in\mathcal{E}$6

where $h,t\in\mathcal{E}$7 is the empirical activation frequency of code $h,t\in\mathcal{E}$8 in codebook $h,t\in\mathcal{E}$9. Higher entropy is reported to correlate positively with downstream KGC performance (Xie et al., 23 Apr 2026).

The paper explicitly notes several limitations. Performance and efficiency remain bounded by the pretrained LLM backbone. The learned code semantics are benchmark-specific, with transfer to substantially different graphs or domains left unclear. The evaluation scope is concentrated on link prediction rather than broader generative reasoning tasks. The text also implies additional failure modes: hierarchy quality depends on clustering quality; weak initial embeddings can degrade semantic supervision; entities with ambiguous or sparse descriptions may be harder to align; and a discrete bottleneck may still lose information for subtle distinctions.

A further point of clarification is terminological. In current arXiv usage, “GS-Quant” refers specifically to Granular Semantic and Generative Structural Quantization for Knowledge Graph Completion, and should be distinguished from unrelated methods with similar abbreviations, such as GSQ for Gumbel-Softmax weight quantization of LLMs (Dadgarnia et al., 20 Apr 2026) and GSQ-Tuning for group-shared-exponent fully quantized fine-tuning (Zhou et al., 18 Feb 2025). Within KGC, however, GS-Quant’s distinctive contribution is precise: it treats discrete codes not as flat compressed indices, but as semantically stratified and causally structured descriptors that can be consumed by an autoregressive LLM.