SAC-KG: Automatic Domain KG Construction

• SAC-KG is a framework that constructs domain knowledge graphs by iteratively generating, verifying, and pruning candidate triples to ensure high precision.
• The approach uses large language models to extract domain-specific information, achieving over 20% precision improvement compared to traditional methods.
• It integrates a Generator, Verifier, and Pruner to mitigate contextual noise, knowledge hallucination, and error propagation during graph expansion.

SAC-KG, short for exploiting LLMs as Skilled Automatic Constructors for domain Knowledge Graph, is a general framework for automatic domain knowledge graph construction from raw domain corpora. It is designed for specialized domains in which knowledge acquisition must be precise and dependable, yet conventional KG construction pipelines remain heavily dependent on human intervention. SAC-KG organizes the construction process around three components—Generator, Verifier, and Pruner—to produce specialized, precise, multi-level KGs while mitigating contextual noise in raw text, knowledge hallucination in LLM outputs, and error propagation during iterative graph expansion. In experiments on a rice-domain corpus, it automatically constructs a domain KG at the scale of over one million nodes and achieves a precision of 89.32%, yielding over 20% increase in precision rate compared with existing state-of-the-art methods for KG construction (Chen et al., 2024).

1. Formal definition and problem setting

SAC-KG operates in the standard triple-based formulation of knowledge graphs. A KG is a collection of factual triples

$(h, r, t),$

where $h$ is the head entity, $r$ is the relation, and $t$ is the tail entity. The target object is a domain knowledge graph

$$\mathcal{G} = (\mathcal{E}, \mathcal{R}, \mathcal{T}),$$

with $$\mathcal{T} \subseteq \mathcal{E} \times \mathcal{R} \times \mathcal{E}$$, constructed from unstructured domain corpora $\mathcal{C}$ and a set of seed entities $E_0$. In SAC-KG, the objective is to extract factual, domain-specific triples from $\mathcal{C}$ with high precision and minimal human intervention.

The framework is motivated by several weaknesses in prior KG construction regimes. Rule-based and open information extraction methods rely on lexical patterns, syntactic parsing, or semantic role labeling, but they require substantial expert engineering and tend to produce uninformative or redundant triples. Distant-supervision and pattern-learning approaches often presuppose existing KGs or annotated data, which is problematic in narrow domains with limited labels. Direct LLM-based extraction, as exemplified by methods such as DeepEx and PIVE, improves expressivity but introduces two prominent failure modes: contextual noise in input corpora and hallucinated knowledge in output triples. SAC-KG is explicitly structured to exploit LLMs as domain experts while constraining them through retrieval, verification, and controlled graph expansion.

A central distinction of SAC-KG is that it treats KG construction as an iterative, multi-level process rather than a one-shot extraction task. For a given entity $e$, the framework first induces a single-level entity-centered subgraph, then decides which generated tails should become new heads in subsequent levels. This converts construction into a controlled entity-induced tree search rather than unrestricted graph growth.

2. Generator–Verifier–Pruner architecture

SAC-KG is organized around three components with distinct operational roles. The Generator proposes triples, the Verifier filters and corrects them, and the Pruner determines whether graph expansion should continue through newly produced tail entities.

Component	Primary function	Output
Generator	LLM-based triple generation from retrieved domain text and open-KG examples	Candidate single-level KG for a head entity
Verifier	Rule-based error detection and targeted correction	Verified triples
Pruner	Binary decision on tail expandability	“growing” or “pruned”

The Generator receives a specified entity $h$0, domain corpora, and example triples from an open KG. It then constructs a prompt comprising a text segment, an instruction segment, and an example segment. The output is a set

$h$1

which forms a specialized single-level KG rooted at $h$2.

The Verifier is a parameter-free control module. It checks the generated triple list for insufficient quantity, malformed output format, head-entity inconsistency, contradiction between head and tail, and logical conflicts. The logical-conflict stage relies on RuleHub, which contains more than 7000 logical rules mined from open KGs. Depending on error type and quantity, the Verifier either re-prompts the LLM with targeted corrective instructions or discards problematic triples.

The Pruner addresses a separate issue: not every correct tail entity is a useful frontier node for further graph construction. SAC-KG therefore casts expansion control as a binary classification problem. For each verified tail entity $h$3,

$h$4

If $h$5 is labeled “growing”, it becomes a head entity at the next level; otherwise, recursive expansion stops at that node.

3. Generation, verification, and pruning mechanics

The Generator uses two retrieval channels to control LLM behavior. The domain corpora retriever segments the domain corpus into sentences, ranks them by frequency of occurrence of the target entity, and concatenates the top-ranked sentences into a fixed-length text ordered by relevance. This is intended to reduce contextual noise and ground generation in specialized domain evidence. The open-KG retriever draws example triples from DBpedia. If the target entity exists in DBpedia, SAC-KG retrieves up to 10 triples with that entity as head; if not, it tokenizes the entity into sub-entities and searches again; if that also fails, it falls back to randomly selected DBpedia triples. These examples specify output structure and relation style for in-context learning.

The Verifier processes each generated triple list sequentially. A quantity check marks outputs with fewer than $h$6 triples, with $h$7, as “Quantity insufficient”. The format check enforces the $h$8 structure, verifies that the head entity is the specified entity $h$9, and detects trivial head–tail contradictions when head and tail are identical. The conflict check uses RuleHub to identify general logical conflicts. Each detected error type is mapped to an error-specific regeneration prompt such as “Please generate it again strictly according to the requirements” or “Please generate it again strictly according to the format requirements, paying attention to the format of the example triples.” If the number of triples in an error bucket exceeds a threshold, the Verifier merges the corrective prompt with the original input and re-invokes the LLM.

The Pruner is implemented as a T5-based text-to-label classifier fine-tuned on DBpedia-derived entity sets. Positive examples are drawn from entities that frequently appear as heads; negative examples are drawn from tail entities that do not overlap with that head list. The classifier outputs the token “growing” or “pruned”. The paper reports LoRA-based fine-tuning with 2 epochs, batch size 64, and learning rate 0.001. Its operational rationale is straightforward: a correct tail such as “20–25 degrees Celsius” in a triple like (rice, optimal growth temperature, 20–25 degrees Celsius) should not initiate another round of KG expansion.

4. Multi-level construction procedure and LLM usage

SAC-KG distinguishes between single-level and multi-level graphs. A single-level entity-induced KG for a head entity $r$0 is the set of triples having $r$1 as head. A multi-level KG is built by iteratively expanding from tail entities labeled “growing”, yielding a tree-like layered structure rooted at the initial seed entities.

Conceptually, if $r$2 denotes the set of heads at level $r$3, the procedure is: generate candidate triples for each $r$4, verify them, add the verified triples to the global graph, and collect all tails labeled “growing” into $r$5. The recursion terminates when the frontier is empty or a depth limit is reached. This layer-by-layer expansion is described as an entity-induced tree search algorithm.

The LLM’s role is confined to generation and regeneration; the framework does not fine-tune the Generator LLM on the target domain. In that sense, the Generator is parameter-free and unsupervised, relying entirely on retrieval and prompt design. The paper evaluates ChatGPT, Qwen 7B, Llama2 7B, and Llama2 13B as Generator backbones, all used in zero-shot or few-shot in-context mode. A low temperature setting of 0.1 is adopted to stabilize outputs and reduce randomness.

The prompt structure is central to the framework’s behavior. The text segment supplies domain evidence, the instruction segment constrains the task to extract triples from the given text with the specified head entity, and the example segment provides DBpedia triples in the intended output format. This arrangement exploits LLM prior knowledge and in-context learning while attempting to bind output structure and semantic scope tightly to the domain corpus.

5. Evaluation methodology and empirical results

Evaluation in SAC-KG is built around three metrics: precision, number of recalls, and domain specificity (Chen et al., 2024). Precision is defined conventionally as

$r$6

where $r$7 and $r$8 denote correct and incorrect triples, respectively. Because full gold-standard domain KGs are unavailable, recall in the classical sense is infeasible; SAC-KG instead reports “Number of recalls”, defined as the average number of verified triples per domain text,

$r$9

where $t$0 is the number of verified triples for text $t$1. Domain specificity is defined as

$t$2

where $t$3 is the set of generated triples, $t$4 is the subset that is both correct and domain-related, and $t$5 is the set of triples already present in DBpedia.

The experimental domain is rice agriculture. The corpus comprises 70 specialized books, 1522 web pages, and 24,000 genealogical records, with HTML tags, images, tables, and meaningless characters removed during preprocessing. Triple correctness is judged automatically by GPT-4 following an LLM-as-a-judge protocol. Human evaluation indicates substantial alignment with human experts: GPT-4 reaches precision 0.906, recall 0.951, F1 0.928, and Cohen’s Kappa 0.613 when compared with human judgments.

On the overall domain KG task, SAC-KG with ChatGPT reaches Number of recalls 8.09, Precision 89.32, and Domain Specificity 81.25. The strongest reported baseline, PIVE with ChatGPT, reaches Number of recalls 5.08, Precision 64.48, and Domain Specificity 51.58. SAC-KG variants with Qwen 7B and Llama2 13B also substantially outperform classical OIE baselines, reporting precisions of 69.89 and 69.40, respectively. The headline result is therefore both quantitative and structural: SAC-KG improves precision by more than 20 percentage points over the best baseline while scaling to a graph with over one million nodes.

Case studies reveal that gains are not uniform across entity classes. In the rice variety category, SAC-KG with ChatGPT reports Number of recalls 13.11, Precision 84.28, and Domain Specificity 76.88, compared with PIVE’s 2.57, 54.48, and 43.58. In the rice expert category, SAC-KG with ChatGPT reaches Precision 93.33 and Domain Specificity 84.43. The framework is also evaluated on open OIE benchmarks—OIE2016, WEB, NYT, and PENN—where, for example, on WEB it records F1 96.6 and AUC 95.7, surpassing reported rule-based and LLM-based comparators.

Ablation studies isolate the role of each component. Removing the open-KG prompt examples, the domain-text retriever, the Verifier, or the Pruner all degrades performance. The first iteration already shows the effect of verification: full SAC-KG reports recalls 13.50, precision 88.81, and domain specificity 80.50, while the variant without Verifier drops to precision 76.47. In deeper iterations, removing the Pruner notably harms precision, indicating that expansion control is particularly important for preventing error propagation in multi-level KG growth.

6. Position in the literature, limitations, and terminological scope

Within the KG-construction literature, SAC-KG occupies a specific position. It differs from rule-based OIE systems such as OpenIE 6 and Stanford OIE by replacing hand-engineered extraction logic with retrieval-grounded LLM generation. It differs from DeepEx and PIVE by making verification and frontier control first-class architectural components rather than relying primarily on direct prompting. The result is a framework oriented toward high-precision, domain-specific, multi-level graph induction rather than generic triple harvesting.

The framework also has clear limitations. It does not inject or update domain knowledge inside the LLM itself; it constructs an external KG rather than a domain-specific LLM. Its Verifier concentrates on format and rule-based conflicts rather than full factual verification, so residual errors remain possible. Its Pruner is trained on DBpedia, which works because “growability” is treated as partly domain-agnostic, but this choice does not guarantee optimality in every domain. The authors further note continuing challenges from hallucinations, corpus noise and bias, rare entities, and the computational cost of tree expansion with repeated LLM calls (Chen et al., 2024).

The name SAC-KG is also potentially confusing because the acronym “SAC” is heavily overloaded across adjacent arXiv literatures. In other contexts, SAC denotes Sensing-Assisted Communications in integrated sensing and communications (Ramos et al., 2024, Yang et al., 2023), Semantic-Anchor Compression in long-context LLM compression (Liu et al., 10 Oct 2025), Selector-Actor-Critic in reinforcement learning (Masadeh et al., 2020), and Soft Actor-Critic in beamforming for joint secret key generation and data transmission (Wang et al., 14 Mar 2026). In the present context, however, SAC-KG specifically denotes the LLM-based domain KG construction framework “Skilled Automatic Constructors for domain Knowledge Graph.” A plausible broader implication is that the acronym may continue to acquire domain-specific reinterpretations around knowledge graphs—for example, spreading-activation-over-KGs (Wu et al., 11 Jun 2025) or KG-oriented extensions of semantic-anchor compression (Liu et al., 10 Oct 2025)—but those uses are best treated as adjacent conceptual developments rather than the established meaning of SAC-KG.