GPTKB v1.5: Materialized LLM Knowledge Graph
- GPTKB v1.5 is a densely interlinked, queryable knowledge base that materializes GPT-4.1's latent factual knowledge into a persistent graph.
- It employs a recursive elicitation pipeline using BFS traversal, constrained decoding, and greedy clustering to consolidate and structure model outputs.
- Designed for LLM knowledge research, GPTKB v1.5 enables systematic analysis of biases, coverage, and internal consistency through SPARQL querying and RDF interoperability.
GPTKB v1.5 is a densely interlinked, web-browsable, queryable knowledge base that materializes the factual knowledge of a frontier LLM, GPT-4.1, into a persistent graph of triples for ad hoc exploration and large-scale analysis. It is presented as the first public demonstration of the GPTKB project and as an instance of “massive-recursive LLM knowledge materialization,” a methodology designed to reduce the availability bias of one-question-at-a-time probing by recursively eliciting, consolidating, and storing the model’s factual knowledge in structured form (Hu et al., 8 Jul 2025).
1. Origins and conceptual position
GPTKB v1.5 emerged from a broader methodological shift in LLM knowledge analysis. Earlier work on “extensive materialization” argued that small, hand-picked benchmark samples impose an availability bias: researchers only measure knowledge on topics they already thought to ask about. In response, the GPTKB line treats an LLM’s latent factual knowledge as something that can be recursively explored as a graph rather than merely sampled through isolated prompts (Hu et al., 2024).
Within that trajectory, GPTKB v1.5 is explicitly framed against GPTKB v1.1. GPTKB v1.1 established the feasibility of the methodology, but its quality was poor: over 60% of triples were estimated hallucinations, and some entities accumulated more than 100k triples that were “virtually entirely hallucinated.” GPTKB v1.5 addresses this by moving to GPT-4.1, which is described as a much stronger frontier model. Compared with v1.1, v1.5 increases the number of entities by 110% while reaching similar triple count, and accuracy rises by 44 percentage points (Hu et al., 8 Jul 2025).
This suggests that GPTKB v1.5 is not merely a larger crawl. It is a materially different instantiation of the same idea: using recursive elicitation to expose an LLM’s factual world model at scale, while making that output persistent enough for browsing, querying, and statistical study.
2. Recursive materialization pipeline
The construction methodology has two main stages: knowledge elicitation and knowledge consolidation. In the elicitation stage, the process begins from a seed subject, prompts the LLM to return knowledge about that subject in triple form, identifies named entities in triple objects using LLM-based NER, enqueues newly found entities for further elicitation, and traverses recursively in a BFS-style graph exploration. Constrained decoding is used to keep outputs in triple format. In the consolidation stage, redundant and variant labels are merged, greedy clustering is applied to relations and classes, and labels are merged when their embeddings are sufficiently similar and one label is more frequent (Hu et al., 8 Jul 2025).
GPTKB v1.5 was generated starting from the seed entity Vannevar Bush. Because crawl parallelization distorts the BFS order, shortest paths from each node to the root are recomputed after construction and stored as two meta-relations, bfsLayer and bfsParent. These meta-relations support structural analysis and browsing. The KB is also converted to RDF and serialized in Turtle syntax for interoperability (Hu et al., 8 Jul 2025).
The system architecture exposes the materialized knowledge through three access modes: a web UI for keyword search and link-based exploration, a SPARQL endpoint at https://gptkb.org/query/ with a 100-second timeout, and an RDF dump on HuggingFace under CC BY 4.0. Implementation details reported for the demonstrator are Django for the web interface, Nginx for hosting, and an OpenLink Virtuoso triple store for storage (Hu et al., 8 Jul 2025).
3. Scale, composition, and data model
The final knowledge base is large enough to function as a research instrument rather than a toy crawl. Its reported composition is as follows (Hu et al., 8 Jul 2025):
| Component | Reported value |
|---|---|
| Entities | 6.1 million |
| Triples | 100 million |
| Triples including meta-relations | 120 million |
| Relations | 936k, reduced to 381k after canonicalization |
| Classes | 220k, reduced to 32k after canonicalization |
| Entity-valued triple objects | 59 million |
| Literal-valued triple objects | 41 million |
| Average triples per entity | 16.3 |
| Average label length | 19.8 characters |
The reported construction cost was $14,136 in OpenAI API calls, and the process took 18 days. Subject precision is reported as 85.3% verifiable, 3.4% plausible, and 11.3% unverifiable; 43% of subjects are in Wikidata. Triple precision is reported as 75.5% true, 5.0% plausible, and 19.5% false. Two evaluations were performed: an automated web-search-based validation on 1,000 random triples and a manual assessment of 100 triples, which agreed closely on the fraction of correct triples, 75.5% automated versus 75% manual (Hu et al., 8 Jul 2025).
Later analysis based on GPTKB v1.5 characterizes the artifact as a recursively elicited set of about 100 million factual assertions about 6.1 million entities, including 42.9 million triples with literal objects, i.e. about 43% of all triples, and 2.7 million date triples that could be parsed into YYYY-MM-DD format for timeliness analysis (Ghosh et al., 8 Oct 2025).
4. Analytical affordances and empirical findings
GPTKB v1.5 is designed for three demonstrator use cases: link-traversal-based LLM knowledge exploration, SPARQL-based structured LLM knowledge querying, and comparative exploration of the strengths and weaknesses of LLM knowledge across models. In the link-traversal interface, users can search by string, navigate entity pages, and use bfsParent and bfsLayer to inspect how entities were discovered during recursive materialization. In the comparative interface, the demo includes five models—Llama 3.3-70B-Instruct, Llama 4-Scout-17B-16E-Instruct, GPT-4o-mini, GPT-4.1, and DeepSeek-R1—and contrasts their outputs for a diverse set of 100 entities using the same prompt as in the earlier GPTKB paper (Hu et al., 8 Jul 2025).
The SPARQL endpoint enables systematic quantitative analysis. Example queries reported in the demonstrator include most frequent classes, nationality bias, gender bias, EU-citizenship counts, and relation symmetry. For instanceOf, top results include person — 1,077,803, human — 138,646, film — 120,497, company — 118,993, book — 111,414, song — 103,538, and fictional_character — 90,499. For nationality, top counts include American — 374,263, British — 133,940, French — 48,294, German — 45,381, Indian — 44,872, Canadian — 41,605, Australian — 30,978, and Japanese — 30,453. For gender, the reported counts are female — 194,785 and male — 167,480. A count over EU demonyms yields 167,454 distinct persons, described as only a lower bound (Hu et al., 8 Jul 2025).
A separate large-scale analysis uses GPTKB v1.5 as a “materialized snapshot” of GPT-4.1’s beliefs and argues that the model’s factual knowledge differs significantly from established knowledge bases. The KB is described as heavily person-centric and media-centric, with many “soft” or subjective relations such as notableWork and notableFor. The same study reports that GPTKB content is 91.5% English, with 9.5% non-English content and a long tail of 168 languages totaling 3.84% of the KB. It also reports a female-to-male ratio of 1.16 overall in gendered entities, a strong bias toward the English-speaking world, and a pronounced decline in date-valued triples around the model’s reported knowledge cutoff date of June 1, 2024 (Ghosh et al., 8 Oct 2025).
5. Accuracy, asymmetry, and internal inconsistency
One of the central empirical results associated with GPTKB v1.5 is that the factual accuracy of the materialized knowledge is substantially lower than standard benchmark performance might suggest. The large-scale analysis reports average factual accuracy of 75.5% and contrasts that with GPT-4.1’s reported 90.2% on MMLU. Domain-level results are reported as follows: Locations 75.7% true, 6.8% plausible, 17.5% false; Persons 74.2% true, 6.5% plausible, 19.3% false; Politics 72.7% true, 0% plausible, 27.3% false; Organizations 72.2% true, 7.8% plausible, 20.0% false; Science 70.6% true, 5.6% plausible, 23.8% false; Business 68.9% true, 8.1% plausible, 23.0% false; Sports 67.5% true, 8.4% plausible, 24.1% false (Ghosh et al., 8 Oct 2025).
The same work emphasizes that hallucination in GPTKB v1.5 is often a near-miss factual association rather than total invention. In a manual sample of 50 unverified examples, 9 triples (18%) had bad subjects, 9 triples (18%) had dubious predicates, and 32 triples (64%) had incorrect objects. Internal consistency is also weak. The study reports 112K subjects connected to 124K paraphrased entities, implying that at least 2% of the entities are overcounted through aliases and paraphrases. For symmetric relations, reciprocity is limited: among spouse relations, the demonstrator reports mutual 65,339, total 402,333, and fraction 0.162, while the mining study summarizes this as only 16% reciprocal. When directly prompted for missing spouse counterparts, GPT-4.1 returned the missing symmetric object only 9% of the time, returned nothing 13% of the time, and otherwise hallucinated (Hu et al., 8 Jul 2025, Ghosh et al., 8 Oct 2025).
A plausible implication is that GPTKB v1.5 should be interpreted as a large-scale representation of model beliefs rather than as a curated truth resource. Its value lies in exposing coverage, bias, asymmetry, duplication, and long-tail structure, not in behaving like a fully normalized, source-attributed encyclopedia.
6. Termination, reproducibility, and methodological significance
Subsequent work studies GPTKB-style materialization through “miniGPTKBs” in order to test whether recursive extraction terminates, whether it is reproducible, and how robust it is under perturbations. In the base setting, all 10 runs for each of three topic-specific KBs terminated, with runtimes roughly 2–3 hours; seed and temperature perturbations also terminated reliably with similar runtime and output size. However, 3 out of 10 language runs did not terminate organically and were manually suspended after 96 hours, and termination was clearly model-dependent: GPT-4.1-mini, gpt-oss-120b, and Llama 3.3 70b terminated reliably, whereas Llama 4 Scout, DeepSeek-R1, and Teuken 7b Instruct did not terminate within 96h (Giordano et al., 8 Oct 2025).
Reproducibility is reported as mixed. Yield is highly reproducible, with low coefficient of variation. Exact lexical overlap is only moderate: for Babylon named entities, Jaccard similarity is around 0.33; for The Big Bang Theory around 0.41; for DAX 40 around 0.51. Semantic similarity is substantially higher: cosine-based similarity for named entities and literals in Babylon is around 0.89, predicates around 0.88, and classes around 0.91. Robustness varies by perturbation type: high for seeds and temperature, lower for languages and models. The same study reports that intersection ensembling across multiple runs improves stability and identifies an elbow point at for all three KBs (Giordano et al., 8 Oct 2025).
These findings position GPTKB v1.5 as both an artifact and a methodology. As an artifact, it is a 100-million-triple, RDF-serializable, web-queryable KB built from GPT-4.1. As a methodology, it is a recursively expanding elicitation-and-consolidation process whose termination, reproducibility, and robustness are strong enough in some settings to support systematic knowledge analysis, but not universal across models, languages, or deployment choices (Hu et al., 8 Jul 2025, Giordano et al., 8 Oct 2025).
7. Role in LLM knowledge research
GPTKB v1.5 is presented as opening two research directions simultaneously: systematic analysis of LLM knowledge and automated KB construction. The first direction concerns statistically grounded study of factual coverage, bias, asymmetry, and errors while avoiding the confirmation bias of benchmark datasets. The second concerns the use of an LLM to generate a reusable KB at very large scale, demonstrating a path toward KBs that are not manually curated but recursively materialized from models (Hu et al., 8 Jul 2025).
Its broader significance lies in the fact that it makes an LLM’s factual knowledge persistent, navigable, and comparable. This enables analyses that conventional prompting does not: long-tail behavior, multilingual subgraphs, temporal concentration, relation asymmetry, duplication, and differences between model families on the same entity set. At the same time, the reported limitations remain fundamental: some triples are unverifiable or semantically incoherent, triple quality is imperfect even in v1.5, the KB reflects model biases and omissions, and open-domain termination and factual validation remain unresolved (Hu et al., 8 Jul 2025, Ghosh et al., 8 Oct 2025, Giordano et al., 8 Oct 2025).
In that sense, GPTKB v1.5 functions less as a replacement for curated resources such as Wikidata than as an instrument for reverse-engineering the factual structure of a frontier model’s beliefs. It turns latent parametric knowledge into a graph that can be inspected, measured, and stress-tested, thereby shifting LLM knowledge analysis from sparse probing toward large-scale materialized observation.
