---
title: 'LinkQA: Synthetic Knowledge-Linked QA Dataset'
url: https://www.emergentmind.com/topics/linkqa
type: topic
---

# LinkQA: Synthetic Knowledge-Linked QA Dataset

Searching arXiv for recent papers directly relevant to “LinkQA” and closely related frameworks.
LinkQA is presented in "LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points" as a **50B-token** synthetic **multi-disciplinary question-answering dataset** generated by **LinkSyn**, a **knowledge point (KP) graph-based synthesis framework** [2508.01317]. The framework extracts KPs from QA seed data, builds a KP graph, samples short graph walks, and uses multiple seed questions that are strongly linked by KPs to synthesize new QA data. In adjacent work, several tasks are described as relevant to **LinkQA-style** settings: document-grounded QA with supporting paragraphs, QA-based entity linking, knowledge-graph question answering through query construction, and search-centric QA over heterogeneous data lakes [2406.11811][2110.02369][2406.06621][2606.10460]. This suggests that LinkQA names both a specific synthetic corpus and a broader link-centric QA perspective in which answering depends on explicit linkage among concepts, passages, entities, relations, or datasets.

## 1. Definition, motivation, and scope

The LinkQA paper frames its contribution as a response to a **training data bottleneck** for large language models: as models scale, **high-quality, diverse, knowledge-rich training data** becomes scarce [2508.01317]. It argues that existing synthetic QA pipelines are limited because **single-seed generation lacks diversity** due to generator bias, **entity-graph approaches** may connect texts through co-mentioned entities that do not represent the **core tested knowledge**, and prior methods offer weak control over **discipline distribution**, **difficulty distribution**, and **knowledge popularity vs long-tail coverage**.

Within that framing, LinkQA is explicitly a **continual pre-training** resource rather than an evaluation benchmark. This distinguishes it from datasets such as RepLiQA, which is designed to evaluate **document-grounded QA** on **unseen reference content**, and LakeQA, which is designed as a **benchmark for search-centric question answering over data lakes** [2406.11811][2606.10460]. The central design choice in LinkQA is to organize synthesis around **knowledge points** rather than documents or entities. The paper defines a knowledge point as a fundamental disciplinary content unit such as a **concept, theorem, principle, method, etc.** and states that the extractor can assign **up to 3 core KPs per item** [2508.01317].

A key empirical premise is that **75.43% of QA instances examine multiple KPs**. This is used to justify a graph over QA-derived KPs rather than a graph over raw documents or named entities [2508.01317]. The paper’s broader claim is therefore not only that QA data can be synthesized at scale, but that the synthesis process should respect the conceptual structure actually assessed by question-answer pairs.

## 2. LinkSyn and the knowledge-point graph

LinkSyn begins from an annotated seed corpus

$$
A=\{D_i\}_{i=1}^{m},
$$

where each annotated item is

$$
D_i=(t_i, s_i, h_i, K_i),
$$

with question text $t_i$, discipline $s_i$, difficulty $h_i$, and associated knowledge points

$$
K_i=\{k_{i1}, k_{i2}, \ldots, k_{in_i}\}.
$$

The paper states that KPs, discipline, and difficulty are automatically labeled using **DeepSeek-R1-distilled Qwen models**: a **Qwen2.5-14B-Instruct** model for KP extraction, **Qwen2.5-7B-Instruct** for discipline classification, and **Qwen2.5-14B-Instruct** for difficulty classification [2508.01317].

The extracted KPs are then consolidated in two stages. The first stage uses **edit-distance / prefix-based clustering**: case normalization, grouping by the first 3 characters, an edit-distance threshold defined by the greater of 3 or $\mathrm{int}(0.5 \times \max(\mathrm{len}(s_1), \mathrm{len}(s_2)))$, and cluster summarization with **Qwen-14B**. The second stage uses **co-occurrence-vector clustering**, where KPs with cosine similarity greater than **0.9** are clustered and summarized again with **Qwen-14B** [2508.01317]. The result is a final set of **10M high-quality KPs**.

The KP graph is defined as

$$
G=(K,E,W,\Phi),
$$

where

$$
K=\bigcup_{i=1}^m K_i,
$$

$$
E=\{ e_{k_p, k_q} \mid k_p, k_q \in K,\ k_p \neq k_q,\ \exists D_l \in A: k_p, k_q \in K_l \},
$$

$$
W(e_{k_p, k_q})=|\{ D_l \in A \mid k_p, k_q \in K_l \}|,
$$

and

$$
\Phi(k)=\{ D_l \in A \mid k \in K_l \}.
$$

Nodes are therefore KPs, edges are undirected KP co-occurrences in the same QA seed, edge weights are co-occurrence counts, and $\Phi$ maps a KP to supporting seed QA instances [2508.01317]. The appendix statistics reported for this graph are **10M nodes**, **153M edges**, a giant connected component containing over **92% of texts** and more than **89% of KPs**, and graph diameter **29**.

## 3. Sampling control, graph walks, and multi-seed synthesis

A distinctive feature of LinkSyn is explicit control over **coverage** and **popularity** in KP sampling. The paper defines a **coverage** objective through the expected number of distinct sampled KPs and identifies the maximizing distribution $p^a$ as uniform over KPs. It defines a **popularity** distribution as

$$
p^b(k_i)=\frac{|\Phi(k_i)|}{\sum_{k_j \in K} |\Phi(k_j)|}.
$$

These are combined through the **Knowledge Distribution Value**

$$
\mathrm{KV}(p)=\lambda D(p \,\|\, p^a) + (1-\lambda)D(p \,\|\, p^b),
$$

with

$$
p^*=\arg\min_p \mathrm{KV}(p).
$$

The appendix further states that when $D$ is squared Euclidean distance or reverse KL, the optimal solution is a convex combination of the uniform and empirical distributions [2508.01317].

In implementation, the paper realizes this balancing idea through two graph-walk policies. Under **coverage sampling**, the next node is sampled uniformly among neighbors:

$$
p^{a}(k_{t+1}\mid k_t)=\frac{1}{|\mathcal{N}(k_t)|}.
$$

Under **popularity sampling**, the next node is sampled proportionally to edge weight:

$$
p^{b}(k_{t+1}\mid k_t)=\frac{W(e_{k_t, k_{t+1}})}{\sum_{k' \in \mathcal{N}(k_t)} W(e_{k_t, k'})}.
$$

Sampled paths have length

$$
l \in \{1,2,3\},
$$

and the paper reports that longer walks improve benchmark performance, with **$l=3$** producing the best ablation scores among tested settings: **MMLU 59.88**, **C-Eval 63.04**, **ARC-C 74.50**, **DROP 46.92**, and **AVG. 61.09** [2508.01317]. For the hybrid balance between coverage and popularity, the reported best setting is **$\alpha=0.5$**, which yields **MMLU 59.94**, **C-Eval 62.89**, **ARC-C 74.00**, **DROP 47.01**, and **AVG. 60.96**.

Seed-instance selection is conditioned on both **discipline** and **difficulty**. Disciplines are organized into **62 first-level categories** using the **GB/T 13745-2008** taxonomy, and the target difficulty distribution is explicitly skewed toward harder items: **10%** for H1, **15%** for H2, **25%** for H3, **25%** for H4, and **25%** for H5 [2508.01317]. The selected seed instance for a KP is the one whose difficulty is closest to the target, with matching discipline preferred if possible.

The synthesis stage uses **DeepSeek-R1** for question generation and **DeepSeek-V3** for answer refinement. The paper describes this as **diffusion-based synthesis**, but the details indicate a reasoning-based semantic expansion process rather than a denoising diffusion probabilistic model. It reports sampling **20M seed groups** for each combination of walk length and sampling policy, plus math-constrained sampling for **LinkQA\(_\text{Math}\)**, followed by blending with **$\alpha=0.5$** [2508.01317].

## 4. Corpus properties, quality control, and pre-training results

LinkQA is described as a **50B-token** synthetic **general** QA corpus spanning **62 first-level disciplines** [2508.01317]. The paper does not provide an exact number of QA instances in the supplied text, but it does report several quality-control stages: **benchmark contamination filtering** through **embedding-based similarity filtering** and **10-gram matching**, **low-quality data filtering**, and **manual quality review**.

The manual review samples **100 QA pairs per difficulty bucket** and counts a pair as correct only if the question is solvable and the answer is fully accurate. Reported correctness is **98%** for **H1/H2**, **94%** for **H3**, and **87%** for **H4/H5** [2508.01317]. The appendix further states that difficulty validation involved **five PhD evaluators**, **Krippendorff’s $\alpha = 0.85$**, and **Pearson’s $r = 0.92$** between the difficulty scorer and human metrics.

For continual pre-training, the main setup uses **Llama-3 8B**, starting from a **2T-token checkpoint**, trained for **40B tokens** on a **1:1 LinkQA + KnowEdu** mixture [2508.01317]. The paper reports that continual pre-training with LinkQA yields an **average improvement of 11.51% on MMLU and CMMLU** and an average gain of **6.45%** across the 12 reported benchmarks.

| Benchmark | Pre-training | LinkQA |
|---|---:|---:|
| MMLU | 55.08 | 63.98 |
| CMMLU | 52.23 | 66.35 |
| ARC-C | 70.50 | 79.50 |
| DROP | 42.31 | 49.94 |
| AVG. | 43.12 | 49.57 |

The same paper reports that LinkQA achieves the best average score among the compared datasets on the 12-benchmark suite, and that the gains persist across **1.7B**, **8B**, and **16B** model sizes and across **2T** and **10T** initial checkpoints [2508.01317]. A math-focused variant, **LinkQA\(_\text{Math}\)**, is also reported, and **LinkQA\(_\text{MathCoT}\)** attains the highest average among the math-specific rows in the provided table.

## 5. Document-grounded evidence linking

A different but closely related interpretation of LinkQA appears in document-grounded QA benchmarks that require an answer to be tied to explicit evidence. RepLiQA is designed precisely to evaluate **document-grounded QA under conditions where benchmark leakage and memorized world knowledge are much less helpful than in standard QA benchmarks** [2406.11811]. Each sample contains a human-authored imaginary reference document absent from the internet, a question, a ground-truth answer derived directly from the document, and the paragraph containing the answer. The data description states that the tuple notation $(d,q,a,p)$ is a faithful description of this structure, where $p$ is the answer-containing paragraph.

RepLiQA contains **89,770 question-answer pairs** over **17,954 documents**, with **five questions per document** and documents of about **1,000 words**. About **20%** of questions are intentionally unanswerable, and the long-answer field is normalized to **`UNANSWERABLE`** in those cases [2406.11811]. The relevance to LinkQA-style evaluation is direct: a system is expected not just to answer, but to ground its answer in a supplied document and ideally identify the supporting evidence.

The benchmark is especially notable because it includes a paragraph-level supervision signal for the supporting evidence. The paper itself does not benchmark paragraph retrieval separately, but the data structure supports recasting each instance as: given $(d,q)$, retrieve the correct paragraph $p$, then answer $a$ [2406.11811]. At the same time, the supervision is limited to a **single paragraph**, not token spans, rationale chains, hyperlink anchors, or multiple linked documents. This suggests a passage-linked interpretation of LinkQA, but not a full multi-document linking benchmark.

## 6. Entity, graph, and website-centered formulations

Other systems operationalize the “link” in LinkQA differently: as entity grounding, knowledge-graph query construction, or website-restricted answer routing. The following formulations are explicitly described as relevant to LinkQA-style systems.

| Formulation | Link target | Characteristic |
|---|---|---|
| EntQA | KB entities and mention spans | Retrieves candidate entities, then finds mentions of each candidate in the document [2110.02369] |
| LinkQ | Wikidata entities, properties, and SPARQL | LLM-assisted visual interface for KG query construction and grounded execution [2406.06621] |
| QAnswer | Wikidata KG facts and Wikipedia passages | Website search that combines KG QA and free-text QA with integral and exclusive indexing [2401.09175] |

EntQA explicitly reframes **entity linking as question answering**. Instead of first detecting mentions and then disambiguating them, it **first retrieves candidate entities** with a dense retriever and then **scrutinizes the document to find mentions of each candidate** with a reader module [2110.02369]. In the paper’s formulation, entities behave like latent questions and mention spans behave like answers. The system uses overlapping passages of length **$L=32$** with stride **$S=16$**, top-**$K=100$** entity retrieval at test time, and thresholded joint reranking and span confidence to emit linked spans. On GERBIL, EntQA reports **85.8 F1** on AIDA test and **60.5** macro average across eight datasets.

LinkQ addresses **knowledge graph question answering** through a different mechanism: a **human-in-the-loop text-to-KG-query system** that uses an LLM to interpret a user’s question, extract entities, relations, and properties, map them to actual KG identifiers via KG search, generate **SPARQL**, preview the query, execute it on **Wikidata**, and summarize grounded results [2406.06621]. The reported end-to-end question-to-query protocol takes about **10–15 seconds**. The system emphasizes that the safest role for the LLM is to help **construct queries**, not to fabricate answers from parametric memory.

QAnswer, by contrast, is a **website-search-oriented integration** of **QA over free text** on **Wikipedia** and **QA over knowledge graphs** on **Wikidata** [2401.09175]. Its defining constraints are **integral indexing**, meaning the system indexes all relevant content of the website, and **exclusive indexing**, meaning it indexes only data available on the corresponding website. The orchestration is a confidence-threshold fallback: it tries **Wikidata KG QA** first, and if confidence is below threshold it tries **Wikipedia text QA**. The text side uses **BM25** in **Anserini**, a top-**29** retrieval cap, and a **RoBERTa base** reader fine-tuned on **SQuADv2.0**.

## 7. Search-centric and large-scale evidence linking

At larger scales, LinkQA-style answering becomes a search and orchestration problem over heterogeneous corpora rather than a synthesis or single-document task. LakeQA is introduced as a benchmark for **search-centric question answering over data lakes that jointly emphasizes searching and reasoning capabilities** [2606.10460]. It is built over approximately **9.5 TB** and about **40M unique files/documents**, combining **Wikipedia** and **Data.gov** sources with both **structured** and **unstructured** artifacts. The benchmark includes **1007 tasks** in **LakeQA-full** and **135 tasks** in **LakeQA-mini**, with average reasoning intensity reported as **13.12**.

The LakeQA task definition requires an agent to search for relevant documents, retrieve and inspect them, extract supporting facts, use intermediate results to formulate the next search, compose evidence across sources, and generate the final answer [2606.10460]. Experimental results are deliberately difficult: on **LakeQA-full**, **Claude-sonnet-4.5** achieves **32.87% EM**, while **GPT-5.2** achieves only **18.37% EM**. The reported failure analysis indicates that **search and exploration are the primary bottleneck**, with large gaps between the set of retrieved candidates and the set of documents actually selected and analyzed.

Web-scale LinkQA-style pipelines also face a post-retrieval compression problem. An index-based web content extraction method published later reframes webpage extraction as **index prediction** over structure-aware HTML blocks and is described as a natural **post-retrieval, pre-generation component** for LinkQA-like systems [2512.06641]. In downstream RAG QA experiments, the method improves QA accuracy relative to raw HTML, raw Markdown, chunk-rerank, HtmlRAG, prompted extraction, Firecrawl Extract, and ReaderLM-v2, while using substantially fewer extracted tokens on average. This suggests that large-scale LinkQA systems may require not only better search and reasoning, but also structure-aware evidence localization after retrieval.

Taken together, these neighboring lines of work indicate that LinkQA is not a single stabilized task definition. The specific dataset of that name is a **KP-linked synthetic QA corpus for continual pre-training** [2508.01317], whereas adjacent **LinkQA-style** formulations include **passage-linked document grounding**, **entity-conditioned linking**, **KG-backed query construction**, and **search-centric evidence chaining** across large corpora [2406.11811][2110.02369][2406.06621][2606.10460]. This suggests that the enduring technical theme is not merely question answering, but question answering under explicit linkage constraints.

Source: https://www.emergentmind.com/topics/linkqa