LinkQA: Synthetic Knowledge-Linked QA Dataset

• The paper introduces LinkQA as a synthetic QA corpus generated by linking multiple seed questions via knowledge points to tackle data bottlenecks in training large models.
• It employs the LinkSyn framework with graph-based sampling strategies that balance coverage and popularity to ensure diverse, high-quality question-answer pairs.
• Continual pre-training with LinkQA shows significant benchmark improvements, highlighting the benefits of link-centric synthesis in enhancing model performance.

Searching arXiv for papers directly relevant to “LinkQA” and closely related frameworks. LinkQA is presented in "LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points" as a 50B-token synthetic multi-disciplinary question-answering dataset generated by LinkSyn, a knowledge point (KP) graph-based synthesis framework (Zhang et al., 2 Aug 2025). The framework extracts KPs from QA seed data, builds a KP graph, samples short graph walks, and uses multiple seed questions that are strongly linked by KPs to synthesize new QA data. In adjacent work, several tasks are described as relevant to LinkQA-style settings: document-grounded QA with supporting paragraphs, QA-based entity linking, knowledge-graph question answering through query construction, and search-centric QA over heterogeneous data lakes (Monteiro et al., 2024, Zhang et al., 2021, Li et al., 2024, Wang et al., 9 Jun 2026). This suggests that LinkQA names both a specific synthetic corpus and a broader link-centric QA perspective in which answering depends on explicit linkage among concepts, passages, entities, relations, or datasets.

1. Definition, motivation, and scope

The LinkQA paper frames its contribution as a response to a training data bottleneck for LLMs: as models scale, high-quality, diverse, knowledge-rich training data becomes scarce (Zhang et al., 2 Aug 2025). It argues that existing synthetic QA pipelines are limited because single-seed generation lacks diversity due to generator bias, entity-graph approaches may connect texts through co-mentioned entities that do not represent the core tested knowledge, and prior methods offer weak control over discipline distribution, difficulty distribution, and knowledge popularity vs long-tail coverage.

Within that framing, LinkQA is explicitly a continual pre-training resource rather than an evaluation benchmark. This distinguishes it from datasets such as RepLiQA, which is designed to evaluate document-grounded QA on unseen reference content, and LakeQA, which is designed as a benchmark for search-centric question answering over data lakes (Monteiro et al., 2024, Wang et al., 9 Jun 2026). The central design choice in LinkQA is to organize synthesis around knowledge points rather than documents or entities. The paper defines a knowledge point as a fundamental disciplinary content unit such as a concept, theorem, principle, method, etc. and states that the extractor can assign up to 3 core KPs per item (Zhang et al., 2 Aug 2025).

A key empirical premise is that 75.43% of QA instances examine multiple KPs. This is used to justify a graph over QA-derived KPs rather than a graph over raw documents or named entities (Zhang et al., 2 Aug 2025). The paper’s broader claim is therefore not only that QA data can be synthesized at scale, but that the synthesis process should respect the conceptual structure actually assessed by question-answer pairs.

2. LinkSyn and the knowledge-point graph

LinkSyn begins from an annotated seed corpus

$A=\{D_i\}_{i=1}^{m},$

where each annotated item is

$D_i=(t_i, s_i, h_i, K_i),$

with question text $t_i$, discipline $s_i$, difficulty $h_i$, and associated knowledge points

$K_i=\{k_{i1}, k_{i2}, \ldots, k_{in_i}\}.$

The paper states that KPs, discipline, and difficulty are automatically labeled using DeepSeek-R1-distilled Qwen models: a Qwen2.5-14B-Instruct model for KP extraction, Qwen2.5-7B-Instruct for discipline classification, and Qwen2.5-14B-Instruct for difficulty classification (Zhang et al., 2 Aug 2025).

The extracted KPs are then consolidated in two stages. The first stage uses edit-distance / prefix-based clustering: case normalization, grouping by the first 3 characters, an edit-distance threshold defined by the greater of 3 or $$\mathrm{int}(0.5 \times \max(\mathrm{len}(s_1), \mathrm{len}(s_2)))$$, and cluster summarization with Qwen-14B. The second stage uses co-occurrence-vector clustering, where KPs with cosine similarity greater than 0.9 are clustered and summarized again with Qwen-14B (Zhang et al., 2 Aug 2025). The result is a final set of 10M high-quality KPs.

The KP graph is defined as

$G=(K,E,W,\Phi),$

where

$K=\bigcup_{i=1}^m K_i,$

$$E=\{ e_{k_p, k_q} \mid k_p, k_q \in K,\ k_p \neq k_q,\ \exists D_l \in A: k_p, k_q \in K_l \},$$

$D_i=(t_i, s_i, h_i, K_i),$0

and

$D_i=(t_i, s_i, h_i, K_i),$1

Nodes are therefore KPs, edges are undirected KP co-occurrences in the same QA seed, edge weights are co-occurrence counts, and $D_i=(t_i, s_i, h_i, K_i),$2 maps a KP to supporting seed QA instances (Zhang et al., 2 Aug 2025). The appendix statistics reported for this graph are 10M nodes, 153M edges, a giant connected component containing over 92% of texts and more than 89% of KPs, and graph diameter 29.

3. Sampling control, graph walks, and multi-seed synthesis

A distinctive feature of LinkSyn is explicit control over coverage and popularity in KP sampling. The paper defines a coverage objective through the expected number of distinct sampled KPs and identifies the maximizing distribution $D_i=(t_i, s_i, h_i, K_i),$3 as uniform over KPs. It defines a popularity distribution as

$D_i=(t_i, s_i, h_i, K_i),$4

These are combined through the Knowledge Distribution Value

$D_i=(t_i, s_i, h_i, K_i),$5

with

$D_i=(t_i, s_i, h_i, K_i),$6

The appendix further states that when $D_i=(t_i, s_i, h_i, K_i),$7 is squared Euclidean distance or reverse KL, the optimal solution is a convex combination of the uniform and empirical distributions (Zhang et al., 2 Aug 2025).

In implementation, the paper realizes this balancing idea through two graph-walk policies. Under coverage sampling, the next node is sampled uniformly among neighbors:

$D_i=(t_i, s_i, h_i, K_i),$8

Under popularity sampling, the next node is sampled proportionally to edge weight:

$D_i=(t_i, s_i, h_i, K_i),$9

Sampled paths have length

$t_i$0

and the paper reports that longer walks improve benchmark performance, with $t_i$1 producing the best ablation scores among tested settings: MMLU 59.88, C-Eval 63.04, ARC-C 74.50, DROP 46.92, and AVG. 61.09 (Zhang et al., 2 Aug 2025). For the hybrid balance between coverage and popularity, the reported best setting is $t_i$2, which yields MMLU 59.94, C-Eval 62.89, ARC-C 74.00, DROP 47.01, and AVG. 60.96.

Seed-instance selection is conditioned on both discipline and difficulty. Disciplines are organized into 62 first-level categories using the GB/T 13745-2008 taxonomy, and the target difficulty distribution is explicitly skewed toward harder items: 10% for H1, 15% for H2, 25% for H3, 25% for H4, and 25% for H5 (Zhang et al., 2 Aug 2025). The selected seed instance for a KP is the one whose difficulty is closest to the target, with matching discipline preferred if possible.

The synthesis stage uses DeepSeek-R1 for question generation and DeepSeek-V3 for answer refinement. The paper describes this as diffusion-based synthesis, but the details indicate a reasoning-based semantic expansion process rather than a denoising diffusion probabilistic model. It reports sampling 20M seed groups for each combination of walk length and sampling policy, plus math-constrained sampling for LinkQA$t_i$3, followed by blending with $t_i$4 (Zhang et al., 2 Aug 2025).

4. Corpus properties, quality control, and pre-training results

LinkQA is described as a 50B-token synthetic general QA corpus spanning 62 first-level disciplines (Zhang et al., 2 Aug 2025). The paper does not provide an exact number of QA instances in the supplied text, but it does report several quality-control stages: benchmark contamination filtering through embedding-based similarity filtering and 10-gram matching, low-quality data filtering, and manual quality review.

The manual review samples 100 QA pairs per difficulty bucket and counts a pair as correct only if the question is solvable and the answer is fully accurate. Reported correctness is 98% for H1/H2, 94% for H3, and 87% for H4/H5 (Zhang et al., 2 Aug 2025). The appendix further states that difficulty validation involved five PhD evaluators, Krippendorff’s $t_i$5, and Pearson’s $t_i$6 between the difficulty scorer and human metrics.

For continual pre-training, the main setup uses Llama-3 8B, starting from a 2T-token checkpoint, trained for 40B tokens on a 1:1 LinkQA + KnowEdu mixture (Zhang et al., 2 Aug 2025). The paper reports that continual pre-training with LinkQA yields an average improvement of 11.51% on MMLU and CMMLU and an average gain of 6.45% across the 12 reported benchmarks.

Benchmark	Pre-training	LinkQA
MMLU	55.08	63.98
CMMLU	52.23	66.35
ARC-C	70.50	79.50
DROP	42.31	49.94
AVG.	43.12	49.57

The same paper reports that LinkQA achieves the best average score among the compared datasets on the 12-benchmark suite, and that the gains persist across 1.7B, 8B, and 16B model sizes and across 2T and 10T initial checkpoints (Zhang et al., 2 Aug 2025). A math-focused variant, LinkQA$t_i$7, is also reported, and LinkQA$t_i$8 attains the highest average among the math-specific rows in the provided table.

5. Document-grounded evidence linking

A different but closely related interpretation of LinkQA appears in document-grounded QA benchmarks that require an answer to be tied to explicit evidence. RepLiQA is designed precisely to evaluate document-grounded QA under conditions where benchmark leakage and memorized world knowledge are much less helpful than in standard QA benchmarks (Monteiro et al., 2024). Each sample contains a human-authored imaginary reference document absent from the internet, a question, a ground-truth answer derived directly from the document, and the paragraph containing the answer. The data description states that the tuple notation $t_i$9 is a faithful description of this structure, where $s_i$0 is the answer-containing paragraph.

RepLiQA contains 89,770 question-answer pairs over 17,954 documents, with five questions per document and documents of about 1,000 words. About 20% of questions are intentionally unanswerable, and the long-answer field is normalized to UNANSWERABLE in those cases (Monteiro et al., 2024). The relevance to LinkQA-style evaluation is direct: a system is expected not just to answer, but to ground its answer in a supplied document and ideally identify the supporting evidence.

The benchmark is especially notable because it includes a paragraph-level supervision signal for the supporting evidence. The paper itself does not benchmark paragraph retrieval separately, but the data structure supports recasting each instance as: given $s_i$1, retrieve the correct paragraph $s_i$2, then answer $s_i$3 (Monteiro et al., 2024). At the same time, the supervision is limited to a single paragraph, not token spans, rationale chains, hyperlink anchors, or multiple linked documents. This suggests a passage-linked interpretation of LinkQA, but not a full multi-document linking benchmark.

6. Entity, graph, and website-centered formulations

Other systems operationalize the “link” in LinkQA differently: as entity grounding, knowledge-graph query construction, or website-restricted answer routing. The following formulations are explicitly described as relevant to LinkQA-style systems.

Formulation	Link target	Characteristic
EntQA	KB entities and mention spans	Retrieves candidate entities, then finds mentions of each candidate in the document (Zhang et al., 2021)
LinkQ	Wikidata entities, properties, and SPARQL	LLM-assisted visual interface for KG query construction and grounded execution (Li et al., 2024)
QAnswer	Wikidata KG facts and Wikipedia passages	Website search that combines KG QA and free-text QA with integral and exclusive indexing (Guo et al., 2024)

EntQA explicitly reframes entity linking as question answering. Instead of first detecting mentions and then disambiguating them, it first retrieves candidate entities with a dense retriever and then scrutinizes the document to find mentions of each candidate with a reader module (Zhang et al., 2021). In the paper’s formulation, entities behave like latent questions and mention spans behave like answers. The system uses overlapping passages of length $s_i$4 with stride $s_i$5, top-$s_i$6 entity retrieval at test time, and thresholded joint reranking and span confidence to emit linked spans. On GERBIL, EntQA reports 85.8 F1 on AIDA test and 60.5 macro average across eight datasets.

LinkQ addresses knowledge graph question answering through a different mechanism: a human-in-the-loop text-to-KG-query system that uses an LLM to interpret a user’s question, extract entities, relations, and properties, map them to actual KG identifiers via KG search, generate SPARQL, preview the query, execute it on Wikidata, and summarize grounded results (Li et al., 2024). The reported end-to-end question-to-query protocol takes about 10–15 seconds. The system emphasizes that the safest role for the LLM is to help construct queries, not to fabricate answers from parametric memory.

QAnswer, by contrast, is a website-search-oriented integration of QA over free text on Wikipedia and QA over knowledge graphs on Wikidata (Guo et al., 2024). Its defining constraints are integral indexing, meaning the system indexes all relevant content of the website, and exclusive indexing, meaning it indexes only data available on the corresponding website. The orchestration is a confidence-threshold fallback: it tries Wikidata KG QA first, and if confidence is below threshold it tries Wikipedia text QA. The text side uses BM25 in Anserini, a top-29 retrieval cap, and a RoBERTa base reader fine-tuned on SQuADv2.0.

7. Search-centric and large-scale evidence linking

At larger scales, LinkQA-style answering becomes a search and orchestration problem over heterogeneous corpora rather than an overview or single-document task. LakeQA is introduced as a benchmark for search-centric question answering over data lakes that jointly emphasizes searching and reasoning capabilities (Wang et al., 9 Jun 2026). It is built over approximately 9.5 TB and about 40M unique files/documents, combining Wikipedia and Data.gov sources with both structured and unstructured artifacts. The benchmark includes 1007 tasks in LakeQA-full and 135 tasks in LakeQA-mini, with average reasoning intensity reported as 13.12.

The LakeQA task definition requires an agent to search for relevant documents, retrieve and inspect them, extract supporting facts, use intermediate results to formulate the next search, compose evidence across sources, and generate the final answer (Wang et al., 9 Jun 2026). Experimental results are deliberately difficult: on LakeQA-full, Claude-sonnet-4.5 achieves 32.87% EM, while GPT-5.2 achieves only 18.37% EM. The reported failure analysis indicates that search and exploration are the primary bottleneck, with large gaps between the set of retrieved candidates and the set of documents actually selected and analyzed.

Web-scale LinkQA-style pipelines also face a post-retrieval compression problem. An index-based web content extraction method published later reframes webpage extraction as index prediction over structure-aware HTML blocks and is described as a natural post-retrieval, pre-generation component for LinkQA-like systems (Chen et al., 7 Dec 2025). In downstream RAG QA experiments, the method improves QA accuracy relative to raw HTML, raw Markdown, chunk-rerank, HtmlRAG, prompted extraction, Firecrawl Extract, and ReaderLM-v2, while using substantially fewer extracted tokens on average. This suggests that large-scale LinkQA systems may require not only better search and reasoning, but also structure-aware evidence localization after retrieval.

Taken together, these neighboring lines of work indicate that LinkQA is not a single stabilized task definition. The specific dataset of that name is a KP-linked synthetic QA corpus for continual pre-training (Zhang et al., 2 Aug 2025), whereas adjacent LinkQA-style formulations include passage-linked document grounding, entity-conditioned linking, KG-backed query construction, and search-centric evidence chaining across large corpora (Monteiro et al., 2024, Zhang et al., 2021, Li et al., 2024, Wang et al., 9 Jun 2026). This suggests that the enduring technical theme is not merely question answering, but question answering under explicit linkage constraints.