DLR-Book: Design-Logic Synthesis for Reasoning

• DLR-Book is a synthetic, doc-centric dataset that applies design logic to generate over 3M multi-step reasoning questions spanning 75 academic fields.
• It utilizes a structured retrieval-and-synthesis pipeline with design logic extraction and deduplication to ensure high quality and diversity.
• Empirical results show that training on DLR-Book significantly improves benchmark performance on MMLU and GPQA compared to web-based datasets.

Searching arXiv for recent and related papers on DLR-Book and adjacent reasoning-data methodologies. arXiv search: "DESIGNER DLR-Book reasoning dataset" Design-Logic-Reasoning-Book (DLR-Book) is the book-sourced half of the DESIGNER pipeline’s synthetic reasoning corpus. It contains 3,040,620 challenging, graduate-level questions generated from curated books across 75 academic disciplines, and it is intended to address a gap identified in open reasoning data: most open reasoning datasets concentrate on math and code and depend on existing competition or problem platforms, leaving many university disciplines underrepresented and failing to consistently elicit multi-step reasoning (Liu et al., 18 Aug 2025).

1. Definition and scope

DLR-Book is a doc-centric, design-logic-guided synthetic dataset. Its core idea is to start from curated book chapters or segments and apply recovered “design logics” to author new questions that require multi-step reasoning over the source context. In this formulation, a “design logic” is an abstract, transferable meta-structure that encodes the educator’s chain of design decisions for crafting a challenging question from knowledge points. The resulting dataset is explicitly distinguished from shallow doc-centric recall pipelines and from query-centric evolution methods that are bounded by a seed pool’s domain coverage (Liu et al., 18 Aug 2025).

The dataset is paired with Design-Logic-Reasoning-Web (DLR-Web), which uses the same pipeline and the same library of design logics but draws its source material from filtered web text rather than books. Within controlled comparisons, books are described as, on average, higher-quality and more structured, and DLR-Book yields slightly stronger training outcomes under equal-size comparisons; one cited example is GPQA-Diamond Pass@1, where book reaches 58.33% and web 57.07% (Liu et al., 18 Aug 2025).

A common misconception is to treat DLR-Book as a conventional human-authored textbook. The dataset is instead synthetic, large-scale, and question-centric: each item is generated from book segments under an explicit logic-guided synthesis pipeline rather than copied from a pre-existing instructional volume. This suggests that the term “Book” designates the provenance of the source corpus, not the editorial form of the output.

2. Design Logic as the organizing principle

The paper defines “Design Logic” as an abstract, transferable meta-structure that encodes the educator’s chain of design decisions for crafting a challenging question from knowledge points. Practically, a design logic is a sequence such as: select knowledge objectives, construct scenario or context, define multi-step reasoning path with subgoals and checks, introduce traps or distractors, and validate and finalize the question (Liu et al., 18 Aug 2025).

This design-logic layer is extracted rather than manually authored from scratch. From 132,409 curated questions, diversified by in-bank clustering and difficulty sampling, DeepSeek-R1-0528 abstracts a structured logic per question using a dedicated prompt and returns the logic in Mermaid format. These logics are then embedded with Qwen3-Embedding-4B and deduplicated via graph connected components with similarity threshold $\tau = 0.85$. The representative of each component $C$ is selected by maximizing total similarity to peers,

$$i^* = \arg\max_{i \in C} \sum_{j \in C,\, j \ne i} S_{ij}.$$

After deduplication, 125,328 unique design logics remain across 75 disciplines (Liu et al., 18 Aug 2025).

The representational choice is notable. Design logics are stored in English Mermaid format, with flow-like steps and checks. A typical logic may include nodes such as “Identify core theorem or law,” “Construct applied scenario,” “Require subgoal proof or quantitative derivation,” “Add plausible distractors,” “Verify assumptions,” and “Conclude” (Liu et al., 18 Aug 2025). This suggests that DESIGNER treats question generation as controlled structural transfer rather than mere paraphrase or prompt-based expansion.

3. Source corpus and synthesis pipeline

The source is a proprietary library of books. Chapters are split into $\le 5{,}000$-word blocks and then MinHash deduplicated. Each segment is discipline-labeled with a fine-tuned ModernBERT-large classifier aligned to a 75-discipline taxonomy spanning STEM, humanities, social sciences, arts, and professional fields. Quality control combines a BERT-based readability classifier and the fineweb-edu helpfulness score on a $0$–$5$ scale; segments with negative readability are removed, and within each discipline, segments are sorted by helpfulness and sampled from the top until quotas are met (Liu et al., 18 Aug 2025).

Question synthesis proceeds through a coarse-to-fine retrieve-and-generate procedure. For a book segment $t$ and a design logic $d$, both are embedded with Qwen3-Embedding-4B under a retrieval instruction, and cosine similarity is computed as

$s(t,d)=\cos(e(t),e(d)).$

For each segment, the top-5 design logics are retained. DeepSeek-R1-0528 then selects the single most suitable logic among the five and strictly follows it to generate the question (Liu et al., 18 Aug 2025).

Post-processing is designed to reduce redundancy and benchmark contamination. The pipeline applies MinHash-based question deduplication and 13-gram decontamination against all evaluation benchmarks used in the paper, with punctuation ignored; any flagged question is discarded. For each retained question, Qwen3-235B-A22B-Thinking-2507-FP8 generates a corresponding long chain-of-thought solution, yielding paired question-response data for supervised fine-tuning (Liu et al., 18 Aug 2025).

The synthesis prompt enforces a simple JSON structure: $C$6 If the final answer is a single scalar or object, it is boxed as “The final answer is: \boxed{answer}.” This output convention is operational rather than stylistic; it supports automatic checking for single-result items (Liu et al., 18 Aug 2025).

4. Dataset composition, difficulty, and diversity

DLR-Book contains 3,040,620 questions across 75 disciplines. The appendix counts reported in the paper include Mathematics with 299,464 book questions, Biology with 200,078, Chemistry with 199,839, Physics with 199,771, Philosophy with 100,029, Business Administration with 99,739, Psychology with 99,502, and Law with 80,110 (Liu et al., 18 Aug 2025).

Its item-type composition is dominated by problem-solving and multiple-choice formats. Problem-solving constitutes 64.92%, multiple-choice 29.94%, proof 4.39%, and other 0.75%. Each example includes a self-contained question and a concise reference answer, while the long chain-of-thought solution is generated separately (Liu et al., 18 Aug 2025).

The scale of contextualization is reflected in length statistics. Average question length is 1,284.79 characters, and average response length is 18,090.06 characters. In uniformly sampled comparisons of 304,181 instances, these values exceed WebInstruct (Full), which reports 180.43 and 11,133.88, and NaturalReasoning, which reports 332.08 and 17,162.85 (Liu et al., 18 Aug 2025).

Difficulty is assessed by Qwen3-30B labeling into four bins. DLR-Book is distributed as Easy 0.27%, Medium 9.88%, Hard 35.18%, and Very Hard 54.66%; the combined Hard+Very Hard share is 89.84%. The corresponding Very Hard shares reported for baselines are 31.11% in NaturalReasoning and 3.59% in WebInstruct, while Easy questions in DLR-Book are negligible at 0.27% compared with 39.02% in WebInstruct (Liu et al., 18 Aug 2025).

Semantic diversity is measured on an embedding sample of $N=300{,}000$ with five metrics. For DLR-Book,

$C$0

$C$1

$C$2

$C$3

and

$C$4

These exceed the reported values for WebInstruct (Full) and NaturalReasoning across all five metrics (Liu et al., 18 Aug 2025). A plausible implication is that design-logic matching is contributing not only to difficulty but also to reduced near-duplication and broader semantic spread.

5. Supervised fine-tuning results

The paper evaluates DLR-Book through supervised fine-tuning on Qwen3-4B-Base and Qwen3-8B-Base. Training uses 6 epochs, batch size 64, learning rate $C$5, and cosine decay to 0. Evaluation uses temperature 0.6, top-k 20, top-p 0.95, max context 32,768, and zero-shot evaluation with benchmark-specific rollout counts (Liu et al., 18 Aug 2025).

With full-dataset fine-tuning, DLR-Book improves over the official thinking-mode Qwen3 models. For Qwen3-4B-SFT, MMLU Pass@1 rises from 82.87 to 84.73, MMLU-Pro from 69.34 to 73.03, GPQA-Diamond Pass@1 from 54.70 to 62.58, GPQA-Diamond CoT-SC from 58.08 to 68.69, GPQA-Main Pass@1 from 49.51 to 56.85, GPQA-Main CoT-SC from 51.12 to 61.16, and SuperGPQA Pass@1 from 43.30 to 45.86. For Qwen3-8B-SFT, MMLU Pass@1 rises from 85.85 to 87.53, MMLU-Pro from 73.62 to 76.69, GPQA-Diamond Pass@1 from 59.44 to 69.39, GPQA-Diamond CoT-SC from 60.61 to 73.74, GPQA-Main Pass@1 from 57.95 to 65.07, GPQA-Main CoT-SC from 59.38 to 68.30, and SuperGPQA Pass@1 from 47.52 to 50.57 (Liu et al., 18 Aug 2025).

Equal-volume comparisons strengthen the claim that the gains are not reducible to sample count alone. On Qwen3-8B-Base with 304,181 samples per dataset, DLR-Book surpasses WebInstruct (Full) and NaturalReasoning on MMLU, MMLU-Pro, GPQA-Diamond Pass@1, and GPQA-Diamond CoT-SC. The paper also reports ablations showing that removing coarse retrieval or fine LLM ranking lowers performance across benchmarks, while the full DESIGNER configuration is best overall (Liu et al., 18 Aug 2025).

This suggests that DLR-Book’s performance is tied to the interaction of source quality, design-logic retrieval, and logic-constrained synthesis rather than to scale alone.

6. Position in the broader reasoning-data landscape

DLR-Book belongs to a broader movement toward structurally guided reasoning corpora, but its methodology is distinct. LogicPro synthesizes complex logical reasoning data from LeetCode-style algorithm problems, standard Python solutions, and intermediate variable outputs, yielding a 540K synthesized dataset constructed solely from 2,360 algorithm problems (Jiang et al., 2024). “Simply Logical,” by contrast, is an interactive online educational resource organized around Prolog, SWISH-backed code blocks, and modular teaching materials rather than large-scale synthetic question generation (Flach et al., 2022). DLR-Book is therefore neither a program-trace-guided corpus nor an executable logic textbook; it is a doc-centric, multidisciplinary synthesis framework grounded in design logics.

The dataset also has important limitations. The book corpus and question bank are proprietary. The paper does not specify licensing terms or public download links in the text, and users are directed to the project page for release status, downloads, and licensing. The authors also note source biases, synthesis artifacts, and residual contamination risk despite 13-gram decontamination. Long rationales may contain subtle inconsistencies, and the dataset should not be treated as authoritative domain knowledge in high-stakes fields such as medicine or law without verification (Liu et al., 18 Aug 2025).

Future directions proposed in the paper include strengthening the formalization of design logics, improving matching via learned re-rankers or differentiable retrieval, extending disciplinary coverage and balance, integrating more robust factual verification during synthesis, and standardizing evaluation beyond the reported benchmarks (Liu et al., 18 Aug 2025). In that sense, DLR-Book is best understood as both a dataset and a methodological claim: that educator-like question design can be abstracted, retrieved, and reapplied at scale to generate reasoning problems that are broader, harder, and more diverse than prior doc-centric baselines.