Synthetic CoT-Supervised Data

• Synthetic CoT-supervised datasets are training resources that include intermediate reasoning—generated programmatically or by experts—alongside final answers.
• They employ diverse construction pipelines, ranging from algorithmic derivation to semi-automatic expert curation, to enhance model performance.
• These datasets support multi-step, curriculum-based learning, yielding improved results in complex domains like finance, medical imaging, and multimodal reasoning.

Synthetic CoT-supervised datasets are training resources in which the supervision target includes intermediate reasoning in addition to a final answer, while the reasoning itself is commonly programmatically generated, model-generated, distilled from existing corpora, transformed from structured annotations, or filtered from automatically produced traces rather than written entirely from scratch by human annotators. Recent work spans mixed curated and model-generated video reasoning, synthetic long-context finance reasoning, large distilled reasoning corpora, label-free biological trace filtering, semi-automatic medical workflow traces, and interleaved vision-language reasoning, indicating that “synthetic” in this literature usually refers to the provenance of the reasoning supervision rather than to wholly fabricated problems or wholly human-authored rationales (Zhang et al., 10 Jun 2025, Lin et al., 18 Feb 2025, Cai et al., 16 May 2025, Li et al., 22 Jul 2025).

1. Conceptual scope and taxonomy

The category is not uniform. Some works introduce standalone datasets whose primary contribution is a new reasoning corpus, while others contribute a supervision construction method or a training framework that depends on synthetic CoT data. LongFinanceQA is explicitly introduced as a synthetic dataset in finance whose labels contain intermediate reasoning before the final conclusion (Lin et al., 18 Feb 2025). OmniThought is presented as a large synthetic supervision resource built from existing reasoning problems plus teacher-generated and revalidated CoTs, with additional metadata describing reasoning verbosity and cognitive difficulty (Cai et al., 16 May 2025). Zebra-CoT is a mixed multimodal corpus containing interleaved text-image traces, with both curated real-world traces and synthetic examples generated from templates filled in by VLMs (Li et al., 22 Jul 2025).

Other resources occupy a mixed or semi-automatic position rather than a purely synthetic one. Video-CoT is best characterized as a mixed, partially synthetic Chain-of-Thought-supervised video dataset: its 192,000 spatiotemporal QA pairs are largely curated or derived from public datasets and processed annotations, while its 23,000 CoT samples are generated by a VLM and then filtered and manually reviewed (Zhang et al., 10 Jun 2025). Step-CoT is similarly not a fully synthetic clinical corpus; its cases and reports are real, its seven-step reasoning chains are generated from those reports using DeepSeek-R1 under a fixed schema, and all diagnostic procedures and data labels are reviewed and verified by a board-certified physician (Fan et al., 14 Mar 2026).

A separate class of papers does not primarily release a new dataset, but instead repurposes or transforms synthetic CoT supervision. “From Explicit CoT to Implicit CoT” is explicitly a training-method paper rather than a new standalone benchmark dataset; its central contribution is a staged transformation from explicit CoT targets to progressively reduced supervision, chiefly on synthetic multiplication and augmented GSM8K (Deng et al., 2024). SIM-CoT likewise does not introduce a new dataset, but depends centrally on GSM8K-Aug and converts explicit step sequences into latent-step supervision for implicit CoT models (Wei et al., 24 Sep 2025). CoT-Self-Instruct is best understood as a synthetic prompt-generation and curation method that uses CoT during data creation; its resulting datasets are primarily prompt-answer or preference data rather than explicit rationale-supervision corpora in the usual sense (Yu et al., 31 Jul 2025).

This variation has produced a recurring ambiguity in the term “CoT-annotated.” In Video-CoT, for example, “CoT-annotated” does not mean fully human-written reasoning; it denotes Qwen2.5-VL-72B-Instruct-generated traces wrapped in > and </think> tags and then filtered and reviewed (Zhang et al., 10 Jun 2025). A plausible implication is that any taxonomy of synthetic CoT datasets must distinguish at least between fully programmatic traces, teacher-generated traces over existing labels, semi-automatic expert-curated traces, and methods that use CoT to create training prompts without supervising the final model on exposed rationales.

2. Provenance of reasoning supervision

Recent work uses several recurring sources for synthetic reasoning traces.

Construction mode	Representative resources	Trace provenance
Programmatic derivation	(Deng et al., 2024, Kothapalli et al., 21 Feb 2025)	Long multiplication steps; DAG-plus-MLP token chains
Teacher- or VLM-generated traces	(Zhang et al., 10 Jun 2025, Cai et al., 16 May 2025, Li et al., 22 Jul 2025)	Qwen2.5-VL-72B-Instruct; DeepSeek-R1 and QwQ-32B; Gemini-2.5 and GPT-4.1
Agentic generation over source documents	(Lin et al., 18 Feb 2025, Zhao et al., 17 Jul 2025)	PAI property extraction, retrieval, summarization; MKE plus SCR
Semi-automatic expert-curated traces	(Fan et al., 14 Mar 2026, Zhu et al., 28 May 2026)	Report-derived step chains with physician review; Gemini drafts refined by human experts
Label-free selection of synthetic traces	(Stoisser et al., 7 Oct 2025)	Gemini 2.5 Pro traces retained by low uncertainty

The most direct form of synthetic CoT supervision is programmatic derivation. In the multiplication setting of “From Explicit CoT to Implicit CoT,” examples are generated algorithmically from long multiplication, with reversed inputs, reversed intermediate partial products, and reversed outputs; for $12 \times 34$, the paper gives input 2 1 * 4 3, CoT 8 4 + 0 6 3, and output 8 0 4 (Deng et al., 2024). CoT-ICL Lab generalizes this idea to a synthetic tokenized setting in which each chain token $y_c$ is produced by a causal selector $g_c$ and a token processor $h_c$, yielding programmable rationale length, dependency width, and function diversity (Kothapalli et al., 21 Feb 2025).

Teacher-generated traces dominate many large corpora. Video-CoT constructs a pipeline for CoT data generation using Qwen2.5-VL-72B-Instruct over video inputs, prompts, and examples, producing structured reasoning outputs in <think> tags (Zhang et al., 10 Jun 2025). OmniThought uses DeepSeek-R1 and QwQ-32B as teacher large reasoning models to produce multiple CoTs per problem and then revalidates them with an LLM-as-a-judge pipeline (Cai et al., 16 May 2025). Zebra-CoT uses Gemini-2.5 and GPT-4.1 to fill placeholders in templated multimodal traces or to enhance raw reasoning traces into logically coherent interleaved text-image narratives (Li et al., 22 Jul 2025).

Agentic generation pipelines move beyond one-pass prompting. LongFinanceQA uses Property-driven Agentic Inference, a three-stage framework of property extraction, property-based retrieval, and summarization, to create reasoning-augmented answers from annual reports (Lin et al., 18 Feb 2025). Agentar-DeepFinance-300K first expands the QA space through Multi-perspective Knowledge Extraction with Q2A, A2Q, and T2Q branches, then applies Self-Corrective Rewriting so that failed initial CoT samples can be diagnostically reflected on, rewritten, and reverified against golden answers instead of discarded (Zhao et al., 17 Jul 2025). D-SCoRE applies a three-stage document-centric generation pipeline from arbitrary text: QA generation, QA quality control with KEEP/DELETE/TYPEFIX decisions, and counterfactual material generation with three distractors per QA pair (Zhou et al., 2 Aug 2025).

Semi-automatic expert-curated pipelines typically start from natural source data and synthesize only the reasoning layer. Step-CoT converts real chest X-ray reports into a seven-step diagnostic chain aligned to clinical workflow and then validates the outputs clinically (Fan et al., 14 Mar 2026). FakeClue++ combines real/fake labels, artifact localization, explanatory checklists, and physical-law-oriented descriptions, then uses LLMs to rephrase those interpretable annotations into diverse SFT formats and multi-turn dialogue variants (Zhu et al., 28 May 2026).

Label-free filtering forms a distinct provenance regime. In biological perturbation prediction, one greedy trace and eight sampled traces are generated for each unlabeled example, uncertainty is estimated from semantic consistency plus predictive perplexity through CoCoA, and only low-uncertainty traces are retained, with per-class filtering to preserve coverage across outcomes (Stoisser et al., 7 Oct 2025). This suggests a synthetic CoT corpus need not depend on ground-truth labels if confidence signals are used as a proxy for supervision quality.

3. Construction pipelines and dataset engineering

Synthetic CoT datasets are usually assembled as layered pipelines rather than as a single annotation step. Video-CoT illustrates this clearly. Its base supervision consists of 192,000 fine-grained spatiotemporal QA pairs organized into six subtasks—TVL, VC, SVG, STVG, SRR, and TVR—built from publicly available datasets such as VTimeLLM-stage2, MSR-VTT, MSVD, WebVid, HCSTVG-V1, and VidSTG. For grounding and reference tasks, the construction procedure selects query targets, ensures consecutive presence across frames, discards objects that appear for less than 2 seconds, derives start and end timestamps and pixel locations, maintains spatial relationships, and then creates spatiotemporal relational QA data. Only the 23,000 CoT samples form the reasoning-augmented subset, generated by a VLM and retained after task-specific thresholding plus manual expert review (Zhang et al., 10 Jun 2025).

LongFinanceQA uses a different engineering strategy because its source material is long-form and bilingual. It is built from 6,911 annual reports in English and Chinese, collected from SEC-10-K and cninfo, restricted to reports published before 2022 and filtered to a token range of 20K to 80K. Its generation pipeline samples financial metrics and report metadata, creates single-source or multi-source questions, and then uses PAI to produce properties, retrieval-based sub-findings, and final summarized conclusions. The resulting dataset contains 46,457 QA pairs with CoT reasoning, with 45.6% single-source and 54.4% multi-source composition (Lin et al., 18 Feb 2025).

OmniThought scales the same general pattern to a broad reasoning corpus. It aggregates problems from OpenThoughts2-1M and DeepMath-103K, generates multiple CoTs with DeepSeek-R1 and QwQ-32B, validates reasoning and solution separately, and computes Reasoning Verbosity and Cognitive Difficulty scores. The release contains 2.059 million CoT processes for 708K reasoning problems, with at least two validated correct CoTs per problem. Unlike “perfect-only” corpora, it retains lower-quality CoTs unless they produce an incorrect answer, because answer-correct but reasoning-flawed traces can still support DPO or reward-model training (Cai et al., 16 May 2025).

Several later pipelines explicitly optimize the knowledge space before rationale generation. Agentar-DeepFinance-300K uses Q2A direct curation, A2Q counterfactual augmentation, and T2Q CoT knowledge mining, then samples multiple CoTs per QA pair and verifies answer consistency with a lightweight model; if the answer is wrong, SCR adds a reflection chain $T_{rf}$, a rewritten chain $T_{rw}$, and iterative re-verification (Zhao et al., 17 Jul 2025). D-SCoRE operates over arbitrary textual sources by segmenting documents, generating explicit and implicit questions, validating source fidelity and question type, and then generating four-option answer materials with one correct answer and three semantically close but factually wrong distractors (Zhou et al., 2 Aug 2025). Zebra-CoT combines sourced multimodal problems, synthetic task generation, rendered intermediate states, and VLM-written narratives, then filters invalid image-placeholder mappings to ensure that interleaved text-image traces can be automatically parsed (Li et al., 22 Jul 2025).

These pipelines reveal a common engineering pattern. The underlying tasks are often not invented from nothing; instead, the corpus is expanded by curating source datasets, transforming their annotations, generating candidate rationales with strong teachers or programs, and then filtering or repairing those candidates. A plausible implication is that the decisive design variable is not “synthetic versus real” in isolation, but the exact layer at which synthetic reasoning is injected.

4. Supervision formats and objective functions

The defining formal property of a synthetic CoT-supervised dataset is that the target contains reasoning as well as a final answer. LongFinanceQA states this contrast directly by setting ordinary QA as $p_{\theta}(\mathbf{A}\mid \mathbf{X}, \mathbf{Q})$ and reasoning-augmented supervision as $$p_{\theta}(\mathbf{R}, \mathbf{A}\mid \mathbf{X}, \mathbf{Q})$$, with dataset construction factorized as $$p_{\theta}(\mathbf{R}\mid \mathbf{X}, \mathbf{Q}) \cdot p_{\theta}(\mathbf{A}\mid \mathbf{X}, \mathbf{Q}, \mathbf{R})$$. Its rationale is not a free-form essay but a mixed-format sequence containing extracted properties, retrieval-stage sub-answers, and a final summarized conclusion (Lin et al., 18 Feb 2025).

Video-CoT makes the same distinction in multimodal form. It defines Answer-Supervised Fine-Tuning as $(V,Q)\rightarrow A$ and CoT-Supervised Fine-Tuning as $y_c$0, where $y_c$1 is the video, $y_c$2 is the question, $y_c$3 is the answer, and $y_c$4 is the reasoning chain. The paper describes curriculum learning from simple 2-step reasoning scenarios to sophisticated 5-step chains, and the CoT traces are structured reasoning outputs wrapped in <think> tags (Zhang et al., 10 Jun 2025).

Other works change the granularity rather than the basic rationale-plus-answer design. SIM-CoT aligns each latent token $y_c$5 with one explicit reasoning step $y_c$6 and trains an auxiliary decoder under

$y_c$7

while the base LLM predicts the final answer $y_c$8 from the latent sequence. Its training source, GSM8K-Aug, removes natural-language chains and preserves structured mathematical expressions such as <\<12*3=36>><\<9*2=18>><\<17*2=34>><\<36+18+34=88>>, so the step targets are symbolic expression chains rather than free-form rationales (Wei et al., 24 Sep 2025).

“From Explicit CoT to Implicit CoT” treats rationale visibility as a curriculum variable. Starting from explicit CoT supervision

$y_c$9

it progressively removes the first $g_c$0 rationale tokens and trains on

$g_c$1

The same examples therefore appear in three supervision regimes: full explicit CoT, partial CoT, and no visible CoT (Deng et al., 2024).

Multimodal work further diversifies the serialization format. Zebra-CoT uses a trace schema with QUESTION, REASONING [TRACE](https://www.emergentmind.com/topics/trace), THOUGHT n, interleaved image placeholders such as <image_start>[reasoning_image_1]<image_end>, and a FINAL ANSWER, while Bagel-Zebra-CoT training additionally wraps textual reasoning in <think> ... and answers in <answer> ... </answer> (Li et al., 22 Jul 2025). Step-CoT instead structures each case as JSON with step, question, options, answer, and reasoning, organized around a seven-step chest-radiograph workflow (Fan et al., 14 Mar 2026). FakeVLM-R1 constrains the reasoning format during RL with <think> and <answer> tags plus a dialectical pattern [Clue] \rightarrow [Why real/fake] \rightarrow [If fake/real] (Zhu et al., 28 May 2026).

These differences matter because “CoT supervision” may denote natural-language rationale tokens, symbolic steps, structured JSON sub-decisions, interleaved image-text traces, or latent-step reconstruction targets. The commonality lies in exposing intermediate structure to the learner, not in any single surface format.

5. Modalities, domains, and benchmark roles

Synthetic CoT supervision now appears across a wide range of modalities and scales.

Resource	Domain and modality	Reported scale
Video-CoT (Zhang et al., 10 Jun 2025)	Video spatiotemporal understanding	192,000 QA pairs; 23,000 CoT samples; 4,500 benchmark QA pairs
LongFinanceQA (Lin et al., 18 Feb 2025)	Long-context bilingual finance	46,457 QA pairs with CoT over 6,911 reports
OmniThought (Cai et al., 16 May 2025)	Mathematics, coding, science, puzzles	2.059M CoTs for 708K problems
Agentar-DeepFinance-300K (Zhao et al., 17 Jul 2025)	Financial reasoning	300K
Zebra-CoT (Li et al., 22 Jul 2025)	Interleaved text-image reasoning	182,384 samples
Step-CoT (Fan et al., 14 Mar 2026)	Medical VQA on chest X-ray	more than 10K cases and 70K VQA pairs

The domain range extends beyond text-only reasoning. Video-CoT targets temporal boundaries, object trajectories, spatial relationships, and video-grounded descriptions (Zhang et al., 10 Jun 2025). Zebra-CoT spans scientific reasoning, 2D visual reasoning, 3D visual reasoning, and visual logic or strategic games, with tasks such as geometry, graph algorithms, jigsaw, embodied planning, chess, ciphers, and Connect Four (Li et al., 22 Jul 2025). Step-CoT aligns medical VQA to a seven-step radiological diagnostic workflow rather than to free-form rationalization (Fan et al., 14 Mar 2026). FakeClue++ extends the idea to multimodal synthetic-image detection by combining artifact explanations with physical-law-based authenticity anchors for real images (Zhu et al., 28 May 2026).

Benchmark roles also vary. Some datasets are primarily training corpora evaluated on external benchmarks. LongFinanceQA is used to fine-tune LongPAI and evaluated mainly on Loong and $g_c$2Bench rather than on an internal benchmark split (Lin et al., 18 Feb 2025). OmniThought likewise functions as a training pool whose effectiveness is tested on AIME2024, MATH500, GPQA-D, and LiveCodeBench V2 (Cai et al., 16 May 2025). Other resources bundle a benchmark with the training corpus: Video-CoT provides 750 benchmark items per task across six tasks, totaling 4,500 video QA pairs (Zhang et al., 10 Jun 2025), and FakeClue++ reserves a 4,000-image benchmark evenly distributed across real and fake samples in four categories (Zhu et al., 28 May 2026).

Some datasets are deliberately filtered down before training. In biological perturbation prediction, the main synthetic pool contains 48k traces, but the main retained training set is only the top 10% per class, yielding 4.8k examples (Stoisser et al., 7 Oct 2025). In Step-CoT, the filtered clinical corpus contains 10,068 frontal-view studies and is split 70%/15%/15% into 7,043 train, 1,507 validation, and 1,518 test cases (Fan et al., 14 Mar 2026). This suggests that synthetic CoT corpora are often best understood as generated candidate pools plus a retention policy, rather than as a monolithic static dataset.

6. Empirical effects, limitations, and recurring controversies

The main empirical result across this literature is that rationale-bearing supervision often outperforms answer-only supervision, but the magnitude depends strongly on task structure. Video-CoT reports a baseline TVL temporal IoU of 4.4 for Qwen2.5-VL 3B, 8.3 after answer-only fine-tuning, and 19.7 after CoT-SFT; the same CoT-SFT run also reports VC MENTOR 15.4, SRR EM 16.9, TVR MENTOR 21.1, SVG sIoU 17.0, STVG tIoU 14.1, and STVG sIoU 9.2 (Zhang et al., 10 Jun 2025). LongPAI trained on LongFinanceQA with reasoning-augmented outputs markedly outperforms the answer-only LongPAI$g_c$3 on long contexts, with Set4 $g_c$4–$g_c$5 performance of 60.36 versus 22.14 on the financial subset of Loong (Lin et al., 18 Feb 2025). In multiplication, Stepwise Internalization raises GPT-2 Small from 0.00 on $g_c$6 and $g_c$7 under No CoT training to 0.95 and 0.99 under ICoT-SI, even though the deployed model no longer emits the intermediate steps (Deng et al., 2024). SIM-CoT reports gains over prior implicit-CoT baselines, including +8.2 on GPT-2 over Coconut and +3.0 on LLaMA-3.1 8B over CODI (Wei et al., 24 Sep 2025). In biology, supervised fine-tuning on the 10% uncertainty-filtered subset yields 0.57 accuracy, compared with 0.52 for all synthetic data and 0.48 for a random 10% subset (Stoisser et al., 7 Oct 2025). Zebra-CoT improves Anole-7B from 4.2% to 16.9% on its held-out test set and reports up to +13% gains on standard VLM benchmarks (Li et al., 22 Jul 2025).

At the same time, the literature repeatedly warns that answer correctness and rationale quality are not identical. OmniThought explicitly retains answer-correct traces even when reasoning is imperfect, because such traces may still be useful for DPO or reward-model training (Cai et al., 16 May 2025). Agentar-DeepFinance-300K verifies answers against golden answers, but the paper itself notes that answer verification guarantees final correctness more than rationale faithfulness (Zhao et al., 17 Jul 2025). The label-free biology pipeline acknowledges a classic failure mode of self-judging systems: if the teacher is confidently wrong, low-uncertainty filtering can still retain a bad trace (Stoisser et al., 7 Oct 2025).

Documentation quality is another recurring controversy. Video-CoT does not provide exact CoT prompts, decoding settings, candidate counts, self-consistency details, inter-annotator agreement for review, or exact task-wise thresholds (Zhang et al., 10 Jun 2025). LongFinanceQA reports no human evaluation of extracted properties, sub-answers, or rationale faithfulness (Lin et al., 18 Feb 2025). Step-CoT reports physician verification but არა systematic annotation-agreement statistics or a large-scale faithfulness audit (Fan et al., 14 Mar 2026). Zebra-CoT emphasizes logical coherence and automatic filtering, yet does not report a dedicated human study of rationale quality or a quantitative decomposition of its synthetic and naturally sourced fractions (Li et al., 22 Jul 2025). A plausible implication is that many synthetic CoT corpora are stronger as supervision resources than as audited explanations.

Teacher bias is also structural. OmniThought relies on DeepSeek-R1 and QwQ-32B for generation, validation, and scoring (Cai et al., 16 May 2025). SIM-CoT depends entirely on GSM8K-Aug, which is expanded partly through GPT-4 synthetic generation (Wei et al., 24 Sep 2025). FakeClue++ combines Gemini-generated drafts, LLM rewrites, and a critic-driven RL stage, making its final reasoning behavior inseparable from the preferences and failure modes of those models (Zhu et al., 28 May 2026). This suggests that synthetic CoT datasets do not simply encode task knowledge; they also encode a teacher’s preferred decomposition style, verbosity, and implicit inductive biases.

For that reason, the most precise contemporary understanding of a synthetic CoT-supervised dataset is neither “a dataset with explanations” nor “a dataset fabricated by an LLM.” It is a training resource in which intermediate reasoning becomes an explicit supervision object through programmatic derivation, teacher generation, structured transformation, or filtered distillation, usually layered on top of pre-existing tasks or annotations. The field’s central question is therefore not whether the traces are synthetic, but how their provenance, filtering, granularity, and alignment to the target learner affect transfer, stability, interpretability, and error modes.