MCoT-Instruct-287K: Dual-Definition Datasets

• The paper in MiCoTA presents MCoT-Instruct-287K as an intermediate chain-of-thought corpus that bridges the learnability gap for small language models through a three-stage teacher assistant merging process.
• In Corvid, MCoT-Instruct-287K is a multimodal dataset of 287K image-text conversations, curated via rewriting, scoring, and filtering to ensure coherent and standardized chain-of-thought instructions.
• Both implementations underscore the dataset’s context-dependent meaning, with distinct methodologies and empirical gains that caution against conflating similarly named resources.

Searching arXiv for papers mentioning “MCoT-Instruct-287K” and related resources. MCoT-Instruct-287K is not a single uniformly defined resource in the arXiv literature. In 2025, the label is used for two distinct 287,000-example instruction-tuning datasets with different modalities, construction pipelines, and training roles. In "MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants," MCoT-Instruct-287K is the released, instruct-style version of the synthetic mid-length chain-of-thought corpus denoted $D_{\mathrm{MiCoTA}}$, designed for small LLM distillation. In "Corvid: Improving Multimodal LLMs Towards Chain-of-Thought Reasoning," MCoT-Instruct-287K is a high-quality multimodal CoT instruction-following dataset composed of 287,000 single-turn image-text conversations. Earlier papers with similar naming do not define this resource: the multilingual mCoT paper centers on mCoT-MATH rather than MCoT-Instruct-287K, and the Markov Chain of Thought paper introduces MCoTInstruct of 82k Markov chains rather than a 287K variant (Ding et al., 2 Jul 2025, Jiang et al., 10 Jul 2025, Lai et al., 2024, Yang et al., 2024).

1. Terminological scope and disambiguation

The term "MCoT-Instruct-287K" has a paper-specific meaning. In MiCoTA, it denotes the public release name for the instruct-style version of $D_{\mathrm{MiCoTA}}$, the intermediate-length CoT corpus used to train small LLMs. In Corvid, it denotes a multimodal dataset refined and standardized from public reasoning sources. The overlap is therefore nominal rather than definitional. This suggests that the identifier is not stable across papers and must be interpreted in the context of the specific training framework in which it appears (Ding et al., 2 Jul 2025, Jiang et al., 10 Jul 2025).

Paper	What the name denotes	Modality
MiCoTA	Released, instruct-style version of $D_{\mathrm{MiCoTA}}$	Primarily English text reasoning
Corvid	High-quality multimodal CoT instruction-following dataset	Image + text

A common misconception is to treat MCoT-Instruct-287K as a single canonical benchmark or corpus. The cited papers do not support that reading. Instead, they document two different datasets that happen to share the same surface name and the same reported size of 287,000 examples.

2. MCoT-Instruct-287K in MiCoTA

Within MiCoTA, MCoT-Instruct-287K is the released, instruct-style realization of $D_{\mathrm{MiCoTA}}$, the synthetic corpus generated by the Mid-CoT Teacher Assistant after model merging. Its purpose is to bridge what the paper calls the "SLMs Learnability Gap": small LLMs degrade when trained directly on long-form CoT from strong teachers. The framework targets two coupled gaps, a capacity gap between strong teachers such as 32B models and small students in the 1.5B-7B range, and a length gap between very long CoTs and what small models can effectively absorb (Ding et al., 2 Jul 2025).

The construction pipeline has three stages. First, the strong teacher R1-Distill-Qwen-32B generates long CoTs under greedy decoding with a maximum length of 16K tokens, and entries with incorrect final answers or overlength responses are filtered out. Second, Qwen2.5-14B-Instruct is fine-tuned on this long-CoT set and then merged in parameter space with Dare and TIES sign consensus to create a Mid-CoT Teacher Assistant. The paper characterizes this assistant as "half-size, half-length": it inherits the reasoning tendencies of the strong teacher while producing approximately half-length outputs. Third, the merged teacher assistant is prompted to synthesize intermediate-length CoTs, again under greedy decoding and length control, retaining only samples with correct final answers and within the length bound. The resulting corpus is packaged as MCoT-Instruct-287K (Ding et al., 2 Jul 2025).

The released corpus contains 287,000 examples, is primarily English, and is described as predominantly mathematics reasoning spanning levels reflected in the evaluations, including AIME-style, AMC-style, Olympiad-style problem solving and GSM8K-style math word problems, plus general step-by-step reasoning instructions consistent with Instruct formatting. Its schema is instruct/chat style, with fields for the instruction, assistant reasoning in intermediate-length CoT form, and a final answer. Basic cleaning and de-duplication are applied, content formatting is standardized, and safety or alignment checks follow the base Qwen Instruct conventions, with no additional alignment steps introduced in the paper (Ding et al., 2 Jul 2025).

Student models are trained by standard supervised fine-tuning on this intermediate CoT distribution. The objective is

$$L_{\mathrm{SFT}} = - \sum_{t=1}^{T} \log q_\theta(y_t \mid x, y_{<t}),$$

and MiCoTA sets the total loss to $L_{\mathrm{total}} = L_{\mathrm{SFT}}$. The paper explicitly notes that no KL regularization to teacher logits is added and that no explicit curriculum scheduling is used; the curriculum effect is instead realized implicitly through replacing long traces with half-length teacher-assistant data (Ding et al., 2 Jul 2025).

3. MCoT-Instruct-287K in Corvid

Within Corvid, MCoT-Instruct-287K is a high-quality, multimodal chain-of-thought instruction-following dataset refined and standardized from diverse public reasoning sources. It contains 287,000 single-turn conversation instances with image context, question, options when applicable, a step-by-step textual CoT, and a final answer. The paper motivates the dataset by arguing that manually created CoTs are often brief and sometimes logically incoherent, whereas AI-generated CoTs can be detailed but noisy, erroneous, and duplicated; the dataset is intended to resolve these weaknesses through rewriting, scoring, filtering, and standardization (Jiang et al., 10 Jul 2025).

The raw pool comprises 292K instances from ten public datasets spanning general visual reasoning, knowledge-intensive visual question answering, visual commonsense reasoning, science problem-solving, geometric reasoning, numerical reasoning, and mathematical reasoning. After rewriting, scoring, and filtering, 287K remain.

Source	Provenance type	Size
GPT-VQA	AI-assisted	26K
A-OKVQA	Manual	18K
VCR	Manual	84K
M$^3$CoT	Manual	9K
SQA-IMG train split	Manual	8K
ArxivQA	AI-assisted	54K
GeomVerse	Manual	9K
R-CoT	AI-assisted	53K
GeoQA	Manual	7K
TabMWP	Manual	24K

The curation pipeline has two main components. For manual-origin CoTs, GPT-4o is used with a specialized prompt to rewrite rationales for coherence, logical consistency, detail, and adherence to a standardized reasoning-then-answering structure. For all sources, GPT-based scoring evaluates CoTs along Faithfulness, Relevance, and Completeness. The higher-scoring CoT between the raw and rewritten versions is retained, and instances with an overall score below 0.6 are filtered out. The standardized output format is single-turn and reasoning-first-then-answering; the paper does not specify a JSON schema or special tokens, but it does enforce clarity, conciseness, faithfulness, and explicit final answers (Jiang et al., 10 Jul 2025).

The paper also makes an explicit leakage-avoidance claim for ScienceQA: only the training split from SQA-IMG is used during curation, and no benchmark test or validation instances are included in MCoT-Instruct.

4. Relation to earlier similarly named resources

The 2024 paper "mCoT: Multilingual Instruction Tuning for Reasoning Consistency in LLMs" does not introduce or describe a resource named MCoT-Instruct-287K. Its central training corpus is mCoT-MATH, described as approximately 6.3 million multilingual CoT samples across 11 languages, created by translating about 580,000 English CoT math samples into ten target languages and retaining the English originals. The paper states that mCoT-MATH is the main instruction-tuning corpus and that a dataset named "MCoT-Instruct-287K" is not described in the publication (Lai et al., 2024).

The 2024 paper "Markov Chain of Thought for Efficient Mathematical Reasoning" introduces a different resource, MCoTInstruct, tied to the Markov Chain of Thought paradigm rather than to either MiCoTA or Corvid. That corpus is reported as 82k Markov chains totaling around 160k step-wise entries, with 29k GPT-4 generated reduced questions. Its data format is step-level, using tuples such as $(q_t, s_t, q_{t+1})$ or $(q_T, s_T, a)$, where each $s_t$ includes text, Python code, and observation, and where reduction rewrites the remaining problem into a standalone sub-question (Yang et al., 2024).

The naming proximity among mCoT-MATH, MCoTInstruct, and MCoT-Instruct-287K can obscure substantial methodological differences. The multilingual mCoT work studies cross-lingual reasoning consistency; the Markov Chain of Thought work studies derive-then-reduce efficiency for mathematical reasoning; MiCoTA studies intermediate-length CoT distillation for small LLMs; and Corvid studies multimodal CoT instruction tuning for multimodal LLMs. Treating these as interchangeable resources is therefore incorrect.

5. Training roles and empirical outcomes

In MiCoTA, MCoT-Instruct-287K is the direct student-training corpus. The paper reports that models trained on this mid-length CoT distribution consistently outperform baselines distilled from strong-teacher long CoT and from Half-size CoT generated by the unmerged teacher assistant. Averaged across AIME 2024, AMC 2023, OlympiadBench, MATH-500, and GSM8K, Qwen2.5-7B-Instruct improves from 45.89 to 49.36, a gain of 3.47 points over the Instruct baseline, and Qwen2.5-3B-Instruct improves from 39.36 to 43.29, a gain of 3.93 points. The paper further reports adapted Bits-Per-Character values indicating better distributional alignment for MiCoTA data than for Strong-Teacher-CoT and Half-size CoT, with 0.13 for 7B, 0.13 for 3B, and 0.14 for 1.5B. Lower BPC is interpreted as a better match between model and data distributions (Ding et al., 2 Jul 2025).

In Corvid, MCoT-Instruct-287K is one of the core datasets used to build chain-of-thought capability across two stages of training. In Stage 2, Corvid-1M includes 124K MCoT-Instruct samples within a 1M mixed SFT set in which CoT-formatted reasoning constitutes 20.2% and reasoning data overall constitutes approximately 51.5%. In Stage 3, the pure-CoT set o1-320K includes 163K MCoT-Instruct samples, 137K MAVIS-Instruct samples, and 20K CamelMath samples. The paper reports that replacing high-quality CoTs with raw rationales reduces average performance from 55.6 to 51.2, and removing CoTs entirely reduces it to 45.7, across MMStar, MMMU, AI2D, MathVista, MathVerse, WeMath, MathVision, and DynaMath. It also reports benchmark gains from Corvid-base-8B to Corvid-o1-8B, including MathVista 64.8 to 72.0, MathVerse 34.8 to 40.1, WeMath 54.0 to 59.8, MathVision 26.8 to 30.1, DynaMath 24.5 to 33.2, MMStar 62.4 to 65.2, MMMU 57.4 to 59.7, and AI2D 82.8 to 85.0. The paper states that these gains are consistent with the added pure-CoT instruction tuning set, where MCoT-Instruct is the largest multimodal CoT component (Jiang et al., 10 Jul 2025).

The two usages therefore occupy different points in the training stack. In MiCoTA, the dataset is the central distillation medium for text-based small-model reasoning. In Corvid, it is a curated multimodal CoT component embedded within larger mixed and pure-CoT training mixtures for an 8B multimodal model.

6. Availability, licensing, and limitations

For MiCoTA, code and materials are provided through the repository at https://github.com/OPPO-PersonalAI/MiCoTA, and the paper states that MCoT-Instruct-287K is publicly released via that repository. The manuscript does not specify a license in text, instructing readers to consult the repository for license terms, download links, and usage guidelines. The paper also notes several limitations: the data is mostly math-centric, broader domains such as code, NLU, and multimodal settings remain to be validated, and stronger correctness verification, length-regularized objectives, or verifier-guided filtering may further improve generalization (Ding et al., 2 Jul 2025).

For Corvid, the paper provides a project page at https://mm-vl.github.io/corvid but does not state explicit licensing terms for MCoT-Instruct-287K or provide a direct dataset download link in the text. It also does not specify whether images are redistributed or merely referenced from source datasets. The limitations section notes that residual bias and noise may persist because the source pool contains both manually created and AI-generated CoTs; the paper further reports no explicit statistics on CoT length, difficulty labeling, or multilingual composition. It recommends additional verification beyond GPT scoring for math and science subsets and emphasizes respecting the licenses of the original source datasets (Jiang et al., 10 Jul 2025).

Taken together, these limitations reinforce the central interpretive point: MCoT-Instruct-287K is a context-dependent label rather than a single standardized benchmark. In MiCoTA it denotes a primarily English, mid-length textual CoT distillation corpus; in Corvid it denotes a filtered and standardized multimodal CoT instruction dataset. Earlier mCoT and MCoTInstruct papers use related acronyms for different resources and should not be conflated with either 287K dataset (Lai et al., 2024, Yang et al., 2024).