Long Chain-of-Thought SFT: Structured Reasoning

• Long Chain-of-Thought SFT is a supervised fine-tuning regime that trains models to generate extended reasoning chains before providing final answers.
• It leverages structured, multi-step supervision from teacher models and synthetic annotation pipelines to improve long-context and domain-specific reasoning.
• However, L-CoT SFT can introduce verbosity and overthinking, necessitating careful data curation, curriculum design, and tuning to balance efficiency and accuracy.

Long Chain-of-Thought Supervised Fine-Tuning (L-CoT SFT) is a post-training regime in which a LLM is optimized to generate extended intermediate reasoning trajectories before producing a final answer. In the long-context formulation, the supervised target is commonly written as $$p_{\theta}(\mathbf{R}, \mathbf{A}|\mathbf{X},\mathbf{Q}) = p_{\theta}(\mathbf{R}|\mathbf{X},\mathbf{Q}) \cdot p_{\theta}(\mathbf{A}|\mathbf{X},\mathbf{Q},\mathbf{R})$$, where $\mathbf{R}$ denotes the reasoning trace and $\mathbf{A}$ the answer (Lin et al., 18 Feb 2025). The method has become a pivotal phase in building large reasoning models because it can directly transfer explicit stepwise behavior, improve downstream reinforcement learning efficiency, and support long-context or domain-specific reasoning; however, its benefits depend strongly on the structure, source, and length of the supervision, and it can introduce overthinking, degraded generalization, or architecture-specific failures when applied naively (Yeo et al., 5 Feb 2025).

1. Conceptual role and problem setting

L-CoT SFT differs from answer-only fine-tuning by supervising the model on intermediate reasoning tokens as first-class outputs. In practice, the training tuples contain a prompt and a long rationale, often distilled from a stronger reasoning model or produced by a synthetic annotation pipeline, so that the student model learns not merely the terminal answer distribution but a trajectory of decomposition, verification, and synthesis (Yu et al., 6 May 2025). For long-context tasks, this supervision is intended to connect evidence scattered across large inputs and to force explicit analytic steps before conclusion formation (Lin et al., 18 Feb 2025).

Within the current post-training stack, L-CoT SFT is often treated as the most direct initial mechanism for eliciting long-form reasoning. A comparative study of long-CoT emergence reported that SFT is not strictly necessary, because reinforcement learning can in principle discover long trajectories, but SFT simplifies training and improves efficiency; it also raises the upper performance limit relative to short-CoT SFT, whose gains saturate earlier (Yeo et al., 5 Feb 2025). In lightweight multimodal models, the same logic appears in stronger form: a long-CoT SFT stage was identified as a critical prerequisite for developing reasoning capabilities that later RL can amplify (Ou, 3 Sep 2025).

A common misconception is that L-CoT SFT is simply “more text supervision.” The literature treats it instead as supervision over a particular computational style: multi-step, explicit, and often reflective reasoning. That emphasis explains both its successes and its liabilities. The same long-form traces that improve hard-problem reasoning can also induce verbosity, redundant branching, or local attention bias, depending on model scale and architecture (Chen et al., 10 Jul 2025).

2. Supervision sources, data construction, and curation

The quality of L-CoT SFT depends heavily on how reasoning traces are sourced. Distilling “emergent” long CoT from strong teacher models is reported to be more effective than constructing long traces from short-CoT models via action prompting or orchestration (Yeo et al., 5 Feb 2025). This result is consistent with the broader observation that the student inherits not only answers but also the teacher’s reasoning topology. In a controlled comparison using verified trajectories from \texttt{DeepSeek-R1-0528} and \texttt{gpt-oss-120b} on identical problem sets, models trained on the latter generalized better because its trajectories were more convergent and deductive, whereas \texttt{DeepSeek-R1-0528} favored divergent, branch-heavy exploration (Li et al., 2 Apr 2026).

Synthetic pipelines have been developed to generate long-CoT corpora when human annotation is infeasible. In long-context finance, LongFinanceQA combines 6,911 reports with 46,457 QA pairs, and its Property-driven Agentic Inference pipeline decomposes annotation into property extraction, retrieval, and summarization so that answers contain both intermediate reasoning and a final conclusion (Lin et al., 18 Feb 2025). A related but architecturally distinct strategy appears in RELAY, where a looped Transformer is trained with iteration-wise supervision aligned to CoT steps, then used to generate accurate reasoning chains beyond the training length for subsequent auto-regressive fine-tuning (Yu et al., 12 Feb 2025).

Curriculum design is another recurring curation mechanism. Light-R1 stages SFT from a large filtered set of 76k math problems to a much smaller 3k subset of harder problems, followed by DPO, and reports that the harder second-stage data alone can substantially improve already strong distilled models (Wen et al., 13 Mar 2025). A different curation result comes from branch filtering rather than difficulty filtering: removing frequently branching trajectories from \texttt{DeepSeek-R1-0528} data improves reasoning performance by up to 5.1% on AIME25, 5.5% on BeyondAIME, and on average 3.6% on five benchmarks (Li et al., 2 Apr 2026). This suggests that L-CoT supervision quality is governed not only by correctness or length, but by the structural efficiency of the reasoning path.

3. Optimization behavior and generalization dynamics

A central finding in recent work is that lower SFT training loss does not necessarily imply better reasoning generalization. The controlled study of \texttt{DeepSeek-R1-0528} versus \texttt{gpt-oss-120b} found a paradox: the former yielded much lower SFT training loss yet consistently worse benchmark performance. Token-level analysis attributed this to dilution by many near-zero-loss tokens—routine or exploratory text—while the high-loss transition tokens that carry logical progress remained similarly difficult across sources (Li et al., 2 Apr 2026). In other words, average loss can be dominated by easy tokens rather than by the tokens that matter most for reasoning transfer.

This observation has motivated objective-level modifications. VCORE reformulates CoT supervision as constrained optimization over token weights, replacing uniform cross-entropy with adaptive token-level allocation based on gradient utility and adding variance-controlled update scaling to stabilize training. Empirically, it consistently outperforms existing token reweighting methods on mathematical and coding benchmarks and also yields a more effective initialization for later RL (Gong et al., 31 Oct 2025). The method is notable because it treats heterogeneous token importance as a first-order property of long reasoning traces rather than as noise around a standard language-modeling loss.

Training dynamics also challenge standard intuitions about sample diversity. Under a fixed update budget, “Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning” shows that training for more epochs on smaller datasets can outperform single-epoch training on larger datasets. On AIME’24/25 and GPQA, Olmo3-7B trained for 128 epochs on 400 samples outperformed the equivalent 1 epoch on 51,200 samples by 12–26 percentage points, with no additional catastrophic forgetting, and training token accuracy reliably signaled when repetition had saturated (Kopiczko et al., 11 Feb 2026). A plausible implication is that, in L-CoT SFT, memorization of a compact set of high-quality traces may act as capability elicitation rather than conventional overfitting.

4. Overthinking, compression, and efficient reasoning

One of the dominant criticisms of L-CoT SFT is that it inherits the “overthinking” problem from teacher models. Long-form SFT can cause models to generate unnecessarily verbose reasoning even when a short solution is adequate, degrading efficiency and sometimes accuracy or user experience (Yu et al., 6 May 2025). Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning addresses this by pairing each example with both a long rationale and a structure-preserved rewritten short rationale, then training on the mixture. Relative to direct SFT, the method reports an average accuracy improvement of 2.3% across benchmarks while reducing response length by approximately 47.61% (Yu et al., 6 May 2025).

Efficiency-oriented work on small LLMs arrives at similar conclusions through pruning rather than mixture design. “Efficient Long CoT Reasoning in Small LLMs” proposes binary cutting search to identify the shortest adequate prefix of a teacher trajectory and validates candidate prefixes with the target model itself. Across mathematical benchmarks, the streamlined traces reduce reasoning tokens by 50–70% while maintaining competitive performance, outperforming both random deletion and simpler first-correct prefix heuristics (Wang et al., 24 May 2025). This is an explicit rejection of the assumption that every teacher-generated step is pedagogically useful for a smaller student.

Subsequent work has moved from pruning to learned succinct reasoning. S3-CoT generates style-aligned variable-length traces from the target model itself via activation steering, filters them by answer correctness or prediction consistency, and then performs SFT with a dual-cognitive setup and a progressive compression curriculum. Across math and medical benchmarks, it reduces CoT length by 17–40% while preserving or slightly improving accuracy (Du et al., 2 Feb 2026). Extra-CoT pushes compression further by first training a semantically preserved compressor, then using mixed-ratio SFT and hierarchical RL; on MATH-500 with Qwen3-1.7B, it achieves over 73% token reduction together with a 0.6% accuracy improvement (Tang et al., 9 Feb 2026). Collectively, these systems recast L-CoT SFT from a length-maximizing practice into a budget-aware supervision problem.

5. Model-scale effects, architectural fragility, and degradation modes

L-CoT SFT is not uniformly beneficial across model sizes. “Through the Valley” identifies Long CoT Degradation in small LLMs, defined by significant deterioration after training on limited long-CoT data. In some settings, models trained on only 8k long-CoT examples lose up to 75% of their original performance before fine-tuning, and for some particularly small models, even 220k long-CoT examples fail to recover or surpass the original baseline (Luo et al., 9 Jun 2025). The paper attributes this to error accumulation: longer responses create room for multi-step reasoning but also for compounding mistakes, while short exposure encourages imitation of reflective style without corresponding competence.

Architecture matters as much as scale. In hybrid linear-attention models such as HypeNet and Jet-Nemotron, CoT-SFT can severely harm long-range recall, a failure termed “attention amnesia.” On Needle-In-A-Haystack, HypeNet-9B on NIAH-S2@256K drops from 67.2% before CoT-SFT to 9.4% after it. The proposed explanation is that CoT-SFT biases gradients toward short-range patterns and disrupts the query-key projections responsible for long-range routing; a training-free remedy, QK-Restore, restores only $W_Q$ and $W_K$ from the pre-SFT checkpoint and recovers long-context capability while largely preserving reasoning gains (Zhou et al., 9 Jun 2026).

Multimodal systems show a mixed picture. In lightweight MLLMs, long-CoT SFT materially improves reasoning and enables additional gains from a later RL stage, with text-only long-form CoT proving more effective than multimodal training data for this purpose (Ou, 3 Sep 2025). By contrast, in reasoning VLMs, long-CoT SFT and RL exhibit a “synergy dilemma”: SFT improves difficult questions through in-depth structured reasoning but introduces verbosity and hurts simpler questions, whereas RL improves all difficulty levels more consistently but offers less benefit on the hardest problems (Chen et al., 10 Jul 2025). These results contradict any architecture-agnostic claim that longer reasoning traces are uniformly beneficial.

6. Relation to reinforcement learning and domain-specific extensions

L-CoT SFT is frequently used as the initialization stage for RL-based post-training. Comparative experiments on long-CoT emergence show that RL initialized from long-CoT SFT benefits much more than RL initialized from short-CoT SFT, which often shows little gain (Yeo et al., 5 Feb 2025). More specialized hybrids build this coupling directly into the training objective. CARFT warms up the model with annotated CoT data, then adds PPO-based RL together with contrastive signals defined over CoT representations, using both annotated and self-generated trajectories. It reports robustness improvements together with gains of up to 10.15% in performance and up to 30.62% in efficiency relative to baselines (Zhu et al., 21 Aug 2025).

Yet the interaction between L-CoT SFT and RL is not universally additive. In multimodal reasoning, two-stage, interleaved, and progressive SFT–RL schedules, as well as data mixing and model merging, all fail to produce additive benefits; instead they induce trade-offs in accuracy, reasoning style, and response length (Chen et al., 10 Jul 2025). This suggests that SFT and RL can favor different operating points in the space of reasoning policies—one biased toward explicit structure and the other toward concise generalization. A plausible implication is that future systems may need difficulty-aware or budget-aware switching mechanisms rather than a single globally optimal post-training recipe.

The method family has also expanded beyond mathematical QA. In automated code review, MelcotCR uses long structured CoT to model multiple dimensions of review and combines predefined reasoning pathways with the Maximum Entropy principle to combat context loss and reasoning logic loss in long prompts. On MelcotCR and CodeReviewer, a 14B Qwen2.5 model fine-tuned in this manner surpasses prior methods and is reported to be on par with the 671B DeepSeek-R1 model (Yu et al., 25 Sep 2025). In long-context finance, supervised CoT reasoning substantially improves both annotation quality and downstream performance: GPT-4o-mini with the PAI annotator improves on Loong by 20.33%, and LongPAI, obtained by fine-tuning LLaMA-3.1-8B-Instruct on LongFinanceQA, gains 24.6% on Loong’s financial subset (Lin et al., 18 Feb 2025). These results indicate that L-CoT SFT is better understood as a general supervision paradigm for structured reasoning than as a math-specific technique.

L-CoT SFT therefore occupies a dual position in contemporary model development. It is both a mechanism for capability transfer and a source of new failure modes. The literature consistently shows that the decisive variables are not merely the presence of long reasoning traces, but their topology, redundancy, source alignment, token-level weighting, and interaction with model architecture and later RL.