Chain-of-Thought Supervised Fine-Tuning

• Chain-of-Thought Supervised Fine-Tuning is a method where models learn from sequences of reasoning steps followed by final answers to build enhanced decision-making capabilities.
• It structures training on triples of input, reasoning chain, and answer, enabling both detailed step-by-step explanation and improved end-task accuracy.
• Multiple supervision formats—including pre-thinking, post-thinking, and multi-tasking—offer flexible design choices that balance reasoning fidelity with answer optimization.

Chain-of-Thought Supervised Fine-Tuning (CoT SFT) is a post-training paradigm in which a LLM is optimized on targets that explicitly contain intermediate reasoning steps in addition to final answers. In the survey formulation, a chain of thought is represented as a sequence of reasoning nodes, and CoT fine-tuning is written as learning from triples of input, reasoning chain, and answer, with the model optimized on the concatenated reasoning-plus-answer target (Chen et al., 15 Oct 2025). Across recent work, CoT SFT functions both as a direct method for transferring reasoning ability from stronger teachers to smaller or non-reasoning models and as a cold-start stage for later reinforcement learning, reflection, or long-context grounding (Yu et al., 6 May 2025, Li et al., 6 Apr 2026, Wang et al., 29 Jan 2026). The same literature also shows that CoT SFT is not a single technique but a family of supervision designs that differ in rationale format, target granularity, token weighting, representation editing, and the relation between visible reasoning and final answers (Gong et al., 31 Oct 2025, Shi et al., 24 Dec 2025, Huang et al., 14 Jul 2025).

1. Definition, scope, and canonical objective

In the survey account, CoT fine-tuning aims to “endow LLMs with reasoning capabilities by training them on curated reasoning traces” (Chen et al., 15 Oct 2025). A single CoT is formalized as

$$\begin{aligned} & \mathcal{C} = \{n_k = \{s_k(r_{k-1}), r_k\} | k=0,1,...K\} \ & \quad\quad\quad\quad s.t. \quad \forall i < K, n_i \rightarrow n_{i+1}; \ & \quad\quad\hspace{0.3cm} s_k(r_{k-1}) = r_k \end{aligned}$$

and CoT fine-tuning overall is written as

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$

where $x$ is the question, $c$ the reasoning chain, and $y$ the answer (Chen et al., 15 Oct 2025).

A closely related token-level formulation appears in work on answer-focused reweighting. Standard supervised fine-tuning is written as

$$\mathcal{L}_{\text{SFT}}(\bm{\theta}) = -\sum_{i=1}^{N} \sum_{t=1}^{L_i} \log P\left(y_{i,t} \mid \bm{x}_i, \bm{y}_{i,<t}; \bm{\theta}\right)$$

and in CoT-style settings the target sequence typically contains a reasoning segment followed by a final-answer segment (Shi et al., 24 Dec 2025). In multimodal CoT-SFT, the same pattern appears explicitly as rationale tokens $r^\star$ followed by answer tokens $y^\star$, optimized under autoregressive negative log-likelihood conditioned on image and question (Sinha et al., 1 Jun 2026).

Within this literature, CoT SFT is distinct from answer-only supervised fine-tuning, distinct from inference-time CoT prompting, and distinct from reinforced fine-tuning. The survey identifies pre-thinking, post-thinking, and multi-task learning as the main supervised regimes (Chen et al., 15 Oct 2025). Other papers sharpen the distinction empirically: inference-time CoT prompting may improve alignment or reasoning only slightly, whereas supervised fine-tuning on explanations or reasoning traces can materially shift model behavior (DiSorbo et al., 4 Mar 2025). This suggests that CoT SFT is best understood as training on reasoning trajectories, not merely eliciting them.

2. Supervision formats and target structures

The dominant supervised format is pre-thinking: the model generates reasoning first and the answer afterward. The survey writes this as optimization on $c \oplus y$, whereas post-thinking instead uses $y \oplus c$, and multi-task learning separates rationale generation and label prediction through different prefixes (Chen et al., 15 Oct 2025). These distinctions matter because they affect whether reasoning serves as a scaffold for answer generation or as a post hoc explanation.

Recent papers use several structured target formats. In SFTKey, responses are explicitly segmented as

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$0

and the second training stage masks out the thinking span so that only answer tokens contribute to the loss (Shi et al., 24 Dec 2025). In SoftCoT, the training sequence is

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$1

with loss applied over the rationale span $$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$2 and answer span $$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$3, while pre-rationale tokens are masked (Xu et al., 17 Feb 2025). In long-context finance reasoning, the supervised target explicitly contains “properties, sub-answers, and final answers,” and the objective counts tokens in both reasoning and answer spans (Lin et al., 18 Feb 2025).

The literature also varies the granularity of what counts as a reasoning trace. Some methods use full natural-language derivations with calculations, as in conventional math CoT distillation (Yu et al., 6 May 2025). Some deliberately abstract away calculations: Solution Guidance Fine-Tuning trains on “problem-solving guidance without calculations or extra explanations,” and CoMT trains on “abstract meta-thoughts excluding numerical calculations” (Bi et al., 2024, Wang et al., 29 Jan 2026). Other methods impose schema-like reasoning stages. Fine-R1 structures targets as “visual analysis, candidate subcategories, comparison, and final prediction,” each enclosed in dedicated tags (He et al., 7 Feb 2026). ProxyCoT uses proxy-derived traces as full-sequence SFT targets on long contexts, without separate symbolic grounding or step-specific labels (Li et al., 6 Apr 2026).

A recurring issue is whether all target tokens should be treated equally. Standard CoT SFT uses uniform token-level cross-entropy. Several later methods argue that this is suboptimal because long rationales dominate the loss. SFTKey addresses this by a second answer-only stage (Shi et al., 24 Dec 2025). VCORE instead derives token weights from first-order gradient utility and constrains the weighting distribution to remain close to uniform for stability (Gong et al., 31 Oct 2025). This suggests that CoT SFT has shifted from a purely sequence-level perspective toward more selective supervision over which reasoning tokens matter most.

3. Data construction and sources of reasoning traces

CoT SFT depends on explicit reasoning traces, and the literature uses both manual and automatic construction. The survey divides data collection into manual annotation and automatic annotation, with automatic pipelines further split into LLM-based and rule-based generation (Chen et al., 15 Oct 2025). Manual explicit traces appear in datasets such as GSM8K and MATH; implicit rationale-style supervision appears in evidence datasets such as e-SNLI and FEVER (Chen et al., 15 Oct 2025).

Teacher-generated CoT is central to modern practice. LS-Mixture SFT begins from “Chain-of-Thought (CoT) reasoning data distilled from large reasoning models (e.g., DeepSeek R1)” and mixes long CoTs with short counterparts obtained by structure-preserved rewriting (Yu et al., 6 May 2025). Fine-R1 uses Qwen2.5-VL-32B to generate rationales, then samples repeatedly until the CoT leads to an exactly matched subcategory and manually verifies prediction consistency (He et al., 7 Feb 2026). SoftCoT relies on tasks “that include annotated reasoning steps,” but the learned conditioning is continuous rather than discrete (Xu et al., 17 Feb 2025). ARES uses teacher-provided sentence-level scores for RL and teacher-corrected rationales as SFT targets, so its supervised stage is built from repaired model outputs rather than raw gold traces (Byun et al., 2024).

Long-context reasoning papers emphasize a second problem: obtaining high-quality CoT on the full input is expensive or unreliable. ProxyCoT addresses this by collecting traces on a compact proxy context $$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$4, either from a teacher or from RL on the proxy, then supervising the same trace on the full context $$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$5 (Li et al., 6 Apr 2026). LongFinanceQA uses a multi-stage annotator, Property-driven Agentic Inference, to generate structured reasoning consisting of property extraction, retrieval-based sub-findings, and final synthesis (Lin et al., 18 Feb 2025).

The provenance and quality of traces are not uniform across papers. Some datasets use LM-generated CoT steps, as in Math10K and Commonsense60K for CRFT (Huang et al., 14 Jul 2025). Some papers explicitly note underspecified provenance: the NL2SQL CoT study states that it “created a specialized dataset including detailed Chain-of-Thought (CoT) steps” but does not clearly specify whether those annotations were manual, generated by a larger model, or programmatically derived (Solanki et al., 24 Mar 2026). A plausible implication is that data curation, verification, and trace quality remain first-order variables in CoT SFT, not merely preprocessing details.

4. Major methodological variants

A large part of recent work modifies standard CoT SFT rather than discarding it. One line addresses verbosity and overthinking. LS-Mixture SFT combines long CoT reasoning data with short rewritten counterparts and reports that, relative to direct SFT, it improves average accuracy by 2.3% while reducing response length by approximately 47.61% (Yu et al., 6 May 2025). SFTKey retains full CoT training in Stage 1 but then fine-tunes only the final-answer span, with the stated goal of balancing long rationale learning against the “Key” portion that determines benchmark correctness (Shi et al., 24 Dec 2025).

A second line changes the representation on which supervision operates. SoftCoT keeps the backbone LLM frozen and learns a projection from assistant-generated soft thought tokens into the target model’s representation space, while still applying next-token loss over rationale and answer spans (Xu et al., 17 Feb 2025). CRFT treats CoT SFT as a representation-editing problem: it identifies hidden states whose perturbation changes correctness and optimizes only those critical representations through a low-rank intervention,

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$6

with the base model frozen (Huang et al., 14 Jul 2025).

A third line modifies the supervision target itself. CoMT argues that full worked trajectories entangle abstract strategy with problem-specific execution, so it trains on “meta-thought” trajectories produced by a teacher under the prompt “Do NOT calculate any specific numbers” (Wang et al., 29 Jan 2026). SGFT similarly replaces full CoT with high-level “Solution Guidance,” which encodes problem decomposition without calculations or detailed derivations (Bi et al., 2024). These methods do not reject supervised reasoning traces; they alter the level of abstraction at which supervision is supplied.

A fourth line tackles token allocation. VCORE defines token utility as

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$7

and chooses token weights by solving

$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$8

which yields a Gibbs weighting distribution over token positions (Gong et al., 31 Oct 2025). This reframes CoT SFT as adaptive supervision allocation rather than uniform token imitation.

A fifth line integrates CoT SFT with subsequent policy optimization. ThinkDrive treats CoT SFT as the cold-start stage for autonomous-driving RL (Zhao et al., 8 Jan 2026). Fine-R1 uses CoT SFT to build a strong open-world fine-grained visual classifier before Triplet Augmented Policy Optimization (He et al., 7 Feb 2026). ProxyCoT uses SFT itself as the second-stage transfer mechanism after proxy-context trace acquisition (Li et al., 6 Apr 2026). ARES alternates RL and SFT, where SFT trains on teacher-corrected rationales after RL-induced exploration (Byun et al., 2024). These papers collectively position CoT SFT as both a standalone method and an initialization layer for broader post-training pipelines.

5. Empirical effects across domains

The empirical record is mixed but substantial. In teacher-distilled settings, CoT SFT can transfer reasoning capability effectively. LS-Mixture SFT reports that, compared to direct SFT, models achieve an average accuracy improvement of 2.3% across benchmarks with much shorter outputs (Yu et al., 6 May 2025). In NL2SQL, adding explicit CoT reasoning to Qwen fine-tuning improves execution accuracy from 45.33% under standard SFT to 54.50% under CoT fine-tuning on the reported 600-query benchmark (Solanki et al., 24 Mar 2026). In long-context finance reasoning, CoT-supervised LongPAI improves overall Average Score on the Loong financial subset from 45.88 to 70.54, while the answer-only ablation LongPAI$$\theta^* = \arg\min_{\theta} \mathbb{E}_{(x, y, c) \sim \mathcal{D}} \left[ \mathcal{L}(f_\theta(x), c \oplus y) \right]$$9 degrades sharply at longer context ranges (Lin et al., 18 Feb 2025).

Small-model studies often present CoT SFT as especially valuable or, conversely, as something that requires simplification. SGFT reports that high-level guidance can outperform CoT-fine-tuned baselines for 6B–7B models under limited-data regimes (Bi et al., 2024). CRFT shows that CoT-bearing supervision combined with targeted representation editing can raise GSM8K accuracy on LLaMA-2-7B from 14.6 to 32.8 while using only 0.016% trainable parameters (Huang et al., 14 Jul 2025). SoftCoT reports gains on five reasoning benchmarks while freezing both assistant and backbone and training only a projection module (Xu et al., 17 Feb 2025).

Long-context and multimodal settings show both the promise and the limits of standard CoT SFT. ProxyCoT demonstrates that SFT on proxy-derived traces conditioned on the full long context can substantially outperform direct long-context baselines on SciTrek and HotpotQA (Li et al., 6 Apr 2026). Fine-R1 reports that CoT SFT alone improves relative semantic similarity by 13.30% in its open-world FGVR ablation, and that direct CoT prompting without training has only limited impact (He et al., 7 Feb 2026). By contrast, a multimodal analysis across Qwen2.5-VL, InternVL3.5, and Gemma3 families shows that off-the-shelf CoT prompting often underperforms direct prompting, and that standard CoT-SFT only partially fixes premature answer commitment and weak visual-token access (Sinha et al., 1 Jun 2026).

The literature also documents settings where standard fine-tuning or CoT prompting underperform. In exception handling, CoT prompting reduces the human–model refusal-rate gap only slightly, from 45.3% to 40.6%, whereas explanation-based SFT reduces it to 15.9% on the reported scenario set (DiSorbo et al., 4 Mar 2025). In long CoT teacher comparison, lower SFT loss on DeepSeek-R1-0528 traces yields worse generalization than higher-loss training on gpt-oss-120b traces, indicating that “correct” trajectories can still encode inefficient reasoning patterns (Li et al., 2 Apr 2026). And a diagnostic study of standard downstream fine-tuning finds that CoT answer accuracy and CoT faithfulness often decrease after task adaptation, particularly for smaller models (Lobo et al., 2024). This suggests that gains in end-task accuracy do not automatically imply gains in reasoning quality.

6. Limitations, controversies, and open directions

Several limitations recur across the literature. The first is overthinking and verbosity. Teacher-distilled long-CoT models often inherit redundant exploratory behavior, verbose chains, or excessive token costs (Yu et al., 6 May 2025, Li et al., 2 Apr 2026). The second is supervision imbalance: long rationale spans dominate the token-level objective even though the final answer is short and benchmark-critical (Shi et al., 24 Dec 2025, Gong et al., 31 Oct 2025). The third is faithfulness. Generated CoTs may be post hoc, weakly causal, or only partially grounded, especially in multimodal systems or after unrelated task fine-tuning (Sinha et al., 1 Jun 2026, Lobo et al., 2024).

A fourth issue is data quality. The survey repeatedly stresses that synthetic traces require filtering, verification, or external checking (Chen et al., 15 Oct 2025). Fine-R1 uses repeated sampling and manual verification (He et al., 7 Feb 2026). ProxyCoT keeps only proxy traces that produce correct answers (Li et al., 6 Apr 2026). ARES builds SFT targets from teacher-corrected rationales precisely because RL outputs are unstable (Byun et al., 2024). The long-CoT teacher study sharpens this further: answer correctness alone is insufficient if the reasoning pattern is branch-heavy and redundant (Li et al., 2 Apr 2026).

A fifth issue concerns generalization and what exactly should be supervised. CoMT argues that complete trajectories are the wrong grain because they entangle reusable strategy with instance-specific execution (Wang et al., 29 Jan 2026). SGFT makes a related claim for small models and low-data settings (Bi et al., 2024). SFTKey, VCORE, and CRFT all imply that standard full-sequence cross-entropy is too coarse, but they intervene at different levels: answer spans, token weights, or critical representations (Shi et al., 24 Dec 2025, Gong et al., 31 Oct 2025, Huang et al., 14 Jul 2025). This suggests that “CoT SFT” is evolving from uniform imitation of full traces toward more structured, selective, and task-aware supervision.

Open directions are correspondingly broad. The survey calls for more faithful, diverse, reflective, transferable, efficiently internalized, and fact-grounded traces (Chen et al., 15 Oct 2025). Multimodal work suggests supervising reasoning dynamics, not just rationale text (Sinha et al., 1 Jun 2026). Long-context work suggests transferring traces from easier proxy contexts or agentic decompositions (Li et al., 6 Apr 2026, Lin et al., 18 Feb 2025). Representation- and utility-based methods suggest that future CoT SFT may operate less through blanket weight updates and more through targeted editing or optimization-theoretic token allocation (Huang et al., 14 Jul 2025, Gong et al., 31 Oct 2025). A plausible implication is that CoT SFT is no longer defined solely by visible step-by-step text, but by a broader design question: which parts of a reasoning process should be supervised, at what granularity, and with what relation to final-answer correctness.