---
title: 'CoPeD: Correctness Perception in CoT Distillation'
url: https://www.emergentmind.com/topics/chain-of-thought-correctness-perception-distillation-coped
type: topic
---

# CoPeD: Correctness Perception in CoT Distillation

Searching arXiv for the cited CoPeD and related chain-of-thought distillation work to ground the article in current papers.
Chain-of-Thought Correctness Perception Distillation (CoPeD) is a distillation paradigm for transferring reasoning capability from large language models to smaller language models while explicitly targeting the correctness-supporting role of intermediate rationales rather than treating any chain-of-thought (CoT) accompanying a correct answer as inherently useful. In the CoPeD formulation, the central failure mode of standard CoT distillation is that noisy, vacuous, or unfaithful rationales induce spurious correlations between questions and answers, allowing the student to predict labels without relying on causally supportive reasoning [2509.05602]. Within the broader literature, CoPeD names a family resemblance across methods that make rationale correctness perceptible and operational during distillation: SCOTT enforces answer-conditioned rationale faithfulness and counterfactual dependence [2305.01879], EDIT isolates correctness-critical edits in dual chains [2405.19737], Chain-of-Probe analyzes whether reasoning is necessary and whether correct answers conceal faulty reasoning [2406.16144], and later medical auditing work shows that answer gains can coexist with worse step-level trace factuality if correctness perception is not explicitly supervised [2605.28301]. In this sense, CoPeD is both a specific method and a more general design principle: the student should learn not merely to imitate reasoning form, but to detect, use, revise, and be evaluated on reasoning content that genuinely supports correct answers.

## 1. Conceptual basis and problem formulation

CoPeD addresses a problem exposed repeatedly in CoT distillation research: teacher-generated rationales may be syntactically fluent yet semantically uninformative, inconsistent with the model’s own answer, or detached from the decision boundary that produces correctness. The CoPeD paper frames this as spurious correlation between question and answer induced by noisy rationales, producing pathological configurations such as “correct rationale, wrong answer” and “wrong rationale, correct answer” [2509.05602]. The issue is not only label noise in the usual supervised-learning sense; it is a process-level mismatch in which the rationale fails to substantiate the answer.

Earlier work already identified adjacent phenomena. SCOTT argues that large models can generate rationales that are hallucinated, off-topic, or inconsistent with their predictions, and that a student trained on such data may simply ignore the rationale and exploit shortcuts between question and answer [2305.01879]. Chain-of-Probe adds a distinct but complementary diagnosis: in many cases the model effectively has the answer before generating the reasoning, a phenomenon termed Early Answering, implying that the generated CoT may be unnecessary for the prediction and therefore weak evidence of faithful reasoning [2406.16144]. The medical audit literature sharpens the concern further by showing that answer-level improvements after distillation can coincide with substantial degradation in step-level factual correctness, especially when compact answer options under-constrain a rich rationale [2605.28301].

Within this landscape, CoPeD can be defined operationally as distillation of correctness perception: the student should learn that a rationale is valuable insofar as it increases the likelihood of the correct answer, decreases the likelihood of incorrect answers, and causes the model’s prediction to change appropriately when the rationale is revised, perturbed, or replaced. This definition is explicit in SCOTT’s mapping to correctness perception, where contrastive decoding and counterfactual training instantiate this dependence [2305.01879], and it is algorithmically refined in EDIT, where only a sparse subset of reasoning tokens—about \(4.7\%\) on average—are identified as key steps whose alteration flips the conclusion [2405.19737].

A plausible implication is that CoPeD should be understood less as a single architecture than as a methodological criterion for process-supervised distillation: a student has learned correctness perception only if its internal and output behavior demonstrably tracks rationale quality rather than rationale style.

## 2. Formalization in the CoPeD method

The 2025 CoPeD method formulates correctness perception through a correctness-aware task setting and a Correctness-Aware Weighted loss [2509.05602]. The training data are constructed as
\[
\mathcal{D}_{\text{train}} = \{(q_i, r_i^+, r_i^-, a_i)\}_{i=1}^n,
\]
where \(q_i\) is the question, \(a_i\) the ground-truth answer, \(r_i^+\) a correct rationale, and \(r_i^-\) an erroneous rationale. The teacher is gpt-3.5-turbo-0613, and rationale correctness is heuristically labeled by whether the teacher’s predicted answer matches the ground truth [2509.05602]. Because this heuristic is itself noisy, CoPeD does not treat the label as perfectly reliable; instead, it uses weighting to attenuate harmful cases.

The first task is answer prediction from a correct rationale. The input is \(q_i\), and the target is \(r_i^+ \oplus rs_t \oplus a_i\), where \(rs_t\) is the status string “the rationale is right.” The corresponding loss is
\[
\mathcal{L}_{\text{ra}}
\;=\;
\mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}}
\left[
\ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big)
\right].
\]
This task teaches the student to generate a rationale and then answer through that rationale, rather than treating the rationale as an inert by-product [2509.05602].

The second task is rationale correction. The input is \(q_i \oplus r_i^- \oplus rs_f\), where \(rs_f\) is “the rationale is wrong,” and the target is \(r_i^+\). The loss is
\[
\mathcal{L}_{\text{rc}}
\;=\;
\mathbb{E}_{(q,r^+,r^-)\sim\mathcal{D}_{\text{train}}}
\left[
\ell\Big(q\oplus r^-\oplus rs_{\mathrm{f},\, r^+\Big)
\right].
\]
This objective introduces “learning from mistakes” directly into the task definition: the student is not only taught to answer from good reasoning but also to revise bad reasoning into good reasoning [2509.05602].

The joint objective is
\[
\mathcal{L}
\;=\;
\mathcal{L}_{\text{ra}}
\;+\;
\alpha\,\mathcal{L}_{\text{rc}},
\]
with default \(\alpha = 0.5\) [2509.05602]. This balance is empirically selected; the paper reports best average performance at \(\alpha=0.5\).

The distinctive weighting mechanism is defined per sample using rationale and answer losses:
\[
\mathcal{L}_r^{(i)} = CE(\hat{r}_i, r_i), \qquad
\mathcal{L}_a^{(i)} = CE(\hat{a}_i, a_i),
\]
and
\[
w_i = \mathrm{softmax}_i\!\left(
-\frac{\mathcal{L}_r^{(i)} + \mathcal{L}_a^{(i)} + \left|\mathcal{L}_r^{(i)} - \mathcal{L}_a^{(i)}\right|}{\tau}
\right).
\]
The final weighted answer-prediction objective becomes
\[
\mathcal{L}_{\text{ra}}
\;=\;
\sum_i w_i\,\Big(\mathcal{L}_r^{(i)} + \mathcal{L}_a^{(i)}\Big).
\]
Here, the composite term \(\mathcal{L}_r + \mathcal{L}_a\) measures sample reliability, and the discrepancy term \(|\mathcal{L}_r - \mathcal{L}_a|\) measures rationale–answer alignment [2509.05602]. Samples whose rationale and answer are both easy and mutually aligned receive greater weight, whereas cases with high loss or misalignment are down-weighted. This makes correctness perception a training-time allocation mechanism as well as a task design.

## 3. Relation to precursor and adjacent methods

The CoPeD idea is tightly connected to several distinct lines of work, each of which operationalizes correctness perception differently.

SCOTT: Self-Consistent Chain-of-Thought Distillation formalizes faithfulness as dependence of the prediction on the rationale and uses a two-stage design: teacher-side contrastive decoding and student-side counterfactual reasoning [2305.01879]. The teacher rationale is generated by preferring tokens whose plausibility increases under the gold answer relative to a perturbed answer:
\[
G(t_i\mid a^*)=\log\frac{P(t_i\mid p,q,a^*,t_{<i})}{P(t_i\mid p,q,a^{\prime},t_{<i})},
\]
with decoding
\[
t_i^{*}=\argmax\Big[\log P(t_i\mid p,q,a^*,t_{<i})+G(t_i\mid a^*)\Big].
\]
The student is then trained not only on factual rationale-plus-answer sequences but also on counterfactual rationales \(r'\) supporting a wrong answer \(a'\), with
\[
\mathcal{L}_{\text{counterfactual}}=-\sum_{t_i\in a′}\log P(t_i\mid q,r',t_{<i}),
\]
and total loss
\[
\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{factual}}+\mathcal{L}_{\text{counterfactual}}.
\]
This design makes the same question yield different answers under different rationales, directly opposing shortcut learning [2305.01879]. In CoPeD terminology, SCOTT operationalizes correctness perception through answer-conditioned rationale generation and counterfactual sensitivity.

EDIT: mistakE-Driven key reasonIng step disTillation takes a more localized view. Instead of supervising entire chains uniformly, it constructs dual CoTs with similar reasoning paths but divergent conclusions and then uses minimum edit distance to identify key steps [2405.19737]. The paper argues that only about \(4.7\%\) of tokens are decisive. For correct and incorrect chains, token weights are assigned as
\[
\omega_t^+ =
\begin{cases}
\alpha & \text{if } CoT_t^+ \text{ is inserted or replaced} \\
0 & \text{otherwise}
\end{cases},
\qquad
\omega_t^- =
\begin{cases}
\beta & \text{if } CoT_t^- \text{ is deleted or replaced} \\
0 & \text{otherwise}
\end{cases},
\]
with \(\alpha = 1.0\) and \(\beta = 0.025\). This yields a weighted key-reasoning-steps learning objective that emphasizes correct decisive edits and suppresses incorrect ones [2405.19737]. Relative to CoPeD, EDIT concentrates correctness perception at sparse edit positions rather than distributing it across whole-sample reliability.

Chain-of-Probe does not perform distillation directly, but it provides a probing-based theory of when CoT is necessary and whether correct answers imply correct reasoning [2406.16144]. It defines the stepwise answer probe
\[
P(y|x; s_{1:i} ;a_*) = \mathrm{softmax}\ \mathrm{LLM}(x; s_{1:i}; a_*),
\]
derives confidence trajectories, and defines Early Answering when the stepwise argmax remains equal to the final answer throughout. It also introduces the CoP Score
\[
\mathrm{CoPS} = \frac{1}{k+1}\sum_{i=0}^{k} p_i^{v^*} + \frac{1}{k}(p_k^{v^*} - p_0^{v^*}),
\]
which correlates positively with answer accuracy [2406.16144]. For CoPeD, this suggests that correctness perception may be distilled not only from text labels but also from internal confidence dynamics and mind-change signals.

Program-aided Distillation provides a verification-oriented contrast case. PaD replaces natural-language CoT with executable Python programs and filters synthetic data by execution correctness:
\[
f_{\mathrm{verify}(\mathbf{x}_i, \mathbf{y}_i, \mathbf{r}_i) =
\begin{cases}
1, & \text{if } \mathrm{exec}(\mathbf{r}_i) = \mathbf{y}_i \ \text{and no exception}\\
0, & \text{otherwise}.
\end{cases}
\]
This yields high-precision reasoning supervision for arithmetic and symbolic tasks [2305.13888]. A plausible implication is that PaD can serve as a hard-verification source for CoPeD-style correctness labels where tasks admit executable semantics.

The later medical auditing work functions as a cautionary counterpoint. It shows that without explicit step-level factuality supervision, CoT distillation may improve answer metrics while degrading the factual reliability of the trace [2605.28301]. That result strengthens the motivation for CoPeD by demonstrating that final-answer supervision plus trace imitation is insufficient in domains where answers under-specify rationales.

## 4. Training pipeline and data construction

The CoPeD training pipeline begins with teacher generation of both correct and erroneous rationales using prompt templates following Dai et al. (2024), with correctness labeled by comparing the teacher answer to the ground truth [2509.05602]. This produces dual-rationale supervision for each question: a candidate rationale assumed correct because the answer matches the label, and a candidate rationale assumed erroneous because the answer does not. The paper explicitly notes that this labeling is heuristic, which motivates the weighting mechanism.

For each training tuple \((q_i, r_i^+, r_i^-, a_i)\), two supervision channels are constructed. The answer-prediction channel uses \(q_i\) as input and \(r_i^+ \oplus rs_t \oplus a_i\) as target; the rationale-correction channel uses \(q_i \oplus r_i^- \oplus rs_f\) as input and \(r_i^+\) as target [2509.05602]. During training, the model computes per-sample rationale and answer losses. In early epochs, uniform weighting is used to establish basic capability; starting from epoch \(n\), the Correctness-Aware Weighted loss is activated. For LLaMA2-7B, the reported best temperature is \(\tau = 5.0\), and the weighting begins at epoch \(n=5\); for TinyLLaMA, \(n=10\) [2509.05602].

The primary student in the paper is LLaMA2-7B, with additional experiments on TinyLLaMA-1.1B, LLaMA2-13B, and Mistral-7B-v0.2. Fine-tuning uses LoRA with rank \(64\), \(\alpha_{\text{LoRA}}=32\), target modules \(q\_proj\) and \(v\_proj\), and dropout \(0.05\). Optimization uses AdamW with \(\beta=(0.9,0.999)\), \(\epsilon=10^{-8}\), no weight decay, and scheduler \(\gamma=0.95\); training runs on \(8\times\) A800 GPUs with gradient accumulation \(=2\), per-device batch size \(=2\), and maximum input length \(1024\) [2509.05602].

The benchmarks span both in-distribution and out-of-distribution settings. The in-distribution data are BIG-Bench Hard (BBH), split 4:1 into BBH-train and BBH-test. Out-of-distribution evaluation uses BB-sub, AGIEval English multiple-choice subsets, ARC-Easy, and ARC-Challenge [2509.05602]. This distributional setup is important because CoPeD is presented not only as a faithfulness intervention but as a generalization intervention: by weakening spurious question–answer shortcuts, the method aims to improve robustness across task families.

SCOTT and EDIT provide alternative data-construction patterns that are often discussed alongside CoPeD. SCOTT uses contrastively decoded teacher rationales and counterfactual rationales produced under wrong-answer conditioning [2305.01879]. EDIT constructs dual sets \(D_{\text{dual}}^+\) and \(D_{\text{dual}}^-\) through rectification of wrong chains and corruption of correct chains, ultimately producing \(5207\) dual CoTs, with \(|D_{\text{dual}}^+| = 3805\) and \(|D_{\text{dual}}^-| = 1402\) [2405.19737]. These alternatives illustrate a shared design pattern: correctness perception becomes learnable when training data expose contrast between supportive and unsupportive reasoning rather than presenting single chains in isolation.

## 5. Evaluation methodology and empirical findings

The CoPeD paper evaluates three broad properties: answer performance, rationale faithfulness, and rationale soundness [2509.05602]. Faithfulness is defined as whether the student’s rationale supports its own predicted answer; soundness is whether the rationale supports the ground-truth answer. Both are measured using binary LLM evaluators, with ChatGPT and GPT-4 used to mitigate single-model bias. Formally, with evaluator \(f_{\mathrm{eval}}(\hat r, a) \in \{0,1\}\),
\[
\text{Faithfulness}(\%) = 100 \cdot \mathbb{E}[f_{\mathrm{eval}}(\hat r_i, \hat a_i)],
\]
\[
\text{Soundness}(\%) = 100 \cdot \mathbb{E}[f_{\mathrm{eval}}(\hat r_i, a_i)].
\]
These metrics are central because CoPeD’s stated objective is not just accuracy but better causal alignment between rationale and answer.

For LLaMA2-7B, average performance across BBH-test, BB-sub, AGIEval, ARC-E, and ARC-C is reported as \(49.9\%\) for CasCoD, \(51.8\%\) for CoPeD-T, \(49.4\%\) for CoPeD-L, and \(52.9\%\) for CoPeD-TL [2509.05602]. The strongest variant, CoPeD-TL, improves over CasCoD by \(+3.0\%\) overall and by \(+9.6\%\) on the in-distribution BBH-test. The paper also reports OOD gains of \(+3.1\%\) on AGIEval, \(+1.5\%\) on ARC-E, and \(+2.7\%\) on ARC-C relative to the strongest baseline [2509.05602].

On rationale quality, CoPeD-TL raises average faithfulness to \(81.2\%\), compared with \(74.2\%\) for Std-CoT and \(77.4\%\) for CasCoD; average soundness rises to \(70.2\%\), compared with \(59.3\%\) for Std-CoT and \(64.9\%\) for CasCoD [2509.05602]. These results support the paper’s claim that correctness-aware task setting and weighting reduce shortcut reliance.

The ablations isolate the two weighting components. On BBH-test, Uniform weighting yields \(58.5\%\), “composite only” yields \(60.2\%\), “discrepancy only” yields \(59.8\%\), and full CoPeD-L yields \(60.9\%\) [2509.05602]. This indicates that both overall sample quality and rationale–answer alignment matter. Temperature sweeps show the best BBH-test performance at \(\tau = 5.0\), with \(69.8\%\), and task balance sweeps show best average performance at \(\alpha = 0.5\) [2509.05602].

Data-efficiency results are also notable. With only \(12.5\%\) of BBH-train, CoPeD-TL improves over CasCoD by \(+16.8\%\) on BBH-test and \(+9.4\%\) on BB-sub [2509.05602]. This is consistent with the claim that correctness perception is not merely a regularizer for high-data settings but a mechanism for using scarce distillation data more productively.

Other papers provide complementary evaluation frameworks. SCOTT uses Leakage-Adjusted Simulatability (LAS), perturbation sensitivity, and rationale refinement gains to assess whether predictions genuinely depend on rationales [2305.01879]. The paper reports that contrastive decoding improves teacher LAS over greedy and human annotations, that counterfactual training raises student LAS, and that SCOTT students show larger performance drops under rationale perturbations and larger gains from oracle rationale refinement [2305.01879]. EDIT evaluates mostly by answer accuracy and shows average gains from \(41.8\%\) for Std-CoT to \(46.5\%\) for EDIT across BBH-test, BB-sub, AGIEval, ARC-E, and ARC-C, with particularly large differences on ARC-C [2405.19737]. Chain-of-Probe adds CoT Accuracy and True Answer with False CoT Ratio, reporting average TAFCR \(21.1\%\) on ARC-Challenge and about \(+13\%\) average improvement in CoT correctness from greedy sampling to CoP Tree resampling [2406.16144].

Taken together, these evaluations suggest that CoPeD-like systems require multi-axis assessment: end-task accuracy alone cannot reveal whether the student has actually learned correctness perception.

## 6. Common misconceptions, limitations, and later critiques

A common misconception is that if a teacher answer is correct, its rationale is a valid supervision signal. The CoPeD paper explicitly rejects this assumption: heuristic correctness labels are noisy, and the weighting mechanism is introduced precisely because a correct answer can co-occur with a flawed rationale [2509.05602]. Chain-of-Probe reinforces the point by showing that many correct responses still contain reasoning errors, using the metric TAFCR and GPT-4-based reasoning annotations [2406.16144].

A second misconception is that higher answer accuracy implies better reasoning traces. The medical audit literature directly contradicts this inference. On MedQA-USMLE, a Qwen3-8B student distilled from a DeepSeek-V3-family teacher improves SC@64 accuracy from \(74.7\%\) to \(84.4\%\), reduces ECE from \(0.096\) to \(0.034\), and reduces Brier score from \(0.158\) to \(0.106\), yet its step-level factual error rate over non-abstained steps rises from \(30.6\%\) to \(50.3\%\) under a style-blind Kimi-K2.6 audit [2605.28301]. The same paper reports that this before–after pattern persists across evaluators, answer-conditioned controls, style and segmentation controls, and multiple student families [2605.28301]. This does not refute CoPeD; rather, it shows what happens when correctness perception is not made an explicit training target.

A third misconception is that rationales are necessarily needed for prediction. Chain-of-Probe’s Early Answering analysis shows that CoT is often unnecessary, especially on simpler tasks, and that accuracy during EA is significantly higher than when the model changes its mind, with the difference generally exceeding \(20\%\) [2406.16144]. This suggests that one should distinguish rationale usefulness from rationale presence: CoPeD is concerned with the former.

The methods also have practical and conceptual limitations. CoPeD depends on teacher quality and heuristic labeling via teacher answer correctness, though weighted loss mitigates the resulting noise [2509.05602]. SCOTT incurs extra teacher-side forward passes during contrastive decoding and extra training cost from counterfactual examples [2305.01879]. EDIT relies on dual CoTs of sufficiently similar form for edit alignment; the paper notes that minimum edit distance is a surface alignment and may miss semantic equivalence or paraphrase [2405.19737]. Chain-of-Probe currently targets single-token answer sets and depends on step segmentation quality [2406.16144]. Program-aided approaches have limited coverage for commonsense and knowledge-heavy tasks lacking clean executable structure [2305.13888]. The medical audit literature adds a broader warning: even when internal signals or hedging cues remain decodable, they may be miscalibrated to a new reliability baseline after distillation [2605.28301].

A plausible implication is that “correctness perception” should not be treated as solved by any one proxy. Heuristic answer matching, simulator-based faithfulness, LLM-as-judge audits, perturbation sensitivity, and executable verification each measure different slices of the underlying property.

## 7. Broader significance and future directions

CoPeD is significant because it reorients CoT distillation away from trace imitation and toward process-sensitive supervision. In the standard paradigm, rationales are often treated as privileged labels. CoPeD and related work instead treat them as objects to be validated, contrasted, weighted, revised, or verified before they can serve as supervision [2509.05602]. This shift aligns with broader process-supervision agendas while remaining tightly grounded in distillation.

Several concrete trajectories are already visible in the literature. SCOTT suggests a counterfactual route in which correctness perception is learned through answer-conditioned rationale generation and rationale-dependent answer switching [2305.01879]. EDIT suggests a sparsity route in which training focuses on the few tokens or steps that actually determine correctness [2405.19737]. Chain-of-Probe suggests an introspective route in which confidence trajectories, Early Answering, and mind-change patterns become teacher signals for process-level distillation [2406.16144]. Program-aided methods suggest a verification route in which correctness perception is grounded in execution where possible [2305.13888]. The medical auditing work suggests an auditing route in which any CoT distillation system, especially in high-stakes domains, should report step-level factuality under style-blind evaluation rather than relying on answer metrics alone [2605.28301].

Two later developments extend the design space further. CoT-Evo, in scientific reasoning, evolves reasoning trajectories before distillation using a fitness
\[
R(t) = s_{EM} + \lambda_1 s_{LEN} + \lambda_2 s_{KNOW},
\]
with \(\lambda_1=0.3\) and \(\lambda_2=0.1\), combining exact-match correctness, length appropriateness as a coherence proxy, and knowledge-usage correctness [2510.13166]. Although not framed as CoPeD originally, the method explicitly connects itself to correctness perception by refining trajectories for correctness, coherence, and knowledge grounding before student fine-tuning. This suggests that CoPeD can be extended beyond dual-rationale or weighted-loss settings into data-synthesis pipelines that optimize rationale quality pre-distillation.

The most immediate future direction, however, is methodological rigor. The medical audit paper argues that step-level factuality supervision, judge diversity, and reporting standards should become explicit components of distillation pipelines [2605.28301]. This suggests that future CoPeD systems may require hybrid objectives combining answer loss, step-level factuality loss, and calibration loss, together with evaluation protocols that report answer accuracy, faithfulness, soundness, perturbation sensitivity, refinement gains, and audited step correctness. Such a program would move CoPeD from being mainly a remedy for spurious question–answer correlations into a more general framework for trustworthy process distillation.

In summary, Chain-of-Thought Correctness Perception Distillation names a transition in reasoning distillation research: from asking whether students can reproduce answers and rationale style, to asking whether they can perceive which reasoning is correctness-supporting, revise what is not, and remain sensitive to the causal role of rationale content. The empirical record across SCOTT, EDIT, Chain-of-Probe, CoPeD, CoT-Evo, and step-level medical audits indicates that this transition is necessary if distilled reasoning traces are to be treated as more than rhetorically plausible artifacts [2305.01879] [2405.19737] [2406.16144] [2509.05602] [2510.13166] [2605.28301].

Source: https://www.emergentmind.com/topics/chain-of-thought-correctness-perception-distillation-coped