CoPeD: Correctness Perception in CoT Distillation
- The paper introduces CoPeD, a novel distillation paradigm that trains models to detect, use, and revise valid reasoning, reducing reliance on spurious question–answer shortcuts.
- It employs a dual-channel approach—answer prediction from correct rationales and rationale correction from erroneous ones—with a weighted loss that balances composite error and rationale–answer alignment.
- Empirical results demonstrate improved answer accuracy, enhanced rationale faithfulness, and better generalization across in-distribution and out-of-distribution benchmarks.
Searching arXiv for the cited CoPeD and related chain-of-thought distillation work to ground the article in current papers. Chain-of-Thought Correctness Perception Distillation (CoPeD) is a distillation paradigm for transferring reasoning capability from LLMs to smaller LLMs while explicitly targeting the correctness-supporting role of intermediate rationales rather than treating any chain-of-thought (CoT) accompanying a correct answer as inherently useful. In the CoPeD formulation, the central failure mode of standard CoT distillation is that noisy, vacuous, or unfaithful rationales induce spurious correlations between questions and answers, allowing the student to predict labels without relying on causally supportive reasoning (Xie et al., 6 Sep 2025). Within the broader literature, CoPeD names a family resemblance across methods that make rationale correctness perceptible and operational during distillation: SCOTT enforces answer-conditioned rationale faithfulness and counterfactual dependence (Wang et al., 2023), EDIT isolates correctness-critical edits in dual chains (Dai et al., 2024), Chain-of-Probe analyzes whether reasoning is necessary and whether correct answers conceal faulty reasoning (Wang et al., 2024), and later medical auditing work shows that answer gains can coexist with worse step-level trace factuality if correctness perception is not explicitly supervised (Jiang et al., 27 May 2026). In this sense, CoPeD is both a specific method and a more general design principle: the student should learn not merely to imitate reasoning form, but to detect, use, revise, and be evaluated on reasoning content that genuinely supports correct answers.
1. Conceptual basis and problem formulation
CoPeD addresses a problem exposed repeatedly in CoT distillation research: teacher-generated rationales may be syntactically fluent yet semantically uninformative, inconsistent with the model’s own answer, or detached from the decision boundary that produces correctness. The CoPeD paper frames this as spurious correlation between question and answer induced by noisy rationales, producing pathological configurations such as “correct rationale, wrong answer” and “wrong rationale, correct answer” (Xie et al., 6 Sep 2025). The issue is not only label noise in the usual supervised-learning sense; it is a process-level mismatch in which the rationale fails to substantiate the answer.
Earlier work already identified adjacent phenomena. SCOTT argues that large models can generate rationales that are hallucinated, off-topic, or inconsistent with their predictions, and that a student trained on such data may simply ignore the rationale and exploit shortcuts between question and answer (Wang et al., 2023). Chain-of-Probe adds a distinct but complementary diagnosis: in many cases the model effectively has the answer before generating the reasoning, a phenomenon termed Early Answering, implying that the generated CoT may be unnecessary for the prediction and therefore weak evidence of faithful reasoning (Wang et al., 2024). The medical audit literature sharpens the concern further by showing that answer-level improvements after distillation can coincide with substantial degradation in step-level factual correctness, especially when compact answer options under-constrain a rich rationale (Jiang et al., 27 May 2026).
Within this landscape, CoPeD can be defined operationally as distillation of correctness perception: the student should learn that a rationale is valuable insofar as it increases the likelihood of the correct answer, decreases the likelihood of incorrect answers, and causes the model’s prediction to change appropriately when the rationale is revised, perturbed, or replaced. This definition is explicit in SCOTT’s mapping to correctness perception, where contrastive decoding and counterfactual training instantiate this dependence (Wang et al., 2023), and it is algorithmically refined in EDIT, where only a sparse subset of reasoning tokens—about on average—are identified as key steps whose alteration flips the conclusion (Dai et al., 2024).
A plausible implication is that CoPeD should be understood less as a single architecture than as a methodological criterion for process-supervised distillation: a student has learned correctness perception only if its internal and output behavior demonstrably tracks rationale quality rather than rationale style.
2. Formalization in the CoPeD method
The 2025 CoPeD method formulates correctness perception through a correctness-aware task setting and a Correctness-Aware Weighted loss (Xie et al., 6 Sep 2025). The training data are constructed as
where is the question, the ground-truth answer, a correct rationale, and an erroneous rationale. The teacher is gpt-3.5-turbo-0613, and rationale correctness is heuristically labeled by whether the teacher’s predicted answer matches the ground truth (Xie et al., 6 Sep 2025). Because this heuristic is itself noisy, CoPeD does not treat the label as perfectly reliable; instead, it uses weighting to attenuate harmful cases.
The first task is answer prediction from a correct rationale. The input is , and the target is , where is the status string “the rationale is right.” The corresponding loss is
$\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$
This task teaches the student to generate a rationale and then answer through that rationale, rather than treating the rationale as an inert by-product (Xie et al., 6 Sep 2025).
The second task is rationale correction. The input is 0, where 1 is “the rationale is wrong,” and the target is 2. The loss is
3
This objective introduces “learning from mistakes” directly into the task definition: the student is not only taught to answer from good reasoning but also to revise bad reasoning into good reasoning (Xie et al., 6 Sep 2025).
The joint objective is
4
with default 5 (Xie et al., 6 Sep 2025). This balance is empirically selected; the paper reports best average performance at 6.
The distinctive weighting mechanism is defined per sample using rationale and answer losses: 7 and
8
The final weighted answer-prediction objective becomes
9
Here, the composite term 0 measures sample reliability, and the discrepancy term 1 measures rationale–answer alignment (Xie et al., 6 Sep 2025). Samples whose rationale and answer are both easy and mutually aligned receive greater weight, whereas cases with high loss or misalignment are down-weighted. This makes correctness perception a training-time allocation mechanism as well as a task design.
3. Relation to precursor and adjacent methods
The CoPeD idea is tightly connected to several distinct lines of work, each of which operationalizes correctness perception differently.
SCOTT: Self-Consistent Chain-of-Thought Distillation formalizes faithfulness as dependence of the prediction on the rationale and uses a two-stage design: teacher-side contrastive decoding and student-side counterfactual reasoning (Wang et al., 2023). The teacher rationale is generated by preferring tokens whose plausibility increases under the gold answer relative to a perturbed answer: 2 with decoding
3
The student is then trained not only on factual rationale-plus-answer sequences but also on counterfactual rationales 4 supporting a wrong answer 5, with
6
and total loss
7
This design makes the same question yield different answers under different rationales, directly opposing shortcut learning (Wang et al., 2023). In CoPeD terminology, SCOTT operationalizes correctness perception through answer-conditioned rationale generation and counterfactual sensitivity.
EDIT: mistakE-Driven key reasonIng step disTillation takes a more localized view. Instead of supervising entire chains uniformly, it constructs dual CoTs with similar reasoning paths but divergent conclusions and then uses minimum edit distance to identify key steps (Dai et al., 2024). The paper argues that only about 8 of tokens are decisive. For correct and incorrect chains, token weights are assigned as
9
with 0 and 1. This yields a weighted key-reasoning-steps learning objective that emphasizes correct decisive edits and suppresses incorrect ones (Dai et al., 2024). Relative to CoPeD, EDIT concentrates correctness perception at sparse edit positions rather than distributing it across whole-sample reliability.
Chain-of-Probe does not perform distillation directly, but it provides a probing-based theory of when CoT is necessary and whether correct answers imply correct reasoning (Wang et al., 2024). It defines the stepwise answer probe
2
derives confidence trajectories, and defines Early Answering when the stepwise argmax remains equal to the final answer throughout. It also introduces the CoP Score
3
which correlates positively with answer accuracy (Wang et al., 2024). For CoPeD, this suggests that correctness perception may be distilled not only from text labels but also from internal confidence dynamics and mind-change signals.
Program-aided Distillation provides a verification-oriented contrast case. PaD replaces natural-language CoT with executable Python programs and filters synthetic data by execution correctness: 4 This yields high-precision reasoning supervision for arithmetic and symbolic tasks (Zhu et al., 2023). A plausible implication is that PaD can serve as a hard-verification source for CoPeD-style correctness labels where tasks admit executable semantics.
The later medical auditing work functions as a cautionary counterpoint. It shows that without explicit step-level factuality supervision, CoT distillation may improve answer metrics while degrading the factual reliability of the trace (Jiang et al., 27 May 2026). That result strengthens the motivation for CoPeD by demonstrating that final-answer supervision plus trace imitation is insufficient in domains where answers under-specify rationales.
4. Training pipeline and data construction
The CoPeD training pipeline begins with teacher generation of both correct and erroneous rationales using prompt templates following Dai et al. (2024), with correctness labeled by comparing the teacher answer to the ground truth (Xie et al., 6 Sep 2025). This produces dual-rationale supervision for each question: a candidate rationale assumed correct because the answer matches the label, and a candidate rationale assumed erroneous because the answer does not. The paper explicitly notes that this labeling is heuristic, which motivates the weighting mechanism.
For each training tuple 5, two supervision channels are constructed. The answer-prediction channel uses 6 as input and 7 as target; the rationale-correction channel uses 8 as input and 9 as target (Xie et al., 6 Sep 2025). During training, the model computes per-sample rationale and answer losses. In early epochs, uniform weighting is used to establish basic capability; starting from epoch 0, the Correctness-Aware Weighted loss is activated. For LLaMA2-7B, the reported best temperature is 1, and the weighting begins at epoch 2; for TinyLLaMA, 3 (Xie et al., 6 Sep 2025).
The primary student in the paper is LLaMA2-7B, with additional experiments on TinyLLaMA-1.1B, LLaMA2-13B, and Mistral-7B-v0.2. Fine-tuning uses LoRA with rank 4, 5, target modules 6 and 7, and dropout 8. Optimization uses AdamW with 9, 0, no weight decay, and scheduler 1; training runs on 2 A800 GPUs with gradient accumulation 3, per-device batch size 4, and maximum input length 5 (Xie et al., 6 Sep 2025).
The benchmarks span both in-distribution and out-of-distribution settings. The in-distribution data are BIG-Bench Hard (BBH), split 4:1 into BBH-train and BBH-test. Out-of-distribution evaluation uses BB-sub, AGIEval English multiple-choice subsets, ARC-Easy, and ARC-Challenge (Xie et al., 6 Sep 2025). This distributional setup is important because CoPeD is presented not only as a faithfulness intervention but as a generalization intervention: by weakening spurious question–answer shortcuts, the method aims to improve robustness across task families.
SCOTT and EDIT provide alternative data-construction patterns that are often discussed alongside CoPeD. SCOTT uses contrastively decoded teacher rationales and counterfactual rationales produced under wrong-answer conditioning (Wang et al., 2023). EDIT constructs dual sets 6 and 7 through rectification of wrong chains and corruption of correct chains, ultimately producing 8 dual CoTs, with 9 and 0 (Dai et al., 2024). These alternatives illustrate a shared design pattern: correctness perception becomes learnable when training data expose contrast between supportive and unsupportive reasoning rather than presenting single chains in isolation.
5. Evaluation methodology and empirical findings
The CoPeD paper evaluates three broad properties: answer performance, rationale faithfulness, and rationale soundness (Xie et al., 6 Sep 2025). Faithfulness is defined as whether the student’s rationale supports its own predicted answer; soundness is whether the rationale supports the ground-truth answer. Both are measured using binary LLM evaluators, with ChatGPT and GPT-4 used to mitigate single-model bias. Formally, with evaluator 1,
2
3
These metrics are central because CoPeD’s stated objective is not just accuracy but better causal alignment between rationale and answer.
For LLaMA2-7B, average performance across BBH-test, BB-sub, AGIEval, ARC-E, and ARC-C is reported as 4 for CasCoD, 5 for CoPeD-T, 6 for CoPeD-L, and 7 for CoPeD-TL (Xie et al., 6 Sep 2025). The strongest variant, CoPeD-TL, improves over CasCoD by 8 overall and by 9 on the in-distribution BBH-test. The paper also reports OOD gains of 0 on AGIEval, 1 on ARC-E, and 2 on ARC-C relative to the strongest baseline (Xie et al., 6 Sep 2025).
On rationale quality, CoPeD-TL raises average faithfulness to 3, compared with 4 for Std-CoT and 5 for CasCoD; average soundness rises to 6, compared with 7 for Std-CoT and 8 for CasCoD (Xie et al., 6 Sep 2025). These results support the paper’s claim that correctness-aware task setting and weighting reduce shortcut reliance.
The ablations isolate the two weighting components. On BBH-test, Uniform weighting yields 9, “composite only” yields 0, “discrepancy only” yields 1, and full CoPeD-L yields 2 (Xie et al., 6 Sep 2025). This indicates that both overall sample quality and rationale–answer alignment matter. Temperature sweeps show the best BBH-test performance at 3, with 4, and task balance sweeps show best average performance at 5 (Xie et al., 6 Sep 2025).
Data-efficiency results are also notable. With only 6 of BBH-train, CoPeD-TL improves over CasCoD by 7 on BBH-test and 8 on BB-sub (Xie et al., 6 Sep 2025). This is consistent with the claim that correctness perception is not merely a regularizer for high-data settings but a mechanism for using scarce distillation data more productively.
Other papers provide complementary evaluation frameworks. SCOTT uses Leakage-Adjusted Simulatability (LAS), perturbation sensitivity, and rationale refinement gains to assess whether predictions genuinely depend on rationales (Wang et al., 2023). The paper reports that contrastive decoding improves teacher LAS over greedy and human annotations, that counterfactual training raises student LAS, and that SCOTT students show larger performance drops under rationale perturbations and larger gains from oracle rationale refinement (Wang et al., 2023). EDIT evaluates mostly by answer accuracy and shows average gains from 9 for Std-CoT to $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$0 for EDIT across BBH-test, BB-sub, AGIEval, ARC-E, and ARC-C, with particularly large differences on ARC-C (Dai et al., 2024). Chain-of-Probe adds CoT Accuracy and True Answer with False CoT Ratio, reporting average TAFCR $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$1 on ARC-Challenge and about $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$2 average improvement in CoT correctness from greedy sampling to CoP Tree resampling (Wang et al., 2024).
Taken together, these evaluations suggest that CoPeD-like systems require multi-axis assessment: end-task accuracy alone cannot reveal whether the student has actually learned correctness perception.
6. Common misconceptions, limitations, and later critiques
A common misconception is that if a teacher answer is correct, its rationale is a valid supervision signal. The CoPeD paper explicitly rejects this assumption: heuristic correctness labels are noisy, and the weighting mechanism is introduced precisely because a correct answer can co-occur with a flawed rationale (Xie et al., 6 Sep 2025). Chain-of-Probe reinforces the point by showing that many correct responses still contain reasoning errors, using the metric TAFCR and GPT-4-based reasoning annotations (Wang et al., 2024).
A second misconception is that higher answer accuracy implies better reasoning traces. The medical audit literature directly contradicts this inference. On MedQA-USMLE, a Qwen3-8B student distilled from a DeepSeek-V3-family teacher improves SC@64 accuracy from $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$3 to $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$4, reduces ECE from $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$5 to $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$6, and reduces Brier score from $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$7 to $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$8, yet its step-level factual error rate over non-abstained steps rises from $\mathcal{L}_{\text{ra}} \;=\; \mathbb{E}_{(q,r^+,a)\sim\mathcal{D}_{\text{train}}} \left[ \ell\Big(q,\, r^+\oplus rs_{\mathrm{t}\oplus a\Big) \right].$9 to 00 under a style-blind Kimi-K2.6 audit (Jiang et al., 27 May 2026). The same paper reports that this before–after pattern persists across evaluators, answer-conditioned controls, style and segmentation controls, and multiple student families (Jiang et al., 27 May 2026). This does not refute CoPeD; rather, it shows what happens when correctness perception is not made an explicit training target.
A third misconception is that rationales are necessarily needed for prediction. Chain-of-Probe’s Early Answering analysis shows that CoT is often unnecessary, especially on simpler tasks, and that accuracy during EA is significantly higher than when the model changes its mind, with the difference generally exceeding 01 (Wang et al., 2024). This suggests that one should distinguish rationale usefulness from rationale presence: CoPeD is concerned with the former.
The methods also have practical and conceptual limitations. CoPeD depends on teacher quality and heuristic labeling via teacher answer correctness, though weighted loss mitigates the resulting noise (Xie et al., 6 Sep 2025). SCOTT incurs extra teacher-side forward passes during contrastive decoding and extra training cost from counterfactual examples (Wang et al., 2023). EDIT relies on dual CoTs of sufficiently similar form for edit alignment; the paper notes that minimum edit distance is a surface alignment and may miss semantic equivalence or paraphrase (Dai et al., 2024). Chain-of-Probe currently targets single-token answer sets and depends on step segmentation quality (Wang et al., 2024). Program-aided approaches have limited coverage for commonsense and knowledge-heavy tasks lacking clean executable structure (Zhu et al., 2023). The medical audit literature adds a broader warning: even when internal signals or hedging cues remain decodable, they may be miscalibrated to a new reliability baseline after distillation (Jiang et al., 27 May 2026).
A plausible implication is that “correctness perception” should not be treated as solved by any one proxy. Heuristic answer matching, simulator-based faithfulness, LLM-as-judge audits, perturbation sensitivity, and executable verification each measure different slices of the underlying property.
7. Broader significance and future directions
CoPeD is significant because it reorients CoT distillation away from trace imitation and toward process-sensitive supervision. In the standard paradigm, rationales are often treated as privileged labels. CoPeD and related work instead treat them as objects to be validated, contrasted, weighted, revised, or verified before they can serve as supervision (Xie et al., 6 Sep 2025). This shift aligns with broader process-supervision agendas while remaining tightly grounded in distillation.
Several concrete trajectories are already visible in the literature. SCOTT suggests a counterfactual route in which correctness perception is learned through answer-conditioned rationale generation and rationale-dependent answer switching (Wang et al., 2023). EDIT suggests a sparsity route in which training focuses on the few tokens or steps that actually determine correctness (Dai et al., 2024). Chain-of-Probe suggests an introspective route in which confidence trajectories, Early Answering, and mind-change patterns become teacher signals for process-level distillation (Wang et al., 2024). Program-aided methods suggest a verification route in which correctness perception is grounded in execution where possible (Zhu et al., 2023). The medical auditing work suggests an auditing route in which any CoT distillation system, especially in high-stakes domains, should report step-level factuality under style-blind evaluation rather than relying on answer metrics alone (Jiang et al., 27 May 2026).
Two later developments extend the design space further. CoT-Evo, in scientific reasoning, evolves reasoning trajectories before distillation using a fitness
02
with 03 and 04, combining exact-match correctness, length appropriateness as a coherence proxy, and knowledge-usage correctness (Feng et al., 15 Oct 2025). Although not framed as CoPeD originally, the method explicitly connects itself to correctness perception by refining trajectories for correctness, coherence, and knowledge grounding before student fine-tuning. This suggests that CoPeD can be extended beyond dual-rationale or weighted-loss settings into data-synthesis pipelines that optimize rationale quality pre-distillation.
The most immediate future direction, however, is methodological rigor. The medical audit paper argues that step-level factuality supervision, judge diversity, and reporting standards should become explicit components of distillation pipelines (Jiang et al., 27 May 2026). This suggests that future CoPeD systems may require hybrid objectives combining answer loss, step-level factuality loss, and calibration loss, together with evaluation protocols that report answer accuracy, faithfulness, soundness, perturbation sensitivity, refinement gains, and audited step correctness. Such a program would move CoPeD from being mainly a remedy for spurious question–answer correlations into a more general framework for trustworthy process distillation.
In summary, Chain-of-Thought Correctness Perception Distillation names a transition in reasoning distillation research: from asking whether students can reproduce answers and rationale style, to asking whether they can perceive which reasoning is correctness-supporting, revise what is not, and remain sensitive to the causal role of rationale content. The empirical record across SCOTT, EDIT, Chain-of-Probe, CoPeD, CoT-Evo, and step-level medical audits indicates that this transition is necessary if distilled reasoning traces are to be treated as more than rhetorically plausible artifacts (Wang et al., 2023, Dai et al., 2024, Wang et al., 2024, Xie et al., 6 Sep 2025, Feng et al., 15 Oct 2025, Jiang et al., 27 May 2026).