Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

Abstract: Self-distillation has emerged as an effective post-training paradigm for LLMs, often improving performance while shortening reasoning traces. However, in mathematical reasoning, we find that it can reduce response length while degrading performance. We trace this degradation to the suppression of epistemic verbalization - the model's expression of uncertainty during reasoning. Through controlled experiments varying conditioning context richness and task coverage, we show that conditioning the teacher on rich information suppresses uncertainty expression, enabling rapid in-domain optimization with limited task coverage but harming OOD performance, where unseen problems benefit from expressing uncertainty and adjusting accordingly. Across Qwen3-8B, DeepSeek-Distill-Qwen-7B, and Olmo3-7B-Instruct, we observe performance drops of up to 40%. Our findings highlight that exposing appropriate levels of uncertainty is crucial for robust reasoning and underscore the importance of optimizing reasoning behavior beyond merely reinforcing correct answer traces.

• The paper reveals that self-distillation suppresses epistemic verbalization, leading to up to 40% accuracy losses on out-of-distribution math benchmarks.
• It demonstrates that context-rich teacher signals compress uncertainty and shorten responses, which impedes robust generalization in diverse tasks.
• The study advocates for uncertainty-aware training objectives to mitigate brittle, overconfident reasoning in large language models.

Analysis of Self-Distillation-Induced Degradation in LLM Mathematical Reasoning

Abstract and Motivation

This paper investigates the mechanisms underlying the observed degradation of mathematical reasoning capabilities in LLMs subjected to self-distillation-based post-training. The authors demonstrate that—contrary to prior observations in chemical and scientific domains—self-distillation can suppress epistemic verbalization and thereby harm performance, particularly in mathematical contexts. Through controlled experiments across multiple LLM architectures and datasets, the paper traces performance degradation to the teacher’s context richness and the resultant compression of uncertainty signals. The implications are broad, challenging established views on the efficacy of post-training objectives focused solely on answer correctness and proposing a nuanced understanding of uncertainty-aware reasoning for robust generalization.

Figure 1: In Chemistry reasoning (Olmo3-7B-Instruct), self-distillation promotes concise, high-scoring responses; the same does not hold for Math tasks.

Information-Theoretic Perspective on Self-Distillation

Self-distillation employs the same model instance as both teacher and student, with the teacher receiving an information-rich context (e.g., ground-truth solutions). Training minimizes the divergence between student and teacher distributions, distilling reasoning strategies discovered by the teacher. The paper formalizes the informativeness of the teacher’s context via conditional mutual information: as $I(y;c|x)$ increases, the model's output becomes more confident and abbreviated, but at the cost of suppressed epistemic verbalization.

Figure 2: Exemplification of epistemic verbalization—expressions like "wait" and "maybe" preserve uncertainty during stepwise reasoning.

Empirical Evaluation: Reasoning and Epistemic Expression

Controlled studies on DAPO-Math-17k and open-weight LLMs (DeepSeek-R1-Distill-Qwen-7B, Qwen3-8B variants, Olmo3-7B-Instruct) reveal that training with context-rich teacher signals reduces response length and sharply diminishes epistemic token usage. This is shown rigorously via token-level analysis, where the suppression effect is monotonic in $I(y;c|x)$.

Figure 3: Suppression of epistemic tokens (“wait,” “maybe,” “perhaps”) correlates with increased informational context.

Figure 4: Model comparison showing epistemic token usage difference as a function of “thinking mode” setting.

Notably, off-policy SFT using solution-guided traces (low epistemic density) leads to strong performance degradation—even though all training targets are correct. In contrast, training on unguided traces retains epistemic signals and maintains benchmark performance.

On-Policy Self-Distillation: Training Dynamics and Out-of-Distribution Evaluation

The paper extensively benchmarks GRPO vs. SDPO (self-distillation) training regimes. With full teacher context, SDPO demonstrates dramatic response length reduction and epistemic suppression, yielding up to 40% accuracy losses on OOD mathematical benchmarks (AIME24, AMC23). Conversely, GRPO preserves and even increases epistemic verbalization, modestly improving or stabilizing OOD outcomes.

Figure 5: DeepSeek-Distill-7B training score and response length dynamics; self-distillation produces shorter, less accurate responses in math.

Figure 6: DeepSeek-Distill-7B: Fixed vs. moving target teacher; moving target amplifies epistemic suppression and performance loss.

Teacher updates (EMA) intensify suppression via a feedback loop, showing that even mild teacher policy drift further degrades uncertainty signaling and, subsequently, generalization performance.

Task Coverage, Epistemic Verbalization, and Generalization

A salient contribution is the elucidation of the relationship between task coverage and the value of epistemic verbalization. In small, homogeneous task sets (e.g., Chemistry), self-distillation rapidly achieves optimal, concise reasoning. When task coverage is broad (Math), the suppression of uncertainty impedes the model's ability to recover or adapt to unseen structures.

Figure 7: Training score and response length: SDPO is highly efficient for small task coverage but fails as diversity increases.

Figure 8: OOD evaluation shows SDPO's performance worsening with larger, more heterogeneous task sets, contrasting GRPO's stability.

Further token-count analysis confirms that GRPO increases epistemic expressions as coverage grows, while SDPO continues to suppress them.

Figure 9: Change in epistemic token counts on AIME24 shows SDPO's suppression failing to generalize for increasing $|\mathcal{D}|$.

Practical and Theoretical Implications

The findings invalidate the prevalent assumption that self-distillation uniformly enhances LLM performance through answer trace reinforcement. Instead, they reveal that compressed, “confident” reasoning only benefits settings where exploration and epistemic uncertainty are largely redundant. Mathematical reasoning is characterized by deep compositionality and frequent OOD evaluation, necessitating iterative belief revision and explicit uncertainty management.

This has direct implications for RL-based LLM post-training objectives: optimizing solely for correctness or brevity is insufficient for robust generalization. Future LLM training frameworks must quantify and preserve uncertainty-aware reasoning features, possibly integrating epistemic token density or reasoning trace entropy as regularization signals. Cross-domain generalization, agentic reasoning, and continual learning scenarios are likely to benefit from such approaches.

Figure 10: Hybrid vs. homogeneous distillation setup performance dynamics validate the domain-specific effectiveness of uncertainty signaling.

Conclusion

This work provides a detailed, rigorous analysis of the unintended consequences of self-distillation in LLMs on mathematical reasoning tasks. By formalizing the information-theoretic underpinnings and empirically demonstrating the suppression of epistemic verbalization, the authors challenge current post-training paradigms. The study suggests that robust, generalizable LLM reasoning must leverage uncertainty signals, and that post-training objectives aimed at answer trace compression can produce brittle, overconfident reasoning. Future research should focus on integrating uncertainty-aware objectives and investigating the granularity of epistemic expression necessary for effective generalization across reasoning domains.