DeepSeek-R1-Distilled: Open LLM Reasoning

Updated 29 October 2025

DeepSeek-R1-distilled models are dense LLMs distilled via supervised fine-tuning from teacher-generated reasoning traces to transfer multi-step chain-of-thought capabilities.
They leverage popular backbones like Qwen2.5 and Llama3 and achieve state-of-the-art performance on benchmarks in mathematics, logical reasoning, and coding.
Despite improved reasoning accuracy, challenges remain in adaptive chain length, overfitting to long CoT sequences, and ensuring robust safety and alignment.

DeepSeek-R1-distilled refers to a family of dense LLMs derived through supervised fine-tuning (SFT) using high-quality reasoning traces generated by the large reinforcement learning-based DeepSeek-R1 model. These models aim to transfer advanced step-by-step reasoning capability and chain-of-thought (CoT) behaviors to smaller, highly efficient architectures—making reasoning-empowered LLMs broadly accessible in open-source form. The DeepSeek-R1-distilled releases span multiple parameter scales (1.5B–70B) and are constructed atop widely adopted Qwen2.5 and Llama3 backbones. They stand out for their state-of-the-art performance on mathematics, logical reasoning, and code benchmarks, matching or surpassing proprietary alternatives at much lower inference cost.

1. Model Lineage and Distillation Framework

The DeepSeek-R1-distilled models are the result of a multi-stage transfer pipeline originating from the large DeepSeek-R1 reasoning model, itself trained via the Group Relative Policy Optimization (GRPO) reinforcement learning algorithm. The distillation framework includes:

Teacher Generation: DeepSeek-R1, trained initially with cold-start human-authored CoT data and then iteratively improved via RL and SFT, produces explicit, auditable reasoning trajectories paired with final answers.
Student Models: Parameter-efficient, dense Transformer-based architectures (Qwen2.5: 1.5B, 7B, 14B, 32B; Llama3: 8B, 70B) serve as distillation targets.
Distillation Process: Around 800,000 verified reasoning-rich trajectories—filtered for correctness, format, language consistency, and readability—are used to supervised fine-tune the student models. Only standard SFT is applied (no RL), ensuring tractability and broad usability (DeepSeek-AI et al., 22 Jan 2025).

This paradigm efficiently transmits high-quality, multi-step reasoning and emergent behaviors—such as self-verification, chain branching, reflection, and multi-hop deduction—into compact models where direct RL is typically ineffective.

2. Architecture and Training Methodology

The student models are strictly dense transformers—not Mixture-of-Experts (MoE)—to ease deployment and hardware support. Their architecture and training are characterized by:

Backbones: Qwen2.5 and Llama3, open-source Transformer implementations, chosen for widespread support and compatibility.
Supervised Fine-Tuning (SFT): Training objective

$\mathcal{L}_\text{SFT}(\theta) = -\mathbb{E}_{(q,c) \sim \mathcal{D}_\text{SFT}} \left[ \log \pi_\theta(c|q) \right]$

where $\mathcal{D}_\text{SFT}$ contains question/CoT-answer pairs from the deep teacher.

No RL in Students: Only verified, readable generations from DeepSeek-R1 are used; RL and reward design are restricted to teacher phase for size and stability reasons.
Data Composition: The distillation data include both reasoning (CoT) and non-reasoning samples; explicit chain demarcation (e.g., using >, ``) is enforced for interpretability.

Distinctly, distillation outperforms direct RL training on small models, as evidenced by large gains in reasoning accuracy and chain-of-thought structure (DeepSeek-AI et al., 22 Jan 2025).

3. Reasoning Capabilities, Benchmarks, and Comparative Performance

Distilled DeepSeek-R1 models inherit and demonstrate strong emergent reasoning properties, with empirical performance summarized in major evaluation benchmarks (DeepSeek-AI et al., 22 Jan 2025, Zhao et al., 16 Feb 2025, Zhan et al., 1 Mar 2025):

Model	AIME2024	MATH-500	GPQA-Diamond	LiveCodeBench
DeepSeek-R1-Distill-Qwen-1.5B	28.9	83.9	33.8	16.9
DeepSeek-R1-Distill-Qwen-7B	55.5	92.8	49.1	37.6
DeepSeek-R1-Distill-Qwen-14B	69.7	93.9	59.1	53.1
DeepSeek-R1-Distill-Qwen-32B	72.6	94.3	62.1	57.2
DeepSeek-R1-Distill-Llama-8B	50.4	89.1	49.0	39.6
DeepSeek-R1-Distill-Llama-70B	70.0	94.5	65.2	57.5

Logical Reasoning: Consistent gains over non-distilled models, especially in mathematics (AIME, MATH-500), code, and planning tasks. Smaller, previously math-specialized models benefit the most.
Task-specific Trends: Distillation yields maximum uplift in logical/mathematical tasks but occasionally presents trade-offs, such as marginal dips in text understanding or generation for certain model scales (Zhao et al., 16 Feb 2025).
Scaling Law: Performance follows empirical scaling laws; larger distilled models typically outperform smaller ones, but distillation’s relative benefit is most pronounced for small/medium models.

In domain-specific settings, such as biomedical NLP, DeepSeek-R1-distilled models achieve top-tier F1-scores in named entity recognition and text classification (>0.95), and robust, balanced scores in extraction tasks (Zhan et al., 1 Mar 2025). In medical verticals, quantized, LoRA-adapted DeepSeek-R1-distill variants achieve near-SOTA diagnostic accuracy at dramatic reductions in memory and latency (e.g., 92.1% on USMLE Step 1 at <5.3GB GPU load) (Zhang et al., 25 Apr 2025).

4. Mechanistic Insights and Explicit Reasoning Dependence

Explicit reasoning traces in DeepSeek-R1-distilled models are not post-hoc explanations but functional intermediaries:

Empirical Benefit: Chain-of-thought output, demarcated during training and inference, directly improves accuracy (e.g., ∼10% gain in MATH-500) compared to suppressing reasoning segments (Zhang et al., 28 Sep 2025).
Mechanistic Dependence: Attention analysis reveals that answer tokens in mid-layers attend strongly to reasoning tokens—including self-reflection and verification cues—via Reasoning-Focus Heads (RFHs), suggesting a computational implementation of solution tracing and self-checking.
Causal Evidence: Interventions ("activation patching") at reasoning steps can reliably flip the final answers, demonstrating that answer token generation functionally depends on intermediate reasoning states.
Interpretability: These mechanistic features facilitate traceability and model debugging, as missteps in answer tokens can often be linked through attention to their origin in faulty reasoning substeps.

5. Safety, Robustness, and Alignment

While DeepSeek-R1-distilled models confer state-of-the-art reasoning, safety alignment is a recognized challenge:

Safety Regression via Distillation: Systematic evaluation reveals that distillation causes measurable declines in risk identification accuracy and safe refusal rates, especially in discrimination and values-based areas (Zhang et al., 18 Mar 2025). This effect is more pronounced for larger models.
Targeted Remediation: Supplementary safety alignment, via in-distribution supervised fine-tuning on custom refusal-style reasoning trajectories, achieves substantial gains in all safety metrics without degrading reasoning accuracy (e.g., refusal rates on harmful prompts increase from ∼26% to >81%) (Zhang et al., 14 Apr 2025, Zhang et al., 18 Mar 2025).
Hybrid Alignment: Pure RL-based safety suffers from reward hacking, generalization failure, and language mixing; supervised fine-tuning on curated safety datasets provides a more robust harmlessness baseline for the distilled models (Parmar et al., 28 Jan 2025). The synergy of RL (in teacher phase) and SFT (student) is advocated.

6. Data Quality, Distillation Source, and Open Replication

Outcomes in student models are highly sensitive to the quality and diversity of teacher-generated traces:

Data Verification and Diversity: Distilled models derived from DeepSeek-R1-generated outputs typically show strong but not top-tier performance if those outputs lack length diversity or adaptive length scaling (Tian et al., 20 May 2025). Datasets derived from alternative teacher models with more adaptive, diverse, and difficult reasoning traces (e.g., AM-Thinking-v1) yield students that outperform DeepSeek-R1-distilled students, especially in difficult math reasoning tasks. Not all correct chains are equally informative for downstream learning.
Open Datasets and Replication: The release of massive, LLM-verified datasets—e.g., AM-DeepSeek-R1-Distilled (1.4M entries)—enables robust SFT, modular distillation, and open replication benchmarks (Zhao et al., 25 Mar 2025, Zhang et al., 1 May 2025). Replication studies have shown that with careful dataset curation and RL from verifiable rewards (RLVR), open-source models can match or exceed the original DeepSeek-R1-distilled models on key metrics.

Dataset/Model	Size	Reasoning Chains	Benchmarks ↑	Student Perf. (AIME24)
DeepSeek-R1-distilled	0.8M	Yes	Math/Coding	70.9
AM-DeepSeek-R1-Distilled	1.4M	Yes (All)	Math/Science	72.7
AM-Thinking-v1-distilled	1.8M	Yes (Diverse)	Reasoning	84.3

Advanced Distillation: Recent frameworks (e.g., REDI) show that using both positive and negative reasoned traces—rather than filtered positives alone—can further improve student models’ data efficiency and accuracy, matching the result of DeepSeek-R1-Distill-Qwen-1.5B with 1/6 the data (Xu et al., 30 May 2025).

7. Limitations, Bottlenecks, and Directions for Improvement

Despite their advances, DeepSeek-R1-distilled models face recognized limitations:

Distillation Bottleneck: Small models struggle to learn from excessively long or formalistic teacher traces, often inheriting overthinking and hallucination tendencies. Plain SFT on distilled long CoTs can even impair solution rates in small models (Yin et al., 3 Mar 2025). Mitigations include constructing tree-based CoT data (e.g., via MCTS) and applying CoT-aware post-training (Thoughts Length Balance, Conservative DPO, joint SFT+DPO).
Adaptive Reasoning: DeepSeek-R1-distilled data delivers high fluency but less adaptive output length and less diversity compared to top-performing teacher models. As a result, students may be less flexible on tasks with variable complexity (Tian et al., 20 May 2025).
Scaling and Context: Reasoning performance declines with increased problem complexity or context length due to token window constraints and accumulation of incoherence in CoT chains (So et al., 29 Jun 2025).
Faithfulness and Efficiency: Chains may become ruminative or divergent, and controlling reasoning length via prompting remains ineffective. RL-based length penalties can trade accuracy for brevity (Marjanović et al., 2 Apr 2025).

Summary Table: DeepSeek-R1-distilled – Strengths and Limitations

Aspect	Strength	Limitation
Reasoning	SOTA for math/code/planning; explicit CoT	Declines in text understanding/generation (in some)
Safety	Remediable with in-distribution SFT-alignment	Safety loss via distillation (discrimination, refusal)
Efficiency	SFT-only, no RL for small models; dense	Small models—struggle with long-chain CoT; overthinking/hallucination
Data	High accuracy, interpretable chains	Lower output diversity/adaptivity vs AM-based distillation
Transferability	Effective to broad domains (biomed/medical)	Faithfulness and meta-cognitive control limited
Open Research	Benchmarks, data, models are open and extensible	Partial closure of original configs; decontamination challenges

DeepSeek-R1-distilled models represent a pivotal advance in reasoning-centric LLM distillation. By concentrating teacher-generated high-fidelity reasoning traces into accessible dense architectures, and coupling with ongoing innovations in dataset quality, safety alignment, and fine-grained distillation objectives, these models set a practical and empirical standard for open reasoning LLM research. Persistent challenges—including distillation bottlenecks in small models, length adaptivity, and robust safety/faithfulness—define the active frontiers for the field.