QR-Distill: Routing and Cooperative Distillation

• The paper introduces QR-Distill, a methodology that filters and routes valid reasoning traces to student models, achieving up to +24% performance gains.
• It employs a conditional routing mechanism using Gumbel Softmax and entropy regularization to dynamically tailor assignments for diverse learning states.
• The approach integrates cooperative peer teaching to align student representations, reducing knowledge gaps and avoiding overfitting during distillation.

Quality-filtered Routing with Cooperative Distillation (QR-Distill) is a methodology for distilling complex model reasoning into compact student models, centered on the selective transfer and collaborative integration of only high-quality reasoning or knowledge traces. QR-Distill filters candidate reasoning paths for their correctness and logical validity, routes filtered paths adaptively to student models based on their current learning states, and employs cooperative peer-teaching among student models to mitigate knowledge gaps and biases. This approach offers substantial performance improvements over traditional distillation paradigms, with broad applicability ranging from LLM reasoning to distributed and quantum systems.

1. Principles of Quality-filtered Routing

QR-Distill enforces a rigorous two-stage filtering process on the reasoning paths provided by large teacher models:

1. Incorrect Answer Removal: For each query $Q^{(i)}$, candidate reasoning paths $R_j^{(i)}$ are retained only if their final answer $\hat{A}_j^{(i)}$ matches the ground truth $A^{(i)}$.
2. LLM-based Internal Reasoning Evaluation: The filtered set is further processed by an LLM-based evaluation module ($\mathcal{J}$), which assesses the logical coherence and reliability of intermediate steps. Paths deemed spurious, illogical, or containing hallucinations are discarded.

Formally, the final cleaned path set for query $Q^{(i)}$ is denoted $\tilde{R}^{(i)}$, comprising only reasoning signals that are correct and pedagogically meaningful (Lei et al., 23 Aug 2025).

This ensures that only empirically verified and logically consistent learning signals are routed onward, reducing noise and erroneous supervision during student distillation.

2. Conditional Routing Mechanism

Each reasoning path $\tilde{R}_j^{(i)}$ is not universally assigned to all students; instead, QR-Distill utilizes a conditional routing mechanism:

• Representation Encoding: $\tilde{R}_j^{(i)}$ is embedded to a fixed-dimensional vector $$h_j^{(i)} = \mathrm{Enc}(\tilde{R}_j^{(i)}) \in \mathbb{R}^d$$.
• Adaptive Routing Decision: $R_j^{(i)}$0 passes through a trainable router (MLP), and assignment to student $R_j^{(i)}$1 is made probabilistically via Gumbel-Softmax sampling:

$R_j^{(i)}$2

where $R_j^{(i)}$3 is number of students; $R_j^{(i)}$4 indicates routing to student $R_j^{(i)}$5.

• Entropy Regularization: To avoid degenerate assignments (e.g., always routing to all students), the mean routing assignment $R_j^{(i)}$6 is entropy-regularized:

$R_j^{(i)}$7

encouraging balanced, dynamic assignment tailored to student diversity and capacity (Lei et al., 23 Aug 2025).

Conditional routing thus individualizes supervision, addressing idiosyncratic strengths and weaknesses of students similar, in principle, to the targeted transfer mechanisms in peer-distillation frameworks (Livanos et al., 2024).

3. Cooperative Peer Teaching and Mutual Distillation

To prevent students from overfitting to narrow reasoning styles, QR-Distill employs a cooperative distillation scheme:

• Peer Representation Extraction: Each student $R_j^{(i)}$8 generates hidden states $R_j^{(i)}$9 for tokens in assigned path $\hat{A}_j^{(i)}$0, projected to shared space via $\hat{A}_j^{(i)}$1.
• Competence Scoring: For each reasoning path, a linear regressor and softmax compute competence scores $\hat{A}_j^{(i)}$2 for all $\hat{A}_j^{(i)}$3 students.
• Ensemble Construction: The aggregate representation is

$\hat{A}_j^{(i)}$4

• Mutual Alignment: Each student minimizes MSE to the ensemble:

$\hat{A}_j^{(i)}$5

binding individual students to a collaboratively distilled representation.

This strategy distributes diverse insights and reduces representational deficits endemic to isolated distillation, approaching the collaborative regimes studied in peer-distillation and counterfactual augmentation (Livanos et al., 2024, Vats et al., 2021).

4. Experimental Performance and Ablation Analysis

Benchmarking QR-Distill against single- and multi-path distillation establishes quantitative advantages:

Method	Mistral-7B-Instruct Avg	Gemma-7B-Instruct Avg	Relative Improvement
QR-Distill	59.23	59.89	+24.32% (single)
Distill Step-by-Step (multi-path)	Lower	Lower	+13.36%
SKD	Lower	Lower	—

Ablation results show:

• Removing quality filtering (no QF) notably degrades performance (e.g., ARC drops from 80.25 to 77.98; Date from 73.37 to 66.86).
• Excluding routing (no Route) and collaboration (no Collab) each reduce performance, with peer teaching especially impactful for architecturally sensitive students (Lei et al., 23 Aug 2025).
• QR-Distill achieves comparable accuracy to full-data training using just 30% of samples under sample-efficient regimes.

This demonstrates the necessity of all QR-Distill components; the modular failure of filtering, routing, or peer teaching induces measurable performance collapse.

5. Connections to Related Methodologies

QR-Distill extends, hybridizes, and systematizes several recent knowledge distillation advances:

• Response-based Distillation Control: By focusing on response entropy and similarity cues, QR-Distill reflects findings that distillation efficacy depends on inter-class information, best preserved by teachers with carefully tuned training regimes (Vats et al., 2021).
• Curriculum/Route-constrained Supervision: Applying stepwise filtering and path selection presents parallels to route-constrained optimization and curriculum-based distillation, where intermediate learning states serve as easier-to-mimic anchors for students (Jin et al., 2019).
• Federated/Distributed Distillation: Principles of selective routing and cooperative knowledge transfer echo federated distillation protocols under communication constraints, employing regularization and mixed-up features to enhance learning in distributed settings (Ahn et al., 2020, Hu et al., 2024).
• Quantum Routing and Distillation: In quantum networks, quality-filtered routing can be formalized via route selection coupled with physical and protocol-layer distillation, optimized jointly for fidelity and delay (Chehimi et al., 2024).

6. Implementation and Code Availability

QR-Distill is implemented in Python using standard deep learning libraries (e.g., PyTorch), requiring:

• Pre-trained LLMs (e.g., Gemini-1.5, Mistral, Gemma)
• QLoRA fine-tuning dependencies
• Dataset processing utilities

Source code and installation instructions are provided at https://github.com/LzyFischer/Distill.

7. Applications and Future Directions

QR-Distill addresses bottlenecks in resource-constrained deployment of large reasoning models, enables privacy-preserving collaborative learning, and aligns student models with diverse, high-quality teacher insights. Potential extensions include:

• Generalizing routing and peer teaching mechanisms to multimodal and cross-architecture models (Livanos et al., 2024).
• Adapting robust quality filtering for distributed learning and quantum networking scenarios (Ahn et al., 2020, Chehimi et al., 2024).
• Evolving routing logic for multi-hop or multi-stage reasoning traces in ensemble learning and distributed inference (Hu et al., 2024).

A plausible implication is that broader adoption of QR-Distill principles may catalyze more reliable, scalable, and efficient knowledge transfer in heterogeneous AI, edge, and quantum networking environments.