Selective Reflection Distillation (SRD)

• Selective Reflection Distillation (SRD) is a data-centric, plug-and-play white-box KD framework that curates training samples based on student reflections.
• It refines data quality by ranking examples using ROUGE-L and cross-entropy metrics, retaining the easiest 75% of samples and applying a baby-step curriculum.
• SRD enhances model performance and training efficiency by jointly improving distillation outcomes and reducing runtime across diverse tasks and architectures.

Selective Reflection Distillation (SRD) is a data-centric, plug-and-play framework for white-box knowledge distillation (KD) of LLMs that treats training-data selection and training order as first-class distillation variables. Instead of modifying the underlying divergence objective, SRD uses the student model’s own reflections on prompt-response pairs to curate a cleaner, more student-compatible subset of the training set, then introduces that curated data through an easy-to-hard curriculum. In the reported formulation, SRD is designed to improve distillation outcomes across diverse white-box KD approaches and model architectures while reducing computational cost during KD training (Liu et al., 8 Aug 2025).

1. Concept and problem setting

SRD is motivated by the claim that prior white-box KD methods focus too narrowly on the divergence loss between teacher and student—such as KLD, RKL, JSD, SKL, and SRKL—while neglecting two crucial factors: training data quality and student-model compatibility. The framework starts from the observation that many KD pipelines use all available prompt-response pairs indiscriminately, even though some samples are too hard, noisy, or poorly aligned with the student’s current capability. It also emphasizes that a sample may be correct as ground truth yet still be too difficult or mismatched for the student to learn from effectively, especially under a large teacher-student capacity gap (Liu et al., 8 Aug 2025).

The paper situates this problem in the classic training-inference mismatch of white-box KD. If the student is required to imitate a complex teacher distribution on hard samples from the beginning of training, optimization can become unstable and transfers can be ineffective. Earlier attempts to improve compatibility often rely on student-generated outputs or on-policy data, but the paper identifies two drawbacks: high computational cost and misguidance early in training. It explicitly notes that generating student outputs during training can consume a large share of runtime, even up to 80% in some settings, and that low-quality student outputs can inject noise when the student is still weak (Liu et al., 8 Aug 2025).

Against this background, SRD is presented not as a new divergence loss, but as a remedy based on one-time sample reflection and staged exposure. The central hypothesis is that distillation quality depends not only on how teacher and student distributions are aligned, but also on which training samples are used and when they are introduced.

2. Two-stage SRD pipeline

SRD is organized into two stages. The first stage is selective reflection on training data. Given a dataset of prompt-response pairs $D=\{(x,y)\}$, the student model $M_s$ generates a response $y_s$ for each prompt $x$. The student’s behavior on that sample is treated as a reflection of sample difficulty. Two signals are then extracted: ROUGE-L between the ground truth $y$ and the student output $y_s$, and cross-entropy loss on the student-generated output. Higher ROUGE-L indicates that the student output is closer to the reference and hence that the sample is easier or more compatible; lower loss indicates higher student confidence and therefore lower difficulty (Liu et al., 8 Aug 2025).

Each example is assigned two ranked positions, $\phi_r$ from ROUGE-L and $\phi_e$ from cross-entropy loss. These are fused into a unified difficulty ranking using reciprocal rank fusion. After ranking, the hardest examples are removed and only the top fraction is retained. The curated set is written as

$$D'' = \left\{ s_i \,\middle|\, i \le \lambda N \right\},$$

where $N$ is the total number of samples and $M_s$0 is the retention ratio. In the reported experiments, $M_s$1, so SRD keeps the easiest 75% and drops the hardest 25% (Liu et al., 8 Aug 2025).

The second stage is curriculum scheduling. After curation, the retained set $M_s$2 is partitioned into $M_s$3 subsets of increasing difficulty,

$M_s$4

Each subset contains roughly $M_s$5 samples. Training then follows a Baby Step curriculum: the student begins with the easiest subset $M_s$6, and at stage $M_s$7 trains on

$M_s$8

This makes SRD a joint method for filtering and ordering rather than a pure sample-pruning heuristic (Liu et al., 8 Aug 2025).

3. Ranking, difficulty estimation, and scheduling mechanics

The ranking core of SRD is reciprocal rank fusion (RRF). If a sample has rank $M_s$9 in ranking list $y_s$0, its fused score is

$y_s$1

with $y_s$2 used as a stabilizing constant. A higher fused score denotes an easier and more suitable sample. The reported ablation shows that the fused ROUGE-L + loss ranking performs better than ROUGE-L alone or loss alone, supporting the claim that complementary student-reflection signals provide a more reliable estimate of difficulty (Liu et al., 8 Aug 2025).

The curriculum is discrete and staged rather than continuous. The paper describes this explicitly as a Baby Step curriculum from easy to hard. It is paired with adaptive schedules for the KD temperature $y_s$3 and supervised fine-tuning weight $y_s$4: $y_s$5

$y_s$6

In the reported experiments, $y_s$7, $y_s$8, $y_s$9, and $x$0. The intended interpretation is that the student first learns from sharper targets and stronger ground-truth supervision, then gradually shifts toward softer teacher distributions and a larger relative KD contribution (Liu et al., 8 Aug 2025).

The training loss remains the standard white-box KD objective,

$x$1

with

$x$2

and

$x$3

The appendix also gives a form with $x$4 scaling depending on the section. A common misconception is that SRD changes the KD divergence itself; the paper instead defines SRD as a wrapper around existing KD losses that changes sample selection, ordering, and schedule parameters rather than the base objective (Liu et al., 8 Aug 2025).

4. Integration with white-box knowledge distillation

SRD is designed to be compatible with both off-policy and on-policy white-box KD. The paper states that it can be combined with off-policy KD losses such as KLD, RKL, JSD, TVD, SKL, and SRKL, as well as with on-policy KD such as GKD. The underlying loss remains intact; SRD only changes which samples are used, in what order they are introduced, and how temperature and supervised fine-tuning weight evolve during training. This is the operational meaning of its plug-and-play characterization (Liu et al., 8 Aug 2025).

The experimental study spans four model families and five task categories.

Component	Reported setup
Model families	GPT-2, OpenLLaMA2, T5 / mT5, Qwen2.5
Instruction following	databricks-dolly-15k; DollyEval, SelfInst, VicunaEval, Super-NI, UnNI
Summarization	XSum
Translation	IWSLT 2017 En-De
Mathematical reasoning	GSM8K
Code generation	MBPP

For model scaling, the paper reports GPT-2 teacher 1.5B with student 0.1B, and OpenLLaMA2 teacher 7B with student 3B. Evaluation uses ROUGE-L for instruction following and summarization, BLEU for translation, and Pass@1 for math reasoning and code generation. Instruction-following results are averaged over five random seeds, and training and evaluation are performed on A100 GPUs (Liu et al., 8 Aug 2025).

Because SRD is external to the KD divergence, it can be interpreted as a training-data and schedule layer around white-box distillation rather than a substitute for established KD formulations. This distinction is central to the method’s scope.

5. Empirical findings and computational profile

The main empirical claim is that SRD improves every baseline across essentially all settings examined. For instruction following on OpenLLaMA2, SRD improves the average ROUGE-L across benchmarks for all KD losses, with gains ranging roughly from about $x$5 to $x$6 depending on the baseline. The strongest reported result is KLD + SRD with $x$7 average improvement. Runtime is reduced by about 35% for offline KD settings and 39.07% for on-policy GKD (Liu et al., 8 Aug 2025).

For GPT-2, the improvements are smaller but still consistent. The paper again reports gains for all KD methods and runtime reductions of about 33–38%. Across XSum, IWSLT, GSM8K, and MBPP, SRD consistently improves over the corresponding baseline for almost all methods; gains are strongest on summarization and translation, and more modest on mathematical reasoning and code generation. The latter pattern is important for interpreting the method’s domain dependence rather than assuming uniform benefits across task types (Liu et al., 8 Aug 2025).

The runtime analysis attributes the speedup to three quantitative factors. SRD uses only 75% of the training data, trains in staged curricula totaling about 60% of the baseline training steps, and incurs a reflection/curation cost of about 5–7% of baseline runtime. On that basis, the overall speedup reaches up to 39.07%. The paper further compares SRD with a traditional curriculum learning baseline that uses handcrafted difficulty features and no data filtering; that baseline yields only marginal improvement, whereas SRD gives much larger gains. This comparison is used to argue that student-informed reflection plus selective filtering is more effective than static proxy-based curriculum design (Liu et al., 8 Aug 2025).

A second common misconception is that SRD is merely a speed-oriented pruning trick. The reported results are explicitly framed as joint quality and efficiency gains: improved distilled model performance together with reduced training runtime.

6. Limitations, assumptions, and open directions

The paper is explicit that SRD works best when similarity metrics such as ROUGE-L and cross-entropy are meaningful proxies for correctness, difficulty, and compatibility. Its performance is more modest on mathematical reasoning and code generation because a tiny arithmetic mistake or syntax error can invalidate an output even when ROUGE-L or token loss still indicates similarity to the target. In those settings, the difficulty estimator may overrate superficially similar but functionally incorrect samples (Liu et al., 8 Aug 2025).

The stated assumptions are correspondingly narrow. SRD assumes that a student model can generate meaningful reflections on training examples, that ROUGE-L and cross-entropy are useful proxies for difficulty and compatibility in the task domain, and that the retained 75% subset preserves most of the useful knowledge. These assumptions are not claimed to hold universally (Liu et al., 8 Aug 2025).

The paper suggests several future directions: execution-based feedback for code, mathematical answer verification, semantic equivalence scoring, task-adaptive difficulty metrics, and adaptive thresholds for filtering. These are presented as natural extensions because the current difficulty estimator is fundamentally lexical and confidence-based. A plausible implication is that SRD’s effectiveness depends not only on model capacity gap and KD loss, but also on the fidelity of the reflection signals to task-specific correctness.

7. Broader selective-distillation context and terminological disambiguation

Within the broader literature on selective KD for autoregressive LLMs, a useful context is the framework that disentangles selection along the position, class, and sample axes and shows that dense supervision is not always optimal (Tavor et al., 1 Feb 2026). That work introduces student-entropy-guided position selection (SE-KD), extends it across the class and sample axes, and reports that simple deterministic Top-$x$8 selection is the most effective and stable policy overall. This suggests that SRD can be read as a sample-level selection and curriculum method within a wider selective-distillation design space, even though SRD itself is framed specifically as student-reflection-based data curation rather than position-level or class-level sparsification (Tavor et al., 1 Feb 2026).

The terminology also invites a potential confusion with several unrelated optical uses of “selective reflection.” In spectroscopy and optical physics, selective reflection refers to phenomena such as high-resolution probing of gases near dielectric windows, Bragg reflection in oblique helicoidal cholesteric cells, Fano-like asymmetric resonance at resonant interfaces, and selective reflection from dilute Fabry–Perot vapor cells (Dutta et al., 8 Jul 2025, Iadlovska et al., 2018, Novitsky, 2011, Khachatryan, 2018). SRD is unrelated to those optical meanings. Here, “reflection” denotes the student model’s responses on training samples and their use as signals for dataset curation.

In that sense, SRD reframes white-box distillation as a data selection and scheduling problem as much as a loss-design problem. It uses the student itself to estimate which ground-truth samples are easiest and most compatible, retains those samples, and introduces them progressively during training. The method’s central claim is therefore not that a new divergence objective is required, but that effective and efficient KD depends materially on data quality, compatibility, and training order (Liu et al., 8 Aug 2025).