Establish Whether On-Policy Self-Distillation Provides Reliable Supervision

Determine whether the teacher distribution constructed by On-Policy Self-Distillation (OPSD) provides reliable supervision on student-generated trajectories.

Background

On-Policy Self-Distillation replaces an external teacher with the same policy conditioned on additional hints. Because the resulting teacher distribution is evaluated on trajectories generated by the student, the reliability of its supervision remains unresolved. The paper identifies this as an open issue for OPSD, distinct from its own analysis of the mechanisms underlying standard On-Policy Distillation.

References

However, OPSD retains the OPD training paradigm, leaving unresolved whether the constructed teacher can provide reliable supervision on student-generated trajectories.

— Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement  (2608.31046 - Ding et al., 31 Aug 2026) in Section “Related Works,” paragraph “Supervision Signals in On-Policy Distillation”

The aggregate results do not isolate how often each regime occurs. Future work should report this breakdown and examine whether the closed-gate policy should vary by domain.

— Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation  (2609.02998 - Zhang et al., 2 Sep 2026) in Section 5.2, “Ablation: What the Gate Does After It Closes,” final paragraph before Section 6