Establish Whether On-Policy Self-Distillation Provides Reliable Supervision

Determine whether the teacher distribution constructed by On-Policy Self-Distillation (OPSD) provides reliable supervision on student-generated trajectories.

Background

On-Policy Self-Distillation replaces an external teacher with the same policy conditioned on additional hints. Because the resulting teacher distribution is evaluated on trajectories generated by the student, the reliability of its supervision remains unresolved. The paper identifies this as an open issue for OPSD, distinct from its own analysis of the mechanisms underlying standard On-Policy Distillation.

References

However, OPSD retains the OPD training paradigm, leaving unresolved whether the constructed teacher can provide reliable supervision on student-generated trajectories.

— Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement  (2608.31046 - Ding et al., 31 Aug 2026) in Section “Related Works,” paragraph “Supervision Signals in On-Policy Distillation”

The aggregate results do not isolate how often each regime occurs. Future work should report this breakdown and examine whether the closed-gate policy should vary by domain.

— Verify Before You Distill: Prompt-Level Teacher Gating for On-Policy Distillation  (2609.02998 - Zhang et al., 2 Sep 2026) in Section 5.2, “Ablation: What the Gate Does After It Closes,” final paragraph before Section 6

Existing approaches enrich teacher predictions through privileged context or visual contrast, but leave open how to exploit complementary evidence discovered across the agent's own same-input interaction rollouts.

— VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents  (2609.38086 - Jiang et al., 29 Sep 2026) in Section 1, Introduction

We hypothesize that as a query transitions from being correctly to incorrectly responded, the reasoning process in the generated response gradually becomes less coherent.

— Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization  (2610.12855 - Tian et al., 9 Oct 2026) in Section 4.1, “Progressive Capability Degradation”

Whether enabling judge thinking or updating the judge improves assessment quality remains untested. Enabling thinking would add decoding cost and latency, depending on the reasoning budget; judge-update costs depend on whether updates reuse student parameters or require separate optimization.

— Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning  (2610.19176 - Pan et al., 13 Oct 2026) in Limitations, “Judge design,” p. 9