Determinants of the OPD absorption rate

Determine what sets the absorption rate in on-policy distillation, namely the proportion of the remaining teacher–student gap that a single optimization update absorbs.

Background

The paper characterizes on-policy distillation as algorithm-starved: although a single query can expose a broad range of supervised states, the student increasingly slowly absorbs the remaining teacher signal. Experiments show that the absorption rate declines over training and does so similarly across training-set sizes, but the mechanism governing this decline is not identified.

The unresolved issue concerns the factors that determine the absorption rate itself, including why alignment slows even when the training states are fixed. Resolving it could guide the development of more step-efficient OPD optimizers and update rules.

References

What sets the absorption rate is still open.

— Rethinking On-Policy Distillation of Large Language Models II: One Training Example  (2609.04172 - Fu et al., 3 Sep 2026) in Section 9, Conclusion, Limitations

Extending these findings to broader models and tasks, relating local update quality to long-horizon learning, and reducing meta-update cost remain open directions.

— MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation  (2610.11989 - Wang et al., 8 Oct 2026) in Section Conclusion

These observations suggest that limited representation adaptation may contribute to learning-signal collapse, a hypothesis that remains to be tested.

— Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals  (2610.11247 - Zhao et al., 8 Oct 2026) in Abstract; Section 1, Introduction

Whether the direction of these small changes limits teacher matching remains a hypothesis for controlled interventions.

— Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals  (2610.11247 - Zhao et al., 8 Oct 2026) in Section 5, “OPD changes the student's parameters and representations only slightly”

Our local guarantee assumes an idealized gradient flow and shared parameterization, while the experiments use AdamW and a noisy gradient proxy; connecting these diagnostics to actual updates remains future work.

— Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals  (2610.11247 - Zhao et al., 8 Oct 2026) in Section 6, Conclusion; Appendix, Section “From the OPD update to the ODE”

Although the fixed-prefix analysis shows that RECAL reduces residual discrepancy on tokens that are heavily affected by pruning, the precise connection between this reduction and downstream recovery remains only partially understood. We are therefore interested in disentangling the roles of trajectory choice, divergence direction, and token weighting, and in developing diagnostics that follow the student states encountered during OPD rather than only fixed teacher prefixes.

— ReCal: Calibrating Structured Pruning for On-Policy Distillation Recovery  (2610.11332 - Jiang et al., 8 Oct 2026) in Section 7, “Limitations,” page 7