Generalization of OPSD findings beyond Qwen3 models

Determine whether the reported findings about privileged reference information in on-policy self-distillation generalize to model families other than Qwen3.

Background

The study evaluates on-policy self-distillation using Qwen3 models at 8B, 4B, and 1.7B scales. Its conclusions concern how reference-conditioned teacher supervision, reference-free teacher behavior, and mismatched references affect student performance and distributional alignment in these models. The authors explicitly leave unresolved whether these findings extend to other model families, making cross-family validation an open empirical problem.

References

Whether the results generalize to other model families remains unknown.

— Rethinking Privileged Information in On-Policy Self-Distillation  (2608.18271 - Shrestha et al., 18 Aug 2026) in Section 6, Limitations

We leave alternative configurations to future work, including task-specific teachers, multi-teacher settings, and on-policy self-distillation, where the teacher is the student itself augmented with privileged information from ground-truth demonstrations~\citep{opsd} or environment feedback~\citep{sdpo}. Whether OPD-then-RL continues to dominate joint optimization in these settings, and whether our identified mechanisms carry over, remain to be investigated.

— Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR  (2609.04108 - Li et al., 3 Sep 2026) in Limitations, paragraph “Scope of teacher--student configurations”

whether the same conclusions hold across families is left to future work.

— MAS-OPD: On-Policy Distillation for Multi-agent Systems  (2609.34234 - Zhong et al., 28 Sep 2026) in Appendix, Section 1, “Limitations”

Other contexts, and λ, κ and ρφ measured on the LLMs, are open.

— Disentangling Self-Distillation: Measuring and Modeling Acquisition and Retention  (2609.39494 - Zuin et al., 30 Sep 2026) in Section 7, Conclusion