Calibrate teacher competence against optimization-signal magnitude

Characterize how to calibrate the trade-off between teacher competence and the magnitude of the learning signal in post-training teacher selection.

Background

The paper separates teacher competence, measured through reward quality, from the student’s optimization toward the teacher. Its theoretical routing analysis primarily concerns the strength of the imitation or optimization signal, while teacher reward quality appears in a separate performance-difference term.

The authors identify an unresolved issue in selecting teachers that are both competent and capable of providing a suitably strong learning signal. A confidently optimized but undesirable teacher can distort the student, whereas a competent teacher with incompatible representations may be difficult to imitate. The paper does not provide a general method for balancing these factors.

References

In general, it is an open question to quantify the trade-off between teacher competence and optimization pressures, but we would nonetheless like to include a brief discussion of success cases and failure modes of our two families of teacher selection.

— Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation  (2610.21234 - Wei et al., 15 Oct 2026) in Appendix D, subsection “Separating imitation from reward”

Indeed, this is the big open problem in post-training, of how to calibrate competence vs. magnitude learning signal.

— Compatibility Matters: A Framework for Token-Level Teacher Selection in On-Policy Distillation  (2610.21234 - Wei et al., 15 Oct 2026) in Appendix D, subsection “On the representation-based selection scheme”