Generalization of BSD beyond small English mathematics models

Determine whether the performance gains of Bridged Self-Distillation hold for students larger than 4 billion parameters, for model families other than Qwen3, and for domains other than English competition mathematics.

Background

The experiments evaluate Bridged Self-Distillation only with student models of up to 4 billion parameters, drawn from a single model family, and on English competition-level mathematics problems. Consequently, it remains unresolved whether the reported improvements are attributable to a broadly generalizable property of the method or depend on this particular scale, architecture family, language, and task domain.

The authors explicitly identify larger students, other model families, and other domains as settings in which the persistence of BSD's gains should be tested. This is an empirical generalization problem rather than a claim that the method is theoretically invalid outside the reported scope.

References

First, due to computational constraints, our experiments are limited to students of up to 4B parameters from one model family and to English competition mathematics. Future work could examine whether the gains of BSD hold for larger students, other model families, and other domains.

— Bridged Self-Distillation for On-Policy Learning  (2610.26397 - Chen et al., 19 Oct 2026) in Limitations, first paragraph