Principled mitigation of spurious linguistic artifacts in SDFT

Develop a principled method to prevent the student model in Self-Distillation Fine-Tuning (SDFT) from inheriting spurious teacher-conditioned linguistic markers (for example, prefatory phrases such as “Based on the text...” or “Following the example...”) that arise because the teacher is conditioned on demonstrations or source passages, thereby eliminating reliance on heuristic loss masking of the initial tokens during training.

Background

SDFT uses a demonstration-conditioned teacher to guide a student model on-policy. A failure mode noted by the authors is that the student can learn superficial linguistic patterns present in the teacher’s outputs, such as phrases indicating the presence of external context, even though the student receives no such context.

The paper reports that masking the loss over the first few tokens suppresses these artifacts in practice, but this is a heuristic workaround. The authors explicitly state that a more principled solution remains an open problem.