Close the distillation gap between open VLM students and the Claude teacher

Close the performance gap between QLoRA-distilled Qwen3.5-9B and Gemma-4-31B models and the Claude Opus 4.8 teacher on the image-versus-paper consistency-checking task.

Background

The paper distills structured traces from a Claude Opus 4.8 consistency checker into Qwen3.5-9B and Gemma-4-31B using QLoRA. Although distillation improves both students substantially on held-out pairs, their performance remains well below that of the Claude teacher. The authors therefore explicitly identify closing this performance gap as unresolved future work.

References

Both students still trail the Claude teacher (77.1 on the same pairs, 84.1 on the 41/72 it actually covers) by a wide margin, so this is a partial distillation rather than a drop-in replacement; closing the gap is left to future work.

— SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task  (2609.00654 - Bao et al., 1 Sep 2026) in Section 5, paragraph “Distilling the consistency check”