Disentangle output-length effects from structured-output failure

Determine whether the lower retention of converted diffusion language models on the CoNLL-to-JSON task is caused by output length or by structured-output failure occurring before free-text generation fails.

Background

The paper associates performance retention with the parent model’s generated output length and observes that in-place conversion degrades sharply on longer-output tasks. CoNLL-to-JSON is an exception to a simple length-based interpretation: it is a mid-length structured task and has low retention in both the in-place and frozen-tower families.

Because the experiment does not independently vary output length and output structure, it cannot establish whether the degradation reflects a length effect, a failure specific to structured output, or both. The authors explicitly identify this separation as unresolved.

References

The figure leaves two things open. CoNLL-JSON is the structured task and also a mid-length one, and it sits lowest for its length in both families. These tasks therefore cannot separate a length effect from structured output failing before free text does.

— Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs  (2610.02657 - Lu et al., 2 Oct 2026) in Appendix, Section “Output length and retention” (Section \ref{app:length})