Effect of fine-tuning on structured-format representational geometry

Determine whether fine-tuning language models on structured formats produces the same representational geometry observed under prompting with structured output schemas.

Background

All experiments study prompted generation and do not include training-time adaptation to structured formats. Consequently, it remains unresolved whether fine-tuning on structured outputs would preserve the same middle-layer separability and other representational effects identified for prompted reasoning-first schemas.

References

All experiments rely on prompting rather than training-time adaptation, so it is unclear whether fine-tuning on structured formats would produce the same representational geometry.

— Structure Tax: How Structured Output affects LLMs Performance  (2610.12056 - Kumar et al., 8 Oct 2026) in Limitations section