Unconstrained or latent reasoning vs. tag-constrained formatting

Determine whether removing the explicit <think></think> and <answer></answer> tag-format constraints used in Logic-RL and adopting an entirely unconstrained or latent reasoning representation yields better results than the current format-constrained approach.

Background

Logic-RL enforces a strict output structure via a rule-based reward that requires the model to place its intermediate reasoning within tags and its final conclusion within <answer></answer> tags. This design was introduced to prevent reward hacking behaviors and ensure verifiable, extractable answers.

While this formatting effectively organizes chain-of-thought and stabilizes training, the authors explicitly acknowledge uncertainty about whether such constraints may limit reasoning quality. They raise the possibility that allowing fully unconstrained outputs or leveraging latent internal representations could lead to superior outcomes, motivating investigation of alternative training objectives and reward structures that do not rely on explicit formatting.