Purpose of non-reasoning tokens in chain-of-thought traces

Determine whether tokens not directly related to reasoning (such as grammatical or stylistic tokens) in chain-of-thought outputs serve any function beyond enabling additional forward passes before the model produces a final answer.

Background

The authors note that chain-of-thought outputs often include tokens that do not appear to contribute to reasoning content, raising doubts about their utility. If such tokens merely extend the number of forward passes, they might function only as a mechanism to increase compute rather than as substantive reasoning steps.

Clarifying the role of these non-reasoning tokens is important for designing efficient reasoning systems and for interpreting the faithfulness of visible reasoning traces to the model’s actual computation.