Principled understanding of chain-of-thought verbalization and balancing implicit versus explicit reasoning

Determine principled theoretical understandings of the role that verbalizations of intermediate knowledge and reasoning steps (e.g., chain-of-thought rationales) play in solving reasoning tasks with language models, and develop methods that decide an appropriate balance between implicit reasoning over parametric memory and explicit, verbalized reasoning for problems with large intrinsic complexity.

Background

The paper focuses on implicit reasoning with transformers, showing that such capability is typically acquired through grokking and analyzing mechanisms that underlie generalization. It contrasts this with explicit verbalizations of reasoning (e.g., chain-of-thought) which can improve performance but are absent during large-scale pretraining.

Within this context, the authors highlight a gap in principled understanding of when and how verbalized reasoning contributes to solving reasoning tasks, and how to algorithmically balance implicit (parametric) and explicit (verbalized) reasoning to tackle complex problems.