Principled understanding of chain-of-thought verbalization and balancing implicit versus explicit reasoning

Determine principled theoretical understandings of the role that verbalizations of intermediate knowledge and reasoning steps (e.g., chain-of-thought rationales) play in solving reasoning tasks with language models, and develop methods that decide an appropriate balance between implicit reasoning over parametric memory and explicit, verbalized reasoning for problems with large intrinsic complexity.

Background

The paper focuses on implicit reasoning with transformers, showing that such capability is typically acquired through grokking and analyzing mechanisms that underlie generalization. It contrasts this with explicit verbalizations of reasoning (e.g., chain-of-thought) which can improve performance but are absent during large-scale pretraining.

Within this context, the authors highlight a gap in principled understanding of when and how verbalized reasoning contributes to solving reasoning tasks, and how to algorithmically balance implicit (parametric) and explicit (verbalized) reasoning to tackle complex problems.

References

Our focus here on implicit reasoning is orthogonal, and it is an interesting open problem to have principled understandings of the role of such verbalizations in reasoning problems, and also develop methods that can decide the appropriate balance between implicit and explicit reasoning to handle challenging problems with large intrinsic complexity.

— Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization  (2405.15071 - Wang et al., 2024) in Section 6: Related Work

A central open problem is therefore to determine where along this spectrum a given task should operate, and how a system should move along the spectrum dynamically as its control requirements change.

— Complex Problem Solving in Large Language Models: A Statistical Control Survey and Diagnostic Framework  (2609.20973 - Cai et al., 17 Sep 2026) in Section 5.1, Structured State, Provenance, and Faithful Trace Interfaces

Our evidence is consistent with the Reliance Hypothesis but does not establish its mechanism.

— The Reliance Hypothesis: Chain-of-Thought Faithfulness Rises as Language Resource Level Falls  (2610.22841 - Resck et al., 16 Oct 2026) in Section 5, Discussion and Conclusion, paragraph “Findings and Mechanism”