Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse (2410.21333v4)

Abstract: Chain-of-thought (CoT) prompting has become a widely used strategy for improving large language and multimodal model performance. However, it is still an open question under which settings CoT systematically reduces performance. In this paper, we seek to identify the characteristics of tasks where CoT reduces performance by drawing inspiration from cognitive psychology, focusing on six representative tasks from the psychological literature where deliberation hurts performance in humans. In three of these tasks, state-of-the-art models exhibit significant performance drop-offs with CoT (up to 36.3\% absolute accuracy for OpenAI o1-preview compared to GPT-4o), while in others, CoT effects are mixed, with positive, neutral, and negative changes. While models and humans do not exhibit perfectly parallel cognitive processes, considering cases where thinking has negative consequences for humans helps identify settings where it negatively impacts models. By connecting the literature on human verbal thinking and deliberation with evaluations of CoT, we offer a perspective for understanding the impact of inference-time reasoning.

• The paper demonstrates that Chain-of-Thought prompting decreases model accuracy by up to 36.3% in implicit learning tasks and increases pattern classification time by 331% when exceptions are present.
• It reveals that verbalized reasoning impairs tasks needing perceptual processing, mirroring the human verbal overshadowing effect in visual recognition.
• The research implies that adopting task-specific prompting strategies is essential to avoid cognitive mismatches and optimize AI model performance.

Analyzing the Cognitive Limitations of Chain-of-Thought in AI Models

The paper "Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse" explores the counterintuitive effects of Chain-of-Thought (CoT) prompting on large language and multimodal models (LLMs and LMMs). The authors draw an innovative parallel to cognitive psychology, investigating cases where verbal deliberation negatively impacts human performance and applying these insights to AI systems. Notably, while CoT generally enhances performance on various tasks, this paper highlights settings where it reduces effectiveness, particularly focusing on implicit statistical learning, visual recognition, and classification tasks with exceptions.

Task Characteristics and CoT Performance Reduction

The research identifies specific task types where CoT adversely affects model performance. These include:

1. Implicit Statistical Learning: Defined by tasks where understanding patterns without explicit rules is crucial, such as artificial grammar learning. The findings show that CoT can lead to a significant decrease in accuracy, with up to a 36.3% drop observed in the OpenAI o1-preview model.
2. Visual Recognition: Particularly tasks like facial recognition, where verbalizing perceptual information hampers identification. This aligns with the 'verbal overshadowing' effect observed in humans, where language representation is unsuited for fine-grained perceptual tasks. Here, CoT prompting results in notable performance declines across tested LMMs.
3. Classifying with Patterns Containing Exceptions: CoT appears to misguide models by focusing on generalizable rules rather than exceptions, increasing learning time for pattern classification with exceptions by up to 331%.

The common thread across these tasks is the mismatch between the processing encouraged by CoT and the type of reasoning that optimizes task performance. The authors effectively transfer cognitive psychology concepts, traditionally used to understand human performance limitations, to machine learning, offering a heuristic for predicting when CoT might decrease AI model effectiveness.

Divergent Outcomes: Tasks Not Mimicking Human Cognitive Constraints

Moreover, the research delineates tasks where human and AI constraints diverge. For example, in tasks requiring logical reasoning and recognizing spatial relations, CoT does not detract from performance because models do not share human limitations like working memory constraints or reliance on perceptual-motor experiences. This emphasizes that the generalizability of cognitive constraints must be critically evaluated before assumptions about CoT's efficacy can be made.

Implications and Future Directions

By identifying when CoT reduces model performance, this paper contributes to the broader understanding of inference-time reasoning's limitations. The implications of these findings are profound, suggesting that default CoT usage should be reconsidered for tasks where the task-structure or reasoning types engender negative effects.

The insights presented open avenues for model design and prompt strategy development, emphasizing the need for task-specific adaptations in LLMs and LMMs. Future work could explore more complex CoT techniques, such as tree-of-thought, in diverse settings to extend the framework proposed here and further dissect the underlying reasons for performance discrepancies.

Overall, this research bridges cognitive psychology and AI, proposing a novel evaluative lens for prompt design in AI. Such interdisciplinary approaches could significantly enhance AI systems' robustness, aligning their capabilities more closely with the nuances of different decision-making contexts.