Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models (2406.02061v4)

Abstract: LLMs are often described as being instances of foundation models - that is, models that transfer strongly across various tasks and conditions in few-show or zero-shot manner, while exhibiting scaling laws that predict function improvement when increasing the pre-training scale. These claims of excelling in different functions and tasks rely on measurements taken across various sets of standardized benchmarks showing high scores for such models. We demonstrate here a dramatic breakdown of function and reasoning capabilities of state-of-the-art models trained at the largest available scales which claim strong function, using a simple, short, conventional common sense problem (AIW problem) formulated in concise natural language, easily solvable by humans. The breakdown is dramatic, as models show strong fluctuations across even slight problem variations that should not affect problem solving, also expressing strong overconfidence in the wrong solutions, often backed up by plausible sounding explanation-like confabulations. Various standard interventions in an attempt to get the right solution, like various type of enhanced prompting, or urging the models to reconsider the wrong solutions again by multi step re-evaluation, fail. We take these initial observations to the scientific and technological community to stimulate urgent re-assessment of the claimed capabilities of current generation of LLMs. Such re-assessment also requires common action to create standardized benchmarks that would allow proper detection of such basic reasoning deficits that obviously manage to remain undiscovered by current state-of-the-art evaluation procedures and benchmarks. Code for reproducing experiments in the paper and raw experiments data can be found at https://github.com/LAION-AI/AIW

Essay: Critical Examination of Reasoning Capabilities in State-of-the-Art LLMs

In the paper titled "Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art LLMs", the authors Nezhurina et al. present a rigorous analysis of reasoning capabilities within current LLMs. Contrary to the high scores these models achieve on conventional benchmarks, the paper reveals profound deficiencies in basic reasoning when faced with deceptively simple common sense tasks. This essay aims to provide a detailed examination of their methods, findings, and implications for future AI developments.

Methodological Approach

The authors introduce a focused, albeit simple, common sense reasoning problem termed the "Alice in Wonderland" (AIW) problem. The problem is structured as follows: "Alice has $N$ brothers and she also has $M$ sisters. How many sisters does Alice's brother have?" Despite its simplicity, solving this problem requires basic arithmetic and relational logic well within the grasp of human adults. For comparative purposes, multiple variations of this problem were presented to various state-of-the-art LLMs, including GPT-4, Claude 3, Mistral, Llama, and others.

The paper's methodology involves:

1. Prompt Variation: Utilization of different prompt types (STANDARD, THINKING, RESTRICTED) to evaluate the models' robustness and variability in responses.
2. Response Evaluation: Quantitative assessment through the correct response ratio, drawn from repeated trials.
3. Model Selection: Inclusion of both closed and open weights models across varying scales, with attention to the latest iterations and leadership in public leaderboards.
4. Benchmark Comparison: Analysis of performance discrepancies between AIW tasks and standardized reasoning benchmarks like MMLU, HellaSwag, and GSM8K.

Key Findings

The findings from this paper are both remarkable and concerning:

1. Significant Breakdown in Reasoning: Most current SOTA LLMs exhibited a severe breakdown in reasoning capabilities when tasked with the AIW problem. For instance, models like Mistral-7B, Mixtral, and Command R+ delivered correct responses at rates close to zero, contradicting their high standardized benchmark scores.
2. Exceptions and Fluctuations: Notably, larger-scale models such as GPT-4 and Claude 3 demonstrated some ability to solve the AIW problem albeit inconsistently, with substantial fluctuations across problem variations. These exceptions hint at the latent presence of generalization capabilities that are, however, poorly controlled.
3. Overconfidence and Confabulations: A striking observation is the models' propensity to express high confidence in incorrect answers and generate persuasive but incorrect and nonsensical explanations, termed confabulations. This miscalibration is a critical safety issue, potentially misleading users regarding the reliability of the models’ outputs.
4. Failure of Standard Benchmarks: The paper highlights a strong mismatch between models' standardized benchmark scores and their performance on AIW tasks. For instance, models like Command R+, which score highly on benchmarks such as MMLU and GSM8K, failed consistently on the AIW problem.

Implications and Future Directions

The evidence presented in the paper prompts a critical reevaluation of current LLMs' claimed reasoning capabilities. High scores on traditional benchmarks do not necessarily translate to robust reasoning ability on simple common sense tasks. This misalignment has several key implications:

1. Challenge of Benchmark Reliability: Existing standardized benchmarks are insufficient for evaluating true reasoning capabilities. New benchmarks, more aligned with common sense reasoning tasks, are necessary. Such benchmarks should be designed under principles of falsifiability to highlight reasoning deficits rather than merely validating strengths.
2. Safety and Trustworthiness: The models' overconfidence in wrong answers and tendency to confabulate raise significant safety concerns. In applications where decision-making is critical, these models' inability to reliably reason can lead to potentially severe consequences.
3. Open Source and Transparency: To advance trustworthy AI, the paper underscores the importance of full transparency in the training pipeline, including dataset composition and training procedures. This openness would enable the community to understand, replicate, and address existing deficiencies.

Conclusion

Nezhurina et al.'s paper provides pivotal insights into the fundamental limitations of current LLMs in performing basic common sense reasoning tasks. The dramatic breakdown observed underscores the urgency for the AI community to develop more robust evaluation frameworks and to pursue further research into enhancing reasoning abilities. Moreover, addressing these foundational issues can pave the way for developing more reliable, safe, and truly intelligent systems.

By pinpointing critical weaknesses and proposing actionable paths forward, this paper serves as a crucial wake-up call, steering AI research toward a future where reasoning in artificial systems matches human-like logical consistency and reliability.