(Ir)rationality and Cognitive Biases in Large Language Models (2402.09193v2)

Abstract: Do LLMs display rational reasoning? LLMs have been shown to contain human biases due to the data they have been trained on; whether this is reflected in rational reasoning remains less clear. In this paper, we answer this question by evaluating seven LLMs using tasks from the cognitive psychology literature. We find that, like humans, LLMs display irrationality in these tasks. However, the way this irrationality is displayed does not reflect that shown by humans. When incorrect answers are given by LLMs to these tasks, they are often incorrect in ways that differ from human-like biases. On top of this, the LLMs reveal an additional layer of irrationality in the significant inconsistency of the responses. Aside from the experimental results, this paper seeks to make a methodological contribution by showing how we can assess and compare different capabilities of these types of models, in this case with respect to rational reasoning.

Evaluation of Rational and Irrational Reasoning in LLMs Using Cognitive Psychology Tasks

Introduction

LLMs have become pivotal in numerous applications, heralding substantial changes in how daily tasks are executed. Their integration raises essential questions about LLMs' ability to reason and whether they replicate the intricate patterns of human thinking, including biases and rationality. This paper explores the rational reasoning capabilities of seven prominent LLMs through a series of cognitive psychology tasks traditionally used to identify biases and heuristics in human reasoning.

Methodological Framework

LLMs Under Review

The paper scrutinizes seven LLMs, including high-profile models such as GPT-3.5, GPT-4, Google Bard, Anthropic's Claude 2, and Meta's Llama 2 in various parameter configurations. These models were chosen for their prevalent usage and the diverse techniques and datasets on which they were trained, providing a broad perspective on the state of rational reasoning across different AI architectures.

Cognitive Tasks and Evaluation Criteria

Cognitive tasks, derived mainly from the work of Kahneman and Tversky, were employed to evaluate the LLMs. These tasks are designed to probe different types of cognitive biases, such as the conjunction fallacy, representativeness heuristic, and base rate neglect. Each task was presented to the models without modification from its traditional human-focused format to assess whether models perform similarly to humans when confronted with identical challenges.

The responses from LLMs were categorized based on two dimensions: correctness and human-likeness. This dual categorization allows the paper to discern whether incorrect answers stem from biases similar to human reasoning or other forms of illogical processing unique to machines.

Results and Analysis

Response Consistency and Reasoning Accuracy

The investigation revealed a marked inconsistency in LLM responses to identical tasks, highlighting a form of irrationality characterized by unpredictable variations in reasoning. Contrary to human subjects, who typically exhibit consistent biases across similar tasks, LLMs displayed erratic reasoning patterns, with accuracy and rationale varying significantly even within the same model.

Notably, incorrect answers rarely aligned with human-like biases, indicating that LLMs do not replicate human irrationality in a directly comparable manner. Instead, mistakes were primarily attributed to flawed logic, erroneous calculations, or misinterpretations of the task requirements, underscoring fundamental differences in how LLMs and humans process information.

Model-Specific Observations

GPT-4 emerged as the most proficient model in terms of producing correct and rational responses, followed by Claude 2, suggesting advancements in model architecture or training data might influence rational reasoning capabilities. Conversely, Meta's Llama 2 models, particularly at lower parameter counts, struggled significantly, often providing responses that were neither correct nor resembled typical human reasoning errors.

Implications and Future Directions

The distinct patterns of irrationality and the absence of human-like biases in LLM responses point to underlying differences in information processing between humans and machines. These findings have profound implications for applications requiring human-AI collaboration or decision-making support, where predictable and understandable reasoning processes are paramount.

This paper lays the groundwork for further research into the mechanisms behind LLM reasoning, suggesting a need for novel benchmarks tailored to evaluate AI rationality comprehensively. Future work could explore the impact of training data diversity, model architecture nuances, or the influence of fine-tuning on reasoning capabilities, potentially leading to models that better emulate or complement human cognitive processes.

Conclusion

In concluding, this paper offers a novel perspective on the rationality of LLMs by applying cognitive psychology tasks as a measure. The findings illuminate the complexities of AI reasoning, challenging preconceived notions about LLMs' ability to mimic human thought processes accurately. As LLMs continue to evolve, understanding and enhancing their reasoning capabilities in alignment with human cognition remains a compelling avenue for research, with significant implications for both theoretical exploration and practical application.