Do large language models reason in a human-like manner?

Determine whether large language models reason in a genuinely human-like manner.

Background

The paper investigates whether LLMs align with human behavior and neurocognition during abstract reasoning, motivated by rapid advances in LLM capabilities and ongoing debate about the nature of their reasoning. Although LLMs achieve high performance on many tasks, it remains unresolved whether their internal processes constitute human-like reasoning rather than pattern matching.

To address this, the authors compare eight open-source LLMs with human participants on an abstract pattern completion task, analyzing both behavioral performance and neural signatures via fixation-related potentials (FRPs) from EEG. They find that only the largest models approach human accuracy and that intermediate model layers exhibit pattern-specific clustering that modestly correlates with human frontal FRPs, suggesting preliminary alignment but leaving open the broader question of whether LLMs truly reason like humans.

References

Yet whether these models reason in a genuinely human‑like manner remains an open question.

— Large Language Models Show Signs of Alignment with Human Neurocognition During Abstract Reasoning  (2508.10057 - Pinier et al., 12 Aug 2025) in Introduction, opening paragraph (page 1)

Moreover, DeReLab didn't evaluate models' consistency against human performance, leaving it an open question whether human reasoning diverges from algorithmic conclusions.

— DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark  (2608.30413 - Sadhu et al., 31 Aug 2026) in Section Limitations

Whether the same observations extend to other domains, such as science, programming, or general factual knowledge, remains an open question.

— Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory  (2609.05245 - Cui et al., 4 Sep 2026) in Section 6, Limitations