An Analysis of "Does ChatGPT Have a Mind?"
The paper "Does ChatGPT Have a Mind?" by Simon Goldstein and B.A. Levinstein provides a comprehensive analysis of whether LLMs such as ChatGPT can be said to possess minds. Central to this question is whether these models have a folk psychology consisting of beliefs, desires, and intentions. The authors undertake this investigation by examining internal representations and dispositional actions, supporting their claims with philosophical theories and machine learning interpretability research.
The authors begin by surveying philosophical frameworks of representation—informational, causal, structural, and teleosemantic—to argue that LLMs satisfy essential conditions for mental representation according to each theory. For instance, they note that informational theories require that states carry probabilistic information, and recent AI probing techniques demonstrate this capacity in LLMs. They reference experiments, such as those involving Othello-GPT, where LLMs trained on game moves appear to internally represent the state of the game board, as verified through causal intervention techniques.
A significant part of the paper addresses challenges to LLM representation: sensory grounding, stochastic parrots, and memorization. The sensory grounding argument asserts that purely text-based LLMs lack a connection to the external world, thereby precluding genuine representation. Goldstein and Levinstein counter this by suggesting that LLMs could form hypotheses about the environment based on text, another form of causal connection supporting representation.
The stochastic parrot argument questions whether LLMs truly understand or merely mimic patterns, given their primary task is predicting text. Here, the authors argue that more complex behaviors could emerge as side effects of the prediction task, such as forming meaningful internal world models. Furthermore, empirical observations of few-shot and in-context learning, as well as successful transfer learning across domains, suggest that LLMs possess emergent reasoning capabilities.
The authors also examine skepticism rooted in memorization, suggesting that LLMs demonstrate substantial generalization beyond memorized data. They use the example of modular arithmetic learning in LLMs where initial memorization gives way to implementing generalizable algorithms.
The second central question is whether LLMs have dispositions to act, which is crucial for claiming a folk psychology. The analysis includes interpretationalist perspectives, such as Dennett's intentional stance, and representationalist perspectives, like those from Fodor and Dretske. These discussions largely depend on whether LLM behaviors can be viewed as goal-oriented or whether they derive merely from computational processes mirroring human-like interaction.
Ultimately, the authors conclude that while there is strong evidence for the presence of internal representations in LLMs, the evidence for universal beliefs, desires, and intentions is less clear-cut, due largely to issues of behavioral consistency and goal representation.
This paper contributes to both practical and theoretical developments. Practically, it highlights the importance of AI transparency and interpretability for detecting and improving folk psychological attributes in models. Theoretically, it urges further exploration of non-standard causal and teleosemantic theories and their implications for LLM capabilities.
In sum, this paper represents a detailed philosophical and technical inquiry into the cognitive attributes of LLMs, challenging existing skepticism and laying a foundation for future research in AI cognition and philosophy of mind.