Language Writ Large: LLMs, ChatGPT, Grounding, Meaning and Understanding (2402.02243v1)

Abstract: Apart from what (little) OpenAI may be concealing from us, we all know (roughly) how ChatGPT works (its huge text database, its statistics, its vector representations, and their huge number of parameters, its next-word training, and so on). But none of us can say (hand on heart) that we are not surprised by what ChatGPT has proved to be able to do with these resources. This has even driven some of us to conclude that ChatGPT actually understands. It is not true that it understands. But it is also not true that we understand how it can do what it can do. I will suggest some hunches about benign biases: convergent constraints that emerge at LLM scale that may be helping ChatGPT do so much better than we would have expected. These biases are inherent in the nature of language itself, at LLM scale, and they are closely linked to what it is that ChatGPT lacks, which is direct sensorimotor grounding to connect its words to their referents and its propositions to their meanings. These convergent biases are related to (1) the parasitism of indirect verbal grounding on direct sensorimotor grounding, (2) the circularity of verbal definition, (3) the mirroring of language production and comprehension, (4) iconicity in propositions at LLM scale, (5) computational counterparts of human categorical perception in category learning by neural nets, and perhaps also (6) a conjecture by Chomsky about the laws of thought. The exposition will be in the form of a dialogue with ChatGPT-4.

Understanding LLMs: Insights and Implications

Propositional Iconicity and Language Understanding

The performance of LLMs such as GPT-4 is influenced by a range of factors, including the propositional patterns present within their extensive databases. While individual words and phrases may not exhibit iconicity, LLMs potentially detect and utilize larger propositional structures, which could act as a bias in linguistic output. This isn't iconicity in a traditional sense but rather indicative of regularities that emerge at scale, shaping the AI's linguistic output to align with patterns inherent within human communication.

The Role of Grammar and Stylistic Regularity

LLMs do not solely rely on propositional structure; they also incorporate regularities in language use, including aspects of Ordinary Grammar (OG) and stylistic patterns. Learned through both unsupervised and supervised methods, these linguistic structures contribute to the efficacy of LLMs in processing and generating text. However, it is essential to note that these forms of grammar and style are derived from human language usage and are not a direct measure of understanding.

Dictionary Minimal Grounding Sets and AI

The concept of Dictionary Minimal Grounding Sets (MinSets) remains important for LLM processing. A small subset of core words enables the definition of the rest within a language, suggesting that foundational grounding of these words could be paramount in establishing comprehensive language comprehension.

Perception, Production, and Mirroring

LLMs might exhibit mirroring effects by generating responses that reflect the user's language input. While this process mimics cognitive mirroring observed in humans, in AI, it is the result of text-based patterns rather than conscious or empathetic understanding.

The Power of Indirect Verbal Grounding

LLMs are capable of indirect verbal grounding (IVG), feeding off extensive text data to simulate meaningful linguistic outputs. However, they lack the capacity for direct sensorimotor grounding (DSG), essential for genuine understanding. The efficacy of LLMs in linguistic tasks, while remarkable, entails replication of human-like language processing, but not true understanding or grounding.

Universal Grammar and Thinkability

The suggestion that LLMs might inherently adhere to Universal Grammar (UG) principles posits that their absence of UG violations could be due to the conformity of their training data to the "laws of thought." Whereas this might explain LLM's remarkable performance, it emphasizes the distinction between the replication of language patterns and true understanding.

Cognitive Modeling vs. AI Tools

The quest for grounding in AI systems, particularly in the distinction between AI tools and cognitive modeling (CM), remains a significant topic. While AI focuses on creating tools for task performance, CM aims to reverse-engineer human cognition. The distinction is crucial when examining the potential for models like LLMs to achieve T3 robotic cognitive capacities identical to human cognition.

Addressing the SGP and the HP

The exploration of the Symbol Grounding Problem (SGP) and the Hard Problem (HP) in the context of LLMs raises profound questions. While LLMs can address aspects of the SGP through learned databases, the HP relates to the consciousness and experience associated with understanding—areas where LLMs do not match human capabilities.

Final Thoughts

In conclusion, multiple biases and constraints influence LLMs like GPT-4, contributing to their surprisingly effective language output. These factors, ranging from detected propositional patterns to learned grammatical regularities and the potential alignment with UG, might indicate more about the nature of language itself rather than the model's intrinsic capabilities. Nevertheless, these models do not encompass true understanding or grounded experience, the essence of human cognition.