Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency (2407.08790v1)

Abstract: In this paper we argue that key, often sensational and misleading, claims regarding linguistic capabilities of LLMs are based on at least two unfounded assumptions; the assumption of language completeness and the assumption of data completeness. Language completeness assumes that a distinct and complete thing such as a natural language' exists, the essential characteristics of which can be effectively and comprehensively modelled by an LLM. The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data. Work within the enactive approach to cognitive science makes clear that, rather than a distinct and complete thing, language is a means or way of acting. Languaging is not the kind of thing that can admit of a complete or comprehensive modelling. From an enactive perspective we identify three key characteristics of enacted language; embodiment, participation, and precariousness, that are absent in LLMs, and likely incompatible in principle with current architectures. We argue that these absences imply that LLMs are not now and cannot in their present form be linguistic agents the way humans are. We illustrate the point in particular through the phenomenon ofalgospeak', a recently described pattern of high stakes human language activity in heavily controlled online environments. On the basis of these points, we conclude that sensational and misleading claims about LLM agency and capabilities emerge from a deep misconception of both what human language is and what LLMs are.

Overview of "Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency"

The paper "Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency", authored by Abeba Birhane and Marek McGann, provides a critical evaluation of the linguistic capabilities attributed to LLMs. The authors argue that such capabilities are frequently overstated and often based on misguided assumptions about language and data completeness. From an enactive cognitive science standpoint, the paper asserts that language, as practiced by humans, cannot be comprehensively modeled by current LLM architectures due to their lack of embodiment, participation, and precarity.

Key Assumptions Challenged

Birhane and McGann identify two foundational but flawed assumptions underpinning the exaggerated claims about LLMs:

1. Language Completeness: This assumption posits that natural language is a stable entity that can be entirely captured and modeled.
2. Data Completeness: This assumption holds that sufficient data can comprehensively represent a language for effective modeling by LLMs.

Enactive Perspective on Language

The paper contrasts these assumptions with the enactive approach to cognitive science, which views language as dynamic, embodied, and participatory. According to this view, language is a living activity—termed 'languaging'—which cannot be encapsulated by static datasets. This framework highlights three key aspects of human linguistic interaction that are absent in LLMs:

• Embodiment: Language arises from our embodied experiences in the world.
• Participation: Linguistic acts are inherently social and collaborative.
• Precariousness: Human language use involves continual negotiation and management of context-specific tensions and risks.

Contrasting LLMs and Human Linguistic Agency

Embodiment

Human language is deeply tied to our bodily experiences and the contexts within which interactions occur. In contrast, LLMs operate by generating text based on statistical relationships in pre-existing data. They lack the physical and experiential grounding that provides depth and nuance to human communication. This disconnect is apparent in situations where context and shared experiences significantly shape the meaning of interaction.

Participation

Human linguistic interactions are collaborative and adaptive. People frequently seek clarification and adjust their communication based on immediate feedback, a coordination that LLMs are not inherently equipped to handle. The participatory nature of human language defies static classification and cannot be fully captured by text corpora used to train LLMs.

Precarity

Human linguistic actions are driven by the necessity to manage social and contextual tensions. The stakes in our linguistic interactions are intrinsically tied to our social identities and lived experiences. LLMs, however, lack any form of self-driven motivation or risk, as their output is devoid of intentionality, ethical considerations, and the subtleties of human social structures and power dynamics.

Algospeak: A Case Study in Human Linguistic Innovation

The phenomenon of 'algospeak'—where users modify language to bypass automated content moderation—is highlighted as a clear example of human linguistic agency. This dynamic adaptation underscores the embodied, participatory, and precarious nature of human language, all elements that current LLMs cannot replicate or understand. Algospeak emerges as a form of resistance and innovation, pointing to the creative and context-sensitive aspects of human communication that LLMs lack.

Practical and Theoretical Implications

The misconceptions about LLM capabilities have significant implications. Practically, they could misinform public policy and social practices, leading to inappropriate reliance on these tools in sensitive domains. Theoretically, they obscure the deeper understanding of what constitutes human linguistic activity. LLMs should be seen as advanced tools aiding human language users rather than as entities capable of true linguistic agency.

Future Directions

Future developments in AI should account for the nuanced, dynamic, and embodied nature of human language. Integrating insights from enactive cognitive science could guide more robust understandings of linguistic agency and inform responsible AI design and deployment. This multidisciplinary approach is essential to address ethical concerns and potential societal impacts, particularly regarding marginalized communities.

Conclusion

In summary, Birhane and McGann's paper emphasizes the need to recalibrate our understanding of LLMs in light of their inherent limitations. Recognizing the distinct nature of human linguistic agency can guide the development of more ethically and socially aware AI technologies. The paper serves as a crucial reminder that while LLMs represent significant engineering advancements, they do not mirror the complex, embodied, and participative nature of human language.