Do Large Language Models Advocate for Inferentialism?

Abstract: The emergence of LLMs such as ChatGPT and Claude presents new challenges for philosophy of language, particularly regarding the nature of linguistic meaning and representation. While LLMs have traditionally been understood through distributional semantics, this paper explores Robert Brandom's inferential semantics as an alternative foundational framework for understanding these systems. We examine how key features of inferential semantics -- including its anti-representationalist stance, logical expressivism, and quasi-compositional approach -- align with the architectural and functional characteristics of Transformer-based LLMs. Through analysis of the ISA (Inference, Substitution, Anaphora) approach, we demonstrate that LLMs exhibit fundamentally anti-representationalist properties in their processing of language. We further develop a consensus theory of truth appropriate for LLMs, grounded in their interactive and normative dimensions through mechanisms like RLHF. While acknowledging significant tensions between inferentialism's philosophical commitments and LLMs' sub-symbolic processing, this paper argues that inferential semantics provides valuable insights into how LLMs generate meaning without reference to external world representations. Our analysis suggests that LLMs may challenge traditional assumptions in philosophy of language, including strict compositionality and semantic externalism, though further empirical investigation is needed to fully substantiate these theoretical claims.

• The paper introduces inferentialist semantics as a viable framework for LLMs, highlighting a departure from traditional representationalist models.
• It demonstrates that LLMs perform material inferences by leveraging linguistic patterns without relying on external referential grounding.
• The study reveals the quasi-compositional and norm-driven nature of LLMs, suggesting a paradigm shift in AI language processing.

An Exploration of LLMs through the Lens of Inferentialist Semantics

The paper "Do LLMs Defend Inferentialist Semantics?" by Yuzuki Arai and Sho Tsugawa offers an intriguing perspective on the semantics underlying LLMs, such as GPT-4, Claude, and others. It challenges the traditional use of distributional semantics to explain LLMs by proposing Robert Brandom's inferentialist semantics as a more suitable framework. The paper argues that LLMs not only embody the principles of logical expressivism and anti-representationalism that are central to inferentialist semantics but also bring about necessary reconsiderations in the philosophy of language.

Theoretical Underpinnings

At the core of the paper is the distinction between representationalist and anti-representationalist paradigms in the philosophy of language. Representationalism, as historically dominant, holds that language mirrors the world, aligning with truth-conditional semantics where the truth of propositions is central. In contrast, inferentialism, an anti-representationalist approach, views meaning as constituted by the roles expressions play within a network of inferences, eschewing direct correlations with external realities.

The authors assert that LLMs, products of modern machine learning philosophies, resonate more with anti-representationalism. This is due to LLMs' reliance on internal language games derived from massive data sets, rather than seeking alignment with an external world. Learning for these models occurs through statistical patterns within language itself, a notion alien to truth-conditional approaches but harmonious with inferentialism's focus on use and norm.

Key Arguments

1. Material Inference in LLMs: The paper argues that LLMs exhibit a form of inference that is material, relying on patterns and structures present in linguistic data rather than principles of formal logic. This suggests a strong correspondence with Brandom's inferentialism, where the inferential roles, rather than representational alignment, confer meaning.
2. Quasi-Compositional Nature: Unlike traditional symbolic systems, which adhere to strict compositionality, LLMs demonstrate a quasi-compositional nature. This aligns with inferentialism's flexible compositional approach, which recognizes the context-sensitive nature of meaning formation in natural language.
3. Normativity and Anaphora: The authors discuss how LLMs' attention mechanisms facilitate anaphoric resolution without explicit semantic representations. This aspect fits within the inferentialist understanding of meaning-making through anaphora and substitution, rather than reference to an external world.
4. Anti-Representational Characteristics: The idea that LLMs function as anti-representational systems is emphasized, as their reliance on linguistic input eschews any necessary grounding in a non-linguistic reality. This extends to the difficulties in applying traditional theories of truth, such as the correspondence theory, to LLMs, as their 'understanding' remains encapsulated within linguistic constructs.

Practical and Theoretical Implications

The implications of adopting inferentialist semantics for LLMs are manifold. On a practical level, this perspective offers a viable explanation for why LLMs, which function without explicit referential connections to the world, still manage to perform impressively in tasks reminiscent of human linguistic competence. Theoretically, this challenges the prevalent philosophical paradigms by suggesting that LLMs can meaningfully operate within an internalist, anti-representationalist framework.

Future Developments and Considerations

Looking ahead, the evolving complexity of LLMs may continue to challenge existing philosophical categories, pressing for additional adaptations in theories of language. The potential expansions into multimodal models could necessitate further reconciliation with world-responsive elements. Through these developments, inferentialism provides a fruitful ground for ongoing exploration, particularly regarding the normativity and pragmatic dimensions of AI.

In conclusion, this paper enriches the discourse on LLMs by proposing a shift towards inferentialist semantics, challenging current associations with distributional semantics. It opens pathways for reevaluating semantic frameworks to better fit the capabilities and characteristics of AI, ultimately fostering a deeper understanding of the intersection between philosophy and technology.