Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs (2401.04854v3)

Abstract: Are LLMs cultural technologies like photocopiers or printing presses, which transmit information but cannot create new content? A challenge for this idea, which we call bibliotechnism, is that LLMs generate novel text. We begin with a defense of bibliotechnism, showing how even novel text may inherit its meaning from original human-generated text. We then argue that bibliotechnism faces an independent challenge from examples in which LLMs generate novel reference, using new names to refer to new entities. Such examples could be explained if LLMs were not cultural technologies but had beliefs, desires, and intentions. According to interpretationism in the philosophy of mind, a system has such attitudes if and only if its behavior is well explained by the hypothesis that it does. Interpretationists may hold that LLMs have attitudes, and thus have a simple solution to the novel reference problem. We emphasize, however, that interpretationism is compatible with very simple creatures having attitudes and differs sharply from views that presuppose these attitudes require consciousness, sentience, or intelligence (topics about which we make no claims).

• The paper demonstrates that LLMs can generate novel references, challenging the view that they merely reproduce existing texts.
• It investigates whether these unpredictable references indicate a rudimentary form of cognitive agency akin to human interpretation.
• The analysis critiques earlier studies and calls for clear benchmarks connecting LLM outputs to real-world referents.

Understanding the Intricacies of LLM References

Exploring the Bibliotechnism Hypothesis

LLMs have fueled debates in various academic fields, particularly in how we understand their relationship with linguistic references. Some argue that LLMs, like conventional cultural technologies such as photocopiers or libraries, merely transmit existing information rather than create original content. This perspective, dubbed "bibliotechnism," suggests that any novel text produced by LLMs is actually derivative, meaning its significance is linked to the human-generated text it mirrors. However, a conceptual gap in this hypothesis surfaces when LLMs generate text that seems to invent references—naming previously anonymous entities or creating entirely new expressions. Can we still classify LLMs as mere cultural tools if they demonstrate such abilities that seem to mimic human creativity?

Aligning With Philosophical Interpretationism

The latest philosophical discussions delve into whether the seemingly novel references made by LLMs imply some form of agency, akin in certain respects to human-like cognitive states such as beliefs, desires, and intentions. The paper examines these queries through the lens of "interpretationism," a doctrine in the philosophy of mind which suggests that such mental states are ascribed to an entity if its behavior is thoroughly explicable by assuming it possesses these states. By analyzing the cases where LLMs appear to create new references, researchers propound that the best explanation might indeed include the existence of cognitive states, pushing the boundaries of how we conceive LLM capacities.

Delving into Previous Studies and Their Limitations

Prior scholarship has grappled with questions around the ability of LLMs to produce meaningful language. Some posit that genuine reference requires sensory experiences, while others believe that LLMs can meaningfully employ words based on inferential relationships within a conceptual framework. Nevertheless, these discussions have not fully addressed the challenges presented by LLM-generated novel text and reference. The assertion that LLMs are part of our language community, and hence capable of reference within it, is also scrutinized, emphasizing the necessity of establishing precise benchmarks that connect LLM output with real-world elements.

Assessing LLMs' Referential Generative Abilities

The paper goes further to present the concept of "novel reference," where LLMs use unprecedented names to denote novel entities. This defies the assumption that LLMs are purely imitative since no prior associations exist within the original human text for such references. The researchers scrutinize several potential sources of derivative meaning—ranging from the human feedback in model training to prompt generation and reader interpretation—all to discern whether they could account for LLMs' capacity for novel reference. Despite exploring these avenues, the paper arrives at the notion that LLMs might indeed harbor a rudimentary form of agency, given the complexity and originality of some of the text they generate.

In summary, the debate over whether LLMs possess a type of agency indicative of beliefs, desires, and intentions remains spirited, with the research suggesting that understanding this may require an extensive investigation of their behaviors. The intricacies of how LLMs process and generate language pose a fascinating window into artificial intelligence, touching on deeper philosophical questions about cognition and the essence of creativity.