Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment (2402.13956v3)

Abstract: Do LMs infer the semantics of text from co-occurrence patterns in their training data? Merrill et al. (2022) argue that, in theory, sentence co-occurrence probabilities predicted by an optimal LM should reflect the entailment relationship of the constituent sentences, but it is unclear whether probabilities predicted by neural LMs encode entailment in this way because of strong assumptions made by Merrill et al. (namely, that humans always avoid redundancy). In this work, we investigate whether their theory can be used to decode entailment relations from neural LMs. We find that a test similar to theirs can decode entailment relations between natural sentences, well above random chance, though not perfectly, across many datasets and LMs. This suggests LMs implicitly model aspects of semantics to predict semantic effects on sentence co-occurrence patterns. However, we find the test that predicts entailment in practice works in the opposite direction to the theoretical test. We thus revisit the assumptions underlying the original test, finding its derivation did not adequately account for redundancy in human-written text. We argue that better accounting for redundancy related to explanations might derive the observed flipped test and, more generally, improve computational models of speakers in linguistics.

• The paper demonstrates that while language models detect entailment above chance, the expected co-occurrence direction is often reversed.
• It reveals that human language frequently employs redundancy for emphasis and explanation, challenging non-redundant theoretical models.
• The study proposes using a flipped entailment test and regression model to more accurately capture semantic relationships in next-word prediction tasks.

Deciphering Entailment Relationships in LLMs through Next-Word Prediction

Introduction

LLMs (LMs), especially those trained on next-word prediction tasks, have been at the forefront of recent advancements in NLP. These models learn from vast amounts of text data, generating new content and understanding language nuances. One area of exploration is how LMs ascertain the semantic relationships between sentences, notably entailment. This research investigates whether the semantics of entailment—where one statement logically follows from another—can be derived from the co-occurrence probabilities found in the training data of neural LMs.

Co-occurrence and Entailment

Fundamental to this investigation is the hypothesis posited by Merrill et al. (2022) that the semantic intricacies of entailment can be decoded from LM predictions. The theory suggests that an optimal LM, by learning sentence co-occurrence probabilities that avoid redundancy as per human linguistic behavior, implicitly models entailment. Entailment relationships, therefore, should be determinable from these co-occurrence probabilities. However, practical application and testing of this theory reveal a discord between the theoretical expectations and empirical findings. Specifically, while entailment relations are detectable above chance, the directionality of the prediction—where higher co-occurrence probabilities should signify non-entailment—is often reversed. This discrepancy prompts a reevaluation of the underlying assumptions about linguistic redundancy and its avoidance in human-generated texts.

Empirical Evaluation and Surprises

The research undertook an empirical evaluation across various entailment benchmarks and a range of LMs. It consistently found that decoding entailment from LM probabilities does not conform entirely to theoretical predictions. Intriguingly, when the direction of the theoretical test is reversed, the modified (flipped) test more accurately detects entailments. This suggests that LMs, albeit imperfectly, capture semantic effects impacting sentence co-occurrence patterns, albeit in a manner opposite to theoretical expectations.

Dissecting this unexpected result required analyzing the performance of this flipped test across different linguistic phenomena and model types. The analysis indicated a complex relationship between an LM's ability to predict next tokens accurately and its capability to model entailment. Furthermore, the paper proposed learning a distributional entailment test by employing a regression model that weights co-occurrence probabilities, which confirmed the robustness and validity of the flipped test direction.

Re-examining Linguistic Redundancy

The authors delved deeper into the reasons behind the flipped test result by examining natural corpora for contextually entailed sentences. Contrary to the idealized non-redundant Gricean speakers model, they found that real human linguistic behavior often embraces redundancy for various communicative purposes, including emphasis and explanation. This observation challenges the foundational assumptions of the original entailment prediction model and suggests a need for more nuanced theories of pragmatic redundancy in language.

Theoretical and Practical Implications

This work opens up new avenues for understanding how semantic relationships like entailment are represented within the probabilistic frameworks of LMs. The surprising result about the flipped test direction not only indicates that LMs may implicitly learn semantic rules governing sentence co-occurrence but also that our theoretical models of linguistic behavior—specifically regarding redundancy—may need refinement.

Future Directions

Looking ahead, this research underscores the potential for using LMs as empirical testing grounds for linguistic theories, particularly those related to pragmatics and semantics. It encourages a more sophisticated examination of human-like linguistic redundancies and their implications for computational models. The intriguing findings from evaluating the flipped entailment test also highlight the importance of aligning theoretical linguistics with empirical data from neural LMs.

Conclusion

This paper provided a critical assessment of the capabilities of LLMs to understand and predict entailment relationships based on sentence co-occurrence probabilities. By challenging existing assumptions and exploring the discrepancies between theory and practice, it paves the way for a deeper understanding of the interface between language understanding and generative AI. Future work in this area will undoubtedly continue to refine our models' linguistic intuitions, pushing the boundaries of what artificial intelligences can achieve in understanding the complexities of human language.