Large Language Models Struggle to Learn Long-Tail Knowledge (2211.08411v2)

Abstract: The Internet contains a wealth of knowledge -- from the birthdays of historical figures to tutorials on how to code -- all of which may be learned by LLMs. However, while certain pieces of information are ubiquitous on the web, others appear extremely rarely. In this paper, we study the relationship between the knowledge memorized by LLMs and the information in pre-training datasets scraped from the web. In particular, we show that a LLM's ability to answer a fact-based question relates to how many documents associated with that question were seen during pre-training. We identify these relevant documents by entity linking pre-training datasets and counting documents that contain the same entities as a given question-answer pair. Our results demonstrate strong correlational and causal relationships between accuracy and relevant document count for numerous question answering datasets (e.g., TriviaQA), pre-training corpora (e.g., ROOTS), and model sizes (e.g., 176B parameters). Moreover, while larger models are better at learning long-tail knowledge, we estimate that today's models must be scaled by many orders of magnitude to reach competitive QA performance on questions with little support in the pre-training data. Finally, we show that retrieval-augmentation can reduce the dependence on relevant pre-training information, presenting a promising approach for capturing the long-tail.

LLMs and the Challenge of Long-Tail Knowledge Acquisition

The paper entitled LLMs Struggle to Learn Long-Tail Knowledge presents a comprehensive investigation into the limitations of current LLMs in capturing so-called long-tail knowledge. LLMs have demonstrated remarkable potential in various natural language tasks by leveraging vast amounts of text data sourced from the Internet. However, the authors argue that while these models perform well on commonly found information, they struggle to grasp and recall less frequent, or long-tail, facts that do not have widespread representation in their pre-training datasets.

Methodology and Key Findings

The research rigorously explores the relationship between an LLM’s ability to provide accurate responses in fact-based question answering (QA) tasks and the frequency of related documents in the pre-training data. Utilizing entity linking techniques, the paper identifies relevant documents by tracking co-occurrences of pertinent entities within the pre-training corpora and question-answer (QA) pairs. This entity linking is performed over extensive pre-training datasets including C4, The Pile, ROOTS, OpenWebText, and Wikipedia.

The paper evidences a direct correlation between document frequency and QA performance across various model families like GPT-Neo, BLOOM, and GPT-3. A significant increase in model accuracy is observed with higher document frequency. For example, BLOOM-176B shows an increase in accuracy from 25% to over 55% as the number of related documents increases. This finding is further validated through counterfactual experiments where re-training models without certain documents resulted in a marked decrease in accuracy.

Implications

The findings indicate that while scaling model parameters and data size contributes to better performance on knowledge-intensive tasks, it is insufficient for achieving high accuracy on questions pertaining to the long tail. The paper critically assesses the potential for scaling data and models, suggesting that exponential increases in model size, potentially reaching up to one quadrillion parameters, might be necessary, which poses significant feasibility issues.

Retrieval-augmented models offer a promising alternative by supplementing LLMs with external document retrieval capabilities, thereby mitigating the dependency on pre-training document frequency. When retrievers successfully identify relevant documents, the models demonstrate improved performance on rare facts, emphasizing the potential impact of retrieval strategies in enhancing QA systems.

Future Directions and Conclusions

The research opens several important avenues for future exploration in both the practical and theoretical realms of AI. It suggests that LLMs' limitations on long-tail knowledge might be addressed through advancements in retrieval-augmentation techniques, potentially leading to more efficient and smarter integration of auxiliary retrieval systems.

Moreover, the paper underlines the necessity for alternative training objectives or strategies, such as increasing epochs or modifying the learning curriculum, which may enhance the memorization capabilities of LLMs without relying solely on brute force scaling.

Overall, this work contributes valuable insights into understanding the dependency of LLMs on pre-training data and the inherent imbalance in their ability to learn from sparse knowledge events. It underscores the need for innovative approaches in both LLM architecture and training methodologies to truly harness the expansive but uneven knowledge available across the digital text landscape.