Data Distributional Properties Drive Emergent In-Context Learning in Transformers (2205.05055v6)

Abstract: Large transformer-based models are able to perform in-context few-shot learning, without being explicitly trained for it. This observation raises the question: what aspects of the training regime lead to this emergent behavior? Here, we show that this behavior is driven by the distributions of the training data itself. In-context learning emerges when the training data exhibits particular distributional properties such as burstiness (items appear in clusters rather than being uniformly distributed over time) and having large numbers of rarely occurring classes. In-context learning also emerges more strongly when item meanings or interpretations are dynamic rather than fixed. These properties are exemplified by natural language, but are also inherent to naturalistic data in a wide range of other domains. They also depart significantly from the uniform, i.i.d. training distributions typically used for standard supervised learning. In our initial experiments, we found that in-context learning traded off against more conventional weight-based learning, and models were unable to achieve both simultaneously. However, our later experiments uncovered that the two modes of learning could co-exist in a single model when it was trained on data following a skewed Zipfian distribution -- another common property of naturalistic data, including language. In further experiments, we found that naturalistic data distributions were only able to elicit in-context learning in transformers, and not in recurrent models. In sum, our findings indicate how the transformer architecture works together with particular properties of the training data to drive the intriguing emergent in-context learning behaviour of LLMs, and how future work might encourage both in-context and in-weights learning in domains beyond language.

Overview of "Data Distributional Properties Drive Emergent In-Context Learning in Transformers"

This paper explores the underlying factors that enable large transformer-based models to perform in-context few-shot learning without explicit training for this purpose. The research identifies that specific distributional properties of training data drive the emergence of this ability. The paper focuses on burstiness, having a large set of rarely occurring classes, and dynamic meanings as key properties. These are common features of natural language but are relatively unexplored in standard supervised learning models, which typically use uniform, i.i.d data distributions.

The experiments reveal that in-context learning emerges when the data possesses such naturalistic properties, and that these, rather than solely the transformer architecture, are essential. The authors provide evidence that these properties can lead to a trade-off with conventional weight-based learning but find that both learning modes can coexist when trained on data with a skewed Zipfian distribution—a naturally occurring characteristic in many real-world datasets, including languages.

Experimental Approach

The authors conducted experiments using the Omniglot dataset, allowing for precise control over training data properties. Models were trained using sequences with varying burstiness and class diversity. It was found that a combination of burstiness and large numbers of classes favored the emergence of in-context learning.

Further manipulation of the data to include dynamic item interpretations, through multiplicity of labels and within-class variation, also biased models towards in-context learning. Importantly, recurrent models did not demonstrate the same learning ability when trained on identical data distributions, underscoring the importance of the transformer architecture in realizing these findings.

Numerical Results

The results demonstrated that models trained under conditions of increased burstiness and a large number of classes showed higher in-context learning performance, though often at the expense of in-weights learning. However, training on Zipfian distributions allowed models to balance both types of learning effectively, offering a potential explanation for in-context learning seen in LLMs like GPT-3.

A Zipf exponent of approximately 1, which aligns with statistics observed in natural languages, provided a sweet spot in which models retained both rapid in-context learning and robust in-weights memory retention, suggesting that real-world data naturally encourages these dual capabilities in transformers.

Theoretical and Practical Implications

These findings shed light on why LLMs exhibit emergent learning abilities without explicit training protocols targeting such features. They emphasize that LLMs benefit from the inherent non-uniformity of naturalistic datasets, which could be exploited beyond the domain of language. By designing datasets with these properties, other AI systems might benefit from similar emergent capabilities.

This research also offers insights into cognitive science, suggesting parallels in human learning patterns where exposure to bursty or skewed distributions may play a role in voluntary, rapid learning abilities. The work also furthers our understanding of complementary learning systems, drawing comparisons between in-context learning in AI and human hippocampal function.

Future Directions

Future research could explore the interactions between distributional properties and different learning algorithms, particularly reinforcement learning. Additional lines of inquiry could focus on how symbolic inputs and context-dependent word meanings might affect learning outcomes. Understanding the disparity between transformers and recurrent models in these tasks can enrich the design of machine learning architectures and their applications.

In conclusion, while the architecture of transformers is crucial, the form and properties of the data are equally important for evoking the full range of emergent learning behaviors seen in state-of-the-art models. This paper serves as a precursor to more conscious data design, aiming to enhance AI capabilities across diverse applications while deepening the syncretism between artificial and natural learning phenomena.