How Much Can We Forget about Data Contamination? (2410.03249v4)

Abstract: The leakage of benchmark data into the training data has emerged as a significant challenge for evaluating the capabilities of LLMs. In this work, we challenge the common assumption that small-scale contamination renders benchmark evaluations invalid. First, we experimentally quantify the magnitude of benchmark overfitting based on scaling along three dimensions: The number of model parameters (up to 1.6B), the number of times an example is seen (up to 144), and the number of training tokens (up to 40B). If model and data follow the Chinchilla scaling laws, minor contamination indeed leads to overfitting. At the same time, even 144 times of contamination can be forgotten if the training data is scaled beyond five times Chinchilla, a regime characteristic of many modern LLMs. Continual pre-training of OLMo-7B corroborates these results. Next, we study the impact of the weight decay parameter on example forgetting, showing that empirical forgetting occurs faster than the cumulative weight decay. This allows us to gauge the degree of example forgetting in large-scale training runs, indicating that many LLMs, including Lllama 3 405B, have forgotten the data seen at the beginning of training.

• The paper demonstrates that data contamination effects escalate with larger models and repeated exposure but diminish with an abundance of training tokens.
• The paper reveals rapid forgetting dynamics, quantifying the gradient steps needed to neutralize the influence of contaminated examples.
• The paper outlines practical implications for LLM training, showing that proper scaling and hyperparameter tuning can limit contamination impact.

Evaluating Data Contamination in LLMs

The paper "How much can we forget about Data Contamination?" confronts a pivotal concern in the assessment of LLMs — the potential leakage of benchmark data into training datasets. The researchers propose an exploration into the nuanced impacts of data contamination, challenging the notion that small-scale contamination categorically invalidates benchmark evaluations.

Main Contributions

The research employs both experimental evidence and theoretical insights to explore data contamination effects on model evaluations. Through a comprehensive methodology, the authors explore benchmark overfitting by manipulating three key dimensions: the number of model parameters, how often examples are presented, and the volume of training tokens.

1. Scaling Effects: The paper shows that the impact of data contamination is enhanced by an increase in model parameters and repetition of contaminated examples, but diminishes as the volume of training data expands beyond a certain threshold. Notably, even substantial contamination can be neutralized when training data significantly exceeds the Chinchilla scaling laws.
2. Forgetting Dynamics: Through experimental investigations, the authors reveal that neural networks undergo rapid forgetting of contaminated data, especially when training involves a continuous influx of novel information. The research quantifies the number of gradient steps necessary to forget past data based on known hyperparameters of the AdamW optimizer.
3. Practical Implications: The findings underscore that moderate contamination may not impact the evaluations of realistically scaled modern LLMs. This insight is critical for developers concerned about the overrepresentation of test samples in training data.

Theoretical Insights

The paper extends its empirical findings with a theoretical model of example forgetting, incorporating cumulative weight decay. This model provides an upper bound on the number of steps required before the influence of contaminated data fades completely. It also highlights the interplay between learning rate schedules and weight decay parameters, offering a new perspective on how these factors influence long-term retention of training examples.

Implications and Future Directions

This research has notable implications for AI development and evaluation practices:

• Training Regimens: The paper suggests that developers can mitigate contamination effects by expanding training datasets and selecting appropriate hyperparameters.
• Benchmark Reliability: By quantifying conditions under which benchmarks remain reliable, the paper provides a foundation for more accurate and fair evaluations of LLMs.
• Further Investigations: While the paper provides initial insights, further research on larger scales and diverse data contexts will be essential to validate and extend these findings.

In conclusion, the analysis presented indicates that while data contamination is a significant concern, its impact can be effectively managed. The theoretical and experimental frameworks proposed in the paper offer robust tools for understanding and mitigating the effects of contamination in modern LLM training pipelines.