Evaluation data contamination in LLMs: how do we measure it and (when) does it matter? (2411.03923v1)

Abstract: Hampering the interpretation of benchmark scores, evaluation data contamination has become a growing concern in the evaluation of LLMs, and an active area of research studies its effects. While evaluation data contamination is easily understood intuitively, it is surprisingly difficult to define precisely which samples should be considered contaminated and, consequently, how it impacts benchmark scores. We propose that these questions should be addressed together and that contamination metrics can be assessed based on whether models benefit from the examples they mark contaminated. We propose a novel analysis method called ConTAM, and show with a large scale survey of existing and novel n-gram based contamination metrics across 13 benchmarks and 7 models from 2 different families that ConTAM can be used to better understand evaluation data contamination and its effects. We find that contamination may have a much larger effect than reported in recent LLM releases and benefits models differently at different scales. We also find that considering only the longest contaminated substring provides a better signal than considering a union of all contaminated substrings, and that doing model and benchmark specific threshold analysis greatly increases the specificity of the results. Lastly, we investigate the impact of hyperparameter choices, finding that, among other things, both using larger values of n and disregarding matches that are infrequent in the pre-training data lead to many false negatives. With ConTAM, we provide a method to empirically ground evaluation data contamination metrics in downstream effects. With our exploration, we shed light on how evaluation data contamination can impact LLMs and provide insight into the considerations important when doing contamination analysis. We end our paper by discussing these in more detail and providing concrete suggestions for future work.

• The paper presents the innovative ConTAM framework to quantify contamination impact on LLM performance using Estimated Performance Gain (EPG).
• It demonstrates that longer contiguous match metrics more accurately identify meaningful contamination across various benchmarks.
• Findings indicate that model scale critically interacts with contamination effects, influencing evaluation outcomes in larger LLMs.

Understanding Evaluation Data Contamination in LLMs: A Critical Assessment

The subject of evaluation data contamination has emerged as a pivotal area of concern in the field of LLMs. This paper, "Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?" addresses the complexities involved in defining, measuring, and understanding the repercussions of data contamination, which complicates the evaluation of LLMs.

The authors introduce the Contamination Threshold Analysis Method (ConTAM), an innovative framework designed to scrutinize the contamination levels across benchmarks and its potential impact on LLM performance. Key to their approach is the estimation of Estimated Performance Gain (EPG), used to empirically evaluate the impact contaminated examples have on model performance.

Upon investigating contamination across four distinct metrics—three derived from existing literature and one novel contribution—the paper reports findings from extensive testing across 13 benchmarks and 7 models. Notably, the longest-match metric, which focuses on the longest contiguous span between examples and pre-training corpora rather than all possible matches, provides superior insight into legitimately impactful contamination. This implies a more accurate detection of contamination that models could leverage to improve performance beyond general learning abilities.

Quantitatively, the findings reveal two noteworthy aspects: (1) reports from earlier LLM publications may have underestimated contamination impact due to potential false negatives or positives in contamination metrics, and (2) certain benchmarks like GSM8K and MATH demonstrate significant EPG in larger models, such as those from the Llama family, underscoring the notion that model scale interacts critically with contamination.

The authors further delve into various parameter choices affecting contamination metrics, such as the size of $n$ in n-gram calculations, minimum counts of occurrences, and skip budgets. Their results highlight that smaller $n$-grams more effectively detect meaningful contamination, while higher minimum counts unexpectedly lead to oversight of impactful data.

Implications of this research traverse both theoretical and practical domains. Theoretically, the paper provides a calibrated lens to view and assess contamination beyond superficial overlap metrics, advancing the discourse on model evaluation purity versus utility. Practically, these results offer clear guidance for data scientists in industry and academia on selecting contamination metrics that meaningfully contribute to understanding model performance.

As AI continues evolving, understanding the implications of benchmark evaluation dynamics will remain crucial. This paper lays a foundation for future exploration, beckoning toward embedding-based approaches and causal analyses to further disentangle the nuanced relationships between LLM training data and evaluation benchmarks.

In summary, the work presents a rigorous, empirical examination of evaluation data contamination, delivering concrete methodology and insights into its often-overlooked impacts on LLM evaluation, setting a precedent for a more meticulous assessment of model benchmarks as AI capabilities continue to surge.