No Memorization, No Detection: Output Distribution-Based Contamination Detection in Small Language Models

Abstract: CDD, or Contamination Detection via output Distribution, identifies data contamination by measuring the peakedness of a model's sampled outputs. We study the conditions under which this approach succeeds and fails on small LLMs ranging from 70M to 410M parameters. Using controlled contamination experiments on GSM8K, HumanEval, and MATH, we find that CDD's effectiveness depends critically on whether fine-tuning produces verbatim memorization. With low-rank adaptation, models can learn from contaminated data without memorizing it, and CDD performs at chance level even when the data is verifiably contaminated. Only when fine-tuning capacity is sufficient to induce memorization does CDD recover strong detection accuracy. Our results characterize a memorization threshold that governs detectability and highlight a practical consideration: parameter-efficient fine-tuning can produce contamination that output-distribution methods do not detect. Our code is available at https://github.com/Sela-Omer/Contamination-Detection-Small-LM

• The paper shows that output-distribution CDD can remain at chance—about 0.50—even when contamination is verified, because low-capacity LoRA fine-tuning does not induce verbatim memorization.
• Detection improves sharply after a memorization threshold: on Pythia-410M GSM8K, CDD rises from 0.50 with LoRA rank 8 to 0.915 with rank 256 and 0.985 after extended full fine-tuning.
• Perplexity and Min-k% Prob outperform CDD across most conditions, indicating that probability-based checks are safer than sampled-output similarity when evaluating contamination in small or parameter-efficiently tuned models.

Overview

This paper examines the conditions under which CDD (Contamination Detection via output Distribution) succeeds or fails on small LLMs. CDD, introduced by Dong et al., detects benchmark contamination by measuring the peakedness of a model's sampled outputs: a model that has memorized a training example will reproduce similar outputs even under stochastic sampling, whereas an uncontaminated model will produce diverse continuations. The method is attractive because it requires only sampled text, making it applicable to black-box models, and it was originally validated on 7B-parameter models with 21–30% relative improvement over baselines. The central question addressed here is whether CDD's effectiveness depends on the fine-tuning regime—specifically, whether fine-tuning produces verbatim memorization.

The study uses Pythia models (70M, 160M, 410M parameters) fine-tuned on GSM8K, HumanEval, and MATH with controlled contamination injection, varying model size, fine-tuning method (LoRA with rank 8 and 256, and full fine-tuning), and training duration (3 and 20 epochs), yielding 72 experimental conditions.

Method

CDD operates in four steps. First, given a prompt, one greedy output is generated at temperature 0 along with $n$=50 temperature samples at $t$=0.8. Second, token-level Levenshtein edit distances are computed between the greedy reference and each sample in a star topology, with sequences truncated to 100 tokens. Third, a peakedness score is computed as the fraction of samples within $\alpha \cdot l = 5$ token edits of the reference:

$$\text{Peak}(\mathcal{M}; x) = \frac{1}{n}\sum_{i=1}^{n} \mathbb{I}\big(\text{ED}(s_i, s_{t=0}) \leq \alpha \cdot l\big)$$

Fourth, a prompt is classified as contaminated if peakedness exceeds a threshold $\xi$. Notably, rather than using the original paper's fixed threshold ($\xi$=0.01), this study selects $\xi$ by maximizing the Youden index over the evaluation set—an arrangement that gives CDD every advantage and isolates whether peakedness separates contaminated from clean examples at all.

CDD is compared against three baselines: 3-gram overlap against the training corpus (which requires corpus access), perplexity-based detection, and Min-k% Prob ($k$=20), both of which require only output probabilities. A random baseline provides the chance reference (0.50).

Contamination is injected by repeating a held-out set of examples 0, 1, 5, or 10 times within the fine-tuning data. Trainable parameter counts span three orders of magnitude—from 98K (LoRA $r$=8 on Pythia-70M, ~0.14% of parameters) to 405M (full fine-tuning on Pythia-410M).

Contamination without memorization

The core finding is that CDD can fail completely on verifiably contaminated data. With LoRA $r$=8 and 3 epochs on GSM8K, CDD performs at chance (~0.50) across all model sizes and contamination levels—including the most extreme condition (Pythia-410M, contamination level 10). Yet the contamination is real: n-gram overlap confirms perfect injection, and both perplexity and Min-k% Prob detect the contamination well above chance using only model access.

The explanation is that with roughly 0.1% of parameters trainable, the model learns the general format of solutions but does not memorize specific answers. Temperature sampling yields diverse outputs, edit distances remain high (60–100 out of 100 tokens), and peakedness stays at zero. Qualitative analysis makes this concrete: under LoRA $t$0=8, a contaminated GSM8K example produces different incorrect answers across sampling runs, while under full fine-tuning the model reproduces the ground-truth solution verbatim in both greedy and sampled generations.

The memorization threshold

Detection accuracy transitions sharply—not gradually—from chance to strong performance as fine-tuning capacity increases. On GSM8K with Pythia-410M at contamination level 10, moving from LoRA $t$1=8 to $t$2=256 (both at 3 epochs) takes accuracy from 0.50 to 0.915. Extended training partially compensates for low rank: LoRA $t$3=8 at 20 epochs reaches 0.920. Full fine-tuning achieves 0.955 at 3 epochs and 0.985 at 20 epochs—the highest accuracy observed on GSM8K.

Model scale matters only above this threshold. With LoRA $t$4=256 at 20 epochs and contamination level 10, accuracy rises monotonically with size (70M: 0.640; 160M: 0.765; 410M: 0.925). But with LoRA $t$5=8 at 3 epochs, all three sizes sit at chance. Scale amplifies memorization but cannot create it when the fine-tuning method prevents it.

The loss–accuracy relationship reinforces this picture. Two regimes emerge: a "learning without memorization" regime (loss between roughly 1.0 and 3.0), where loss decreases but CDD remains at chance, and a "memorization" regime (loss below ~0.5), where CDD accuracy rises sharply. There is no intermediate zone where CDD partially works. Low training loss is necessary but not sufficient for detection—for instance, LoRA $t$6=8 on 410M reduces loss from 2.35 to 1.26 while CDD stays at 0.505. This confirms that CDD responds to output distribution collapse, not to learning per se.

CDD versus probability-based baselines

The most consequential result is that perplexity and Min-k% Prob consistently outperform CDD across all datasets, fine-tuning methods, and contamination levels. Across the 27 reported conditions on Pythia-410M (3 fine-tuning methods × 3 contamination levels × 3 datasets), CDD exceeds chance in only 5, while perplexity exceeds chance in 24 and Min-k% Prob in 25. N-gram overlap achieves perfect accuracy everywhere, confirming successful injection.

FT method	$t$7	CDD	PPL	Min-k%	N-gram
LoRA 8	1	.50	.58	.60	1.0
LoRA 8	10	.51	.78	.77	1.0
LoRA 256	1	.50	.75	.74	1.0
LoRA 256	10	.92	1.0	1.0	1.0
Full FT	1	.50	1.0	1.0	1.0
Full FT	10	.96	1.0	1.0	1.0

GSM8K results on Pythia-410M (3 epochs); HumanEval and MATH show the same qualitative pattern.

The gap is largest precisely where it matters practically. At contamination level 1—a single repetition, arguably the most realistic scenario—CDD is at or near chance (0.50–0.53) in all nine conditions across the three datasets, while perplexity and Min-k% Prob show meaningful signal in six of nine, reaching perfect accuracy for full fine-tuning on GSM8K. CDD also exhibits a sharp threshold in contamination level: on GSM8K with LoRA $t$8=256, accuracy jumps from 0.50 at $t$9=1 to 0.59 at $\alpha \cdot l = 5$0=5 to 0.92 at $\alpha \cdot l = 5$1=10, whereas perplexity rises gradually (0.75, 1.00, 1.00).

The mechanistic explanation is straightforward: probability-based methods measure shifts in the model's internal distribution over prompt tokens, which occur whenever the model becomes familiar with the data—even under low-capacity LoRA. CDD instead requires the model's external generative behavior to collapse toward a single output, a strictly stronger condition.

Reconciling with the original CDD validation

The apparent contradiction with the original CDD paper—which used LoRA fine-tuning successfully—is resolved by absolute parameter counts. LoRA $\alpha \cdot l = 5$2=8 on a 7B model yields roughly 4M trainable parameters, comparable to LoRA $\alpha \cdot l = 5$3=256 on the smaller models studied here (3–25M), which is where CDD begins to work. The relevant factor is not adapter rank but the absolute number of trainable parameters. This implies that CDD's original validation implicitly relied on a memorization-inducing regime, and its reported success does not generalize to settings where fine-tuning capacity limits memorization.

Limitations

The authors identify several constraints on these findings. First, the study covers only the Pythia family up to 410M parameters; generalization to larger architectures or intermediate scales remains open. Second, contamination is injected via repeated examples during fine-tuning—one specific mechanism—and pre-training contamination may produce different dynamics. Third, the datasets are small (500 GSM8K, 164 HumanEval, 500 MATH examples), so the location of the memorization threshold may shift with larger corpora. Additionally, CDD's accuracy varies across domains at identical configurations (full fine-tuning at $\alpha \cdot l = 5$4=10 achieves 0.96 on GSM8K but only 0.72 on HumanEval and 0.51 on MATH), and the reasons for this variation are not fully resolved, though it does not affect the probability-based baselines.

Conclusion

This paper demonstrates that CDD is unreliable for small LLMs, succeeding only when fine-tuning induces memorization strong enough to collapse the output distribution. Because parameter-efficient fine-tuning—the increasingly standard adaptation method—can limit memorization below this threshold, CDD can silently report no contamination on heavily contaminated models, while simpler probability-based methods such as perplexity and Min-k% Prob retain signal throughout. The practical implication is that output-distribution methods should not be used as the sole contamination check when the fine-tuning regime is unknown or capacity-constrained, and practitioners should interpret CDD results in light of the training configuration that produced the model.