Develop comprehensive methods for tabular contamination detection beyond row-level deduplication

Develop comprehensive contamination detection and decontamination methodologies for tabular datasets used in training and evaluating foundation models that go beyond row-level deduplication, accounting for column-name variations, multi-source duplication, and task-level leakage that can make evaluation tasks solvable via memorized associations rather than tabular reasoning.

Background

The paper demonstrates multiple contamination modes in TLM training corpora—complete overlap, direct label exposure across duplicated tables, and task-level leakage—showing that standard row-level deduplication fails to detect or mitigate these issues. Given the vast and heterogeneous nature of tabular corpora, robust detection methods are necessary to ensure valid evaluation.

The authors note the absence of established best practices and emphasize that exhaustive search of large tabular corpora is impractical, underscoring the need for principled, scalable alternatives to current ad hoc approaches.

References

Contamination detection for tabular data lacks established best practices and row-level deduplication is insufficient (as we demonstrated), but comprehensive alternatives remain an open problem.

— The Illusion of Generalization: Re-examining Tabular Language Model Evaluation  (2602.04031 - Gorla et al., 3 Feb 2026) in Section: Limitations

Exact deduplication only. The diversity-aware selection step removes exact input-string and column duplicates but does not penalise high-correlation near-duplicates; soft column-similarity criteria (e.g., cosine-similarity clustering over execution-profile embeddings) may recover additional signal and are an open direction.

— Information-Gain Rewards over Diversity-Pruned Tests: GT-Anchored Verifier Co-Training for Reliable Code Generation  (2609.21208 - Nunez et al., 18 Sep 2026) in Limitations, Section 6