Both Text and Images Leaked! A Systematic Analysis of Multimodal LLM Data Contamination (2411.03823v2)

Abstract: The rapid progression of multimodal LLMs (MLLMs) has demonstrated superior performance on various multimodal benchmarks. However, the issue of data contamination during training creates challenges in performance evaluation and comparison. While numerous methods exist for detecting models' contamination in LLMs, they are less effective for MLLMs due to their various modalities and multiple training phases. In this study, we introduce a multimodal data contamination detection framework, MM-Detect, designed for MLLMs. Our experimental results indicate that MM-Detect is quite effective and sensitive in identifying varying degrees of contamination, and can highlight significant performance improvements due to the leakage of multimodal benchmark training sets. Furthermore, we explore whether the contamination originates from the base LLMs used by MLLMs or the multimodal training phase, providing new insights into the stages at which contamination may be introduced.

• The paper's main contribution is the development of MM-Detect, a framework that differentiates between unimodal and cross-modal data contamination in MLLMs.
• It employs innovative tests like the Option Order Sensitivity Test and Slot Guessing for Perturbation Caption to systematically evaluate contamination across 11 diverse models.
• Experimental results reveal that data contamination can skew model evaluations, emphasizing the need for standardized and contamination-free training protocols.

Systematic Analysis of Multimodal Data Contamination in LLMs

The paper addresses a critical yet underexplored challenge in the development and utilization of multimodal LLMs (MLLMs): data contamination. As MLLMs continue to outperform benchmarks, the threat of misleading results due to data contamination from training or test datasets becomes increasingly significant. This paper introduces a comprehensive framework, MM-Detect, to systematically detect and analyze data contamination in MLLMs.

Key Contributions

1. Definition and Classification of Contamination: The authors provide a cogent definition of multimodal data contamination, distinguishing between unimodal contamination—derived from text input—and cross-modal contamination—which surfaces from pairs of text and image data. This distinction is crucial for understanding potential sources of bias within MLLMs.
2. Proposed MM-Detect Framework: Two innovative techniques form the crux of MM-Detect:
   • The Option Order Sensitivity Test evaluates a model’s bias towards a specific order in multiple-choice tasks, with deviations suggesting potential contamination.
   • The Slot Guessing for Perturbation Caption method assesses a model's ability to predict missing words before and after back-translation, using discrepancies to infer data leakage.
3. Experimental Analysis Across Models and Datasets: The framework was applied to 11 diverse MLLMs, including both open-source and proprietary models, across widely-used datasets. This exhaustive evaluation revealed pervasive contamination across models and tasks. Notably, the framework could pinpoint contamination levels on both dataset and instance levels, underscoring disparities between open-source and proprietary systems.
4. Heuristic Examination of Contamination Sources: Employing a heuristic approach, the authors were able to trace back some contamination to the pre-training phases of LLMs—indicating that such contamination might not solely arise from the multimodal training phases.

Implications and Future Directions

The findings suggest that data contamination significantly skews model evaluation, leading to potential misrepresentation of a model’s actual capabilities. The detection and correction of such contamination is thus imperative for reliable MLLM development. The framework’s ability to discern nuances in contamination at both data and model levels showcases its utility as a robust tool for the AI research community.

Future work could expand upon this paper by standardizing dataset usage to preclude contamination, thus enhancing the reliability and comparability of model assessments. Additionally, broadening the scope of modalities to include audio or video could significantly enrich our understanding of multimodal model robustness.

The MM-Detect framework’s contributions offer a vital lens through which the AI community can re-evaluate benchmark standards and development practices. This systematic detection approach can serve as a foundation for more transparent and equitable AI development processes.