Will Large-scale Generative Models Corrupt Future Datasets? (2211.08095v2)

Abstract: Recently proposed large-scale text-to-image generative models such as DALL$\cdot$E 2, Midjourney, and StableDiffusion can generate high-quality and realistic images from users' prompts. Not limited to the research community, ordinary Internet users enjoy these generative models, and consequently, a tremendous amount of generated images have been shared on the Internet. Meanwhile, today's success of deep learning in the computer vision field owes a lot to images collected from the Internet. These trends lead us to a research question: "\textbf{will such generated images impact the quality of future datasets and the performance of computer vision models positively or negatively?}" This paper empirically answers this question by simulating contamination. Namely, we generate ImageNet-scale and COCO-scale datasets using a state-of-the-art generative model and evaluate models trained with "contaminated" datasets on various tasks, including image classification and image generation. Throughout experiments, we conclude that generated images negatively affect downstream performance, while the significance depends on tasks and the amount of generated images. The generated datasets and the codes for experiments will be publicly released for future research. Generated datasets and source codes are available from \url{https://github.com/moskomule/dataset-contamination}.

Large-scale Generative Models and Their Impact on Future Datasets

The paper "Will Large-scale Generative Models Corrupt Future Datasets?" investigates the potential implications of utilizing large-scale text-to-image generative models on the integrity and reliability of future datasets used in computer vision. With models like DALL·E 2, Midjourney, and StableDiffusion gaining popularity, the Internet is witnessing an influx of generated images that may inadvertently become part of training datasets for future deep learning models. This paper posits a critical question: How do these generative images impact the quality and efficacy of datasets used for training computer vision models?

Research Context and Methods

The backdrop to this research is the proliferation of generative models that create realistically convincing images based on textual prompts. This paper hypothesizes that among the consequences of this technological advancement is the potential contamination of datasets—datasets crucial for training future models. The authors take a pragmatic approach to this hypothesis by simulating ImageNet-scale and COCO-scale datasets infused with generated images and assessing their influence on model performance across tasks like image classification, captioning, and generation.

To explore these implications, the authors used state-of-the-art generative models to create datasets such as SD-ImageNet and SD-COCO by generating images that correspond to ImageNet categories and COCO captions. They then conducted a series of experiments to evaluate how models trained on these "contaminated" datasets perform in various benchmark tasks.

Key Findings

The findings consistently indicate a degradation in downstream model performance in all primary tasks examined:

1. Image Classification: Evaluations showed that as the proportion of generated images within a dataset increases, classification accuracy decreases significantly. Notably, when 80% of the dataset comprised generated images, the accuracy dropped drastically.
2. Image Captioning: Performance metrics on these tasks also revealed a decline. The models fine-tuned on mixtures of generated and real datasets exhibited reduced BLEU, SPICE, and CIDEr scores when compared to those trained exclusively on authentic data.
3. Image Generation: For models tasked with generating images, the quality measures indicated that datasets mingled with generated images could lead to outputs that deviate more from real data distributions, as captured by metrics such as Fréchet Inception Distance (FID).

Discussion on Dataset Integrity and Broader Implications

These findings raise significant concerns about the integrity of future image datasets inadvertently containing generated images. The studies reported empirical declines in not only direct task performance but also robustness to real-world distribution shifts, further suggesting that current generative models do not encapsulate the full diversity of real-world data. Additionally, the paper highlights the difficulty in detecting generated images, as current methods based on frequency domain differences fall short with diffusion model outputs.

The potential negative implications on dataset quality demand actionable considerations. Future datasets must incorporate mechanisms to discern generated images or adopt strategies that mitigate such effects, such as enforcing image origin tracking. Furthermore, the role of self-supervised learning, as positively indicated in this paper, could provide resilience to models built on mixed datasets by focusing on feature extraction without explicit reliance on labeled data.

Future Directions

While this research provides a foundational understanding of the issue, it opens several pathways for future paper. It necessitates deeper exploration into adaptive techniques for dataset curation and refinement of generative models to enhance diversity representation. As generative technologies continue to evolve, so too will the need for strategies that preserve the integrity and utility of datasets critical for advancing machine learning frontiers.

In summary, the paper elucidates the unintended consequences of generative models on data ecosystems foundational to AI progress. It emphasizes the urgency for establishing robust mechanisms to safeguard future datasets against the creeping influence of synthetic data, thus ensuring the continued reliability and evolution of computer vision technologies.