Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasets (2403.17608v2)
Abstract: The widespread adoption of generative image models has highlighted the urgent need to detect artificial content, which is a crucial step in combating widespread manipulation and misinformation. Consequently, numerous detectors and associated datasets have emerged. However, many of these datasets inadvertently introduce undesirable biases, thereby impacting the effectiveness and evaluation of detectors. In this paper, we emphasize that many datasets for AI-generated image detection contain biases related to JPEG compression and image size. Using the GenImage dataset, we demonstrate that detectors indeed learn from these undesired factors. Furthermore, we show that removing the named biases substantially increases robustness to JPEG compression and significantly alters the cross-generator performance of evaluated detectors. Specifically, it leads to more than 11 percentage points increase in cross-generator performance for ResNet50 and Swin-T detectors on the GenImage dataset, achieving state-of-the-art results. We provide the dataset and source codes of this paper on the anonymous website: https://www.unbiased-genimage.org
- Seeing is not always believing: Benchmarking human and model perception of ai-generated images. Advances in Neural Information Processing Systems, 36, 2024.
- Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
- Detecting and simulating artifacts in gan fake images. In 2019 IEEE international workshop on information forensics and security (WIFS), pages 1–6. IEEE, 2019.
- Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7890–7899, 2020.
- Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
- Towards the detection of diffusion model deepfakes. arXiv preprint arXiv:2210.14571, 2022.
- Intriguing properties of synthetic images: from generative adversarial networks to diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 973–982, 2023.
- Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24480–24489, 2023.
- Genimage: A million-scale benchmark for detecting ai-generated image. Advances in Neural Information Processing Systems, 36, 2024.
- Gendet: Towards good generalizations for ai-generated image detection. arXiv preprint arXiv:2312.08880, 2023.
- Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
- Cnn-generated images are surprisingly easy to spot… for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8695–8704, 2020.
- Dire for diffusion-generated image detection. arXiv preprint arXiv:2303.09295, 2023.
- Shadows don’t lie and lines can’t bend! generative models don’t know projective geometry… for now. arXiv preprint arXiv:2311.17138, 2023.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
- Are gan generated images easy to detect? a critical analysis of the state-of-the-art. In 2021 IEEE international conference on multimedia and expo (ICME), pages 1–6. IEEE, 2021.
- Towards universal gan image detection. In 2021 International Conference on Visual Communications and Image Processing (VCIP), pages 1–5. IEEE, 2021.
- Deep image fingerprint: Towards low budget synthetic image detection and model lineage analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4067–4076, 2024.
- Midjourney. https://www.midjourney.com, 2022.
- Stable Diffusion WebUI. https://github.com/AUTOMATIC1111/stable-diffusion-webui, 2022.
- Wukong. https://xihe.mindspore.cn/modelzoo/wukong, 2022.
- Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10696–10706, 2022.
- Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
- Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021.
- Large scale gan training for high fidelity natural image synthesis. international conference on learning representations. 2019.
- Online detection of ai-generated images. In ICCV DeepFake Analysis and Detection Workshop, 2023.
- towardsdatascience. Compression in the ImageNet dataset. https://towardsdatascience.com/compression-in-the-imagenet-dataset-34c56d14d463.
- A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019.
- Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
- Wouaf: Weight modulation for user attribution and fingerprinting in text-to-image diffusion models. arXiv preprint arXiv:2306.04744, 2023.
- Patrick Grommelt (1 paper)
- Louis Weiss (1 paper)
- Franz-Josef Pfreundt (22 papers)
- Janis Keuper (66 papers)