Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attacks (2407.20836v1)

Published 30 Jul 2024 in cs.CV and cs.CR

Abstract: Recent advancements in image synthesis, particularly with the advent of GAN and Diffusion models, have amplified public concerns regarding the dissemination of disinformation. To address such concerns, numerous AI-generated Image (AIGI) Detectors have been proposed and achieved promising performance in identifying fake images. However, there still lacks a systematic understanding of the adversarial robustness of these AIGI detectors. In this paper, we examine the vulnerability of state-of-the-art AIGI detectors against adversarial attack under white-box and black-box settings, which has been rarely investigated so far. For the task of AIGI detection, we propose a new attack containing two main parts. First, inspired by the obvious difference between real images and fake images in the frequency domain, we add perturbations under the frequency domain to push the image away from its original frequency distribution. Second, we explore the full posterior distribution of the surrogate model to further narrow this gap between heterogeneous models, e.g. transferring adversarial examples across CNNs and ViTs. This is achieved by introducing a novel post-train Bayesian strategy that turns a single surrogate into a Bayesian one, capable of simulating diverse victim models using one pre-trained surrogate, without the need for re-training. We name our method as frequency-based post-train Bayesian attack, or FPBA. Through FPBA, we show that adversarial attack is truly a real threat to AIGI detectors, because FPBA can deliver successful black-box attacks across models, generators, defense methods, and even evade cross-generator detection, which is a crucial real-world detection scenario.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning. PMLR, 274–283.
  2. Large Scale GAN Training for High Fidelity Natural Image Synthesis. In International Conference on Learning Representations.
  3. Nicholas Carlini and Hany Farid. 2020. Evading deepfake-image detectors with white-and black-box attacks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 658–659.
  4. What makes fake images detectable? understanding properties that generalize. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16. Springer, 103–120.
  5. A closer look at fourier spectrum discrepancies for cnn-generated images detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7200–7209.
  6. AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors. arXiv preprint arXiv:2310.17419 (2023).
  7. Intriguing properties of synthetic images: from generative adversarial networks to diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 973–982.
  8. On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5.
  9. Raising the Bar of AI-generated Image Detection with CLIP. arXiv preprint arXiv:2312.00195 (2023).
  10. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255.
  11. Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34 (2021), 8780–8794.
  12. BASAR: black-box attack on skeletal action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7597–7607.
  13. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9185–9193.
  14. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
  15. Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7890–7899.
  16. Fourier spectrum discrepancies in deep network generated images. Advances in neural information processing systems 33 (2020), 3022–3032.
  17. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning. PMLR, 3247–3258.
  18. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
  19. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
  20. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  21. Forgerynet: A versatile benchmark for comprehensive forgery analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4360–4369.
  22. Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840–6851.
  23. Evading DeepFake Detectors via Adversarial Statistical Consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12271–12280.
  24. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).
  25. Adversarial deepfakes: Evaluating vulnerability of deepfake detectors to adversarial examples. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 3348–3357.
  26. What are Bayesian neural network posteriors really like?. In International conference on machine learning. PMLR, 4629–4640.
  27. Fingerprintnet: Synthesized fingerprints for generated image detection. In European Conference on Computer Vision. Springer, 76–94.
  28. Exploring frequency adversarial attacks for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4103–4112.
  29. Adv-attribute: Inconspicuous and transferable adversarial attack on face recognition. Advances in Neural Information Processing Systems 35 (2022), 34136–34147.
  30. Glff: Global and local feature fusion for ai-synthesized image detection. IEEE Transactions on Multimedia (2023).
  31. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations.
  32. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4401–4410.
  33. Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114
  34. Adversarial examples in the physical world. In Artificial intelligence safety and security. Chapman and Hall/CRC, 99–112.
  35. Exploring adversarial fake images on face manifold. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5789–5798.
  36. Detecting generated images by real images. In European Conference on Computer Vision. Springer, 95–110.
  37. Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection. arXiv preprint arXiv:2312.16649 (2023).
  38. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.
  39. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8060–8069.
  40. Frequency domain model augmentation for adversarial attack. In European Conference on Computer Vision. Springer, 549–566.
  41. Detecting images generated by deep diffusion models using their local intrinsic dimensionality. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 448–459.
  42. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).
  43. Midjourney. 2022. https://www.midjourney.com/home/.
  44. Adversarial threats to deepfake detection: A practical perspective. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 923–932.
  45. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 24480–24489.
  46. On the Exploitation of DCT-Traces in the Generative-AI Domain. arXiv preprint arXiv:2402.02209 (2024).
  47. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748–8763.
  48. AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error. arXiv preprint arXiv:2401.17879 (2024).
  49. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695.
  50. Evading Forensic Classifiers with Attribute-Conditioned Adversarial Faces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16469–16478.
  51. Bayesian optimization with robust Bayesian neural networks. Advances in neural information processing systems 29.
  52. Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12105–12114.
  53. Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105–6114.
  54. Defending black-box skeleton-based human activity classifiers. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 2546–2554.
  55. CNN-generated images are surprisingly easy to spot… for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8695–8704.
  56. DIRE for Diffusion-Generated Image Detection. arXiv preprint arXiv:2303.09295 (2023).
  57. Wukong. 2022. https://xihe.mindspore.cn/modelzoo/wukong.
  58. On the Adversarial Robustness of Camera-based 3D Object Detection. Transactions on Machine Learning Research (2023).
  59. Stochastic variance reduced ensemble adversarial attack for boosting the adversarial transferability. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14983–14992.
  60. Detecting and simulating artifacts in gan fake images. In 2019 IEEE international workshop on information forensics and security (WIFS). IEEE, 1–6.
  61. Yichi Zhang and Xiaogang Xu. 2023. Diffusion noise feature: Accurate and fast generated image detection. arXiv preprint arXiv:2312.02625 (2023).
  62. Rich and poor texture contrast: A simple yet effective approach for ai-generated image detection. arXiv preprint arXiv:2311.12397 (2023).
  63. GenDet: Towards Good Generalizations for AI-Generated Image Detection. arXiv preprint arXiv:2312.08880 (2023).
  64. Genimage: A million-scale benchmark for detecting ai-generated image. Advances in Neural Information Processing Systems 36 (2024).
  65. Understanding the Robustness of 3D Object Detection With Bird’s-Eye-View Representations in Autonomous Driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21600–21610.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yunfeng Diao (16 papers)
  2. Naixin Zhai (1 paper)
  3. Changtao Miao (12 papers)
  4. Xun Yang (76 papers)
  5. Meng Wang (1063 papers)
Citations (1)
View on Semantic Scholar