Papers
Topics
Authors
Recent
Search
2000 character limit reached

Artifact Feature Purification for Cross-domain Detection of AI-generated Images

Published 17 Mar 2024 in cs.CV | (2403.11172v1)

Abstract: In the era of AIGC, the fast development of visual content generation technologies, such as diffusion models, bring potential security risks to our society. Existing generated image detection methods suffer from performance drop when faced with out-of-domain generators and image scenes. To relieve this problem, we propose Artifact Purification Network (APN) to facilitate the artifact extraction from generated images through the explicit and implicit purification processes. For the explicit one, a suspicious frequency-band proposal method and a spatial feature decomposition method are proposed to extract artifact-related features. For the implicit one, a training strategy based on mutual information estimation is proposed to further purify the artifact-related features. Experiments show that for cross-generator detection, the average accuracy of APN is 5.6% ~ 16.4% higher than the previous 10 methods on GenImage dataset and 1.7% ~ 50.1% on DiffusionForensics dataset. For cross-scene detection, APN maintains its high performance. Via visualization analysis, we find that the proposed method extracts flexible forgery patterns and condenses the forgery information diluted in irrelevant features. We also find that the artifact features APN focuses on across generators and scenes are global and diverse. The code will be available on GitHub.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, volume 31, 2018.
  2. Nice: Non-linear independent components estimation. In International Conference on Learning Representations, 2015.
  3. Density estimation using real NVP. In International Conference on Learning Representations, 2017.
  4. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019.
  5. A style-based generator architecture for generative adversarial networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4217 – 4228, 2021.
  6. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8789–8797, 2018.
  7. AttGAN: Facial attribute editing by only changing what you want. IEEE Transactions on Image Processing, 28(11):5464–5478, 2019.
  8. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
  9. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, volume 34, pages 8780–8794, 2021.
  10. Denoising diffusion implicit models. In International Conference on Learning Representations, May 2020.
  11. Improved denoising diffusion probabilistic models. In Proceedings of International Conference on Machine Learning, pages 8162–8171, 2021.
  12. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  13. GenImage: A million-scale benchmark for detecting ai-generated image. In Advances in Neural Information Processing Systems, volume 36, pages 77771–77782, 2023a.
  14. LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
  15. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of International Conference on Machine Learning, pages 16784–16804, 2022.
  16. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
  17. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10696–10706, 2022a.
  18. Rewon Child. Very deep VAEs generalize autoregressive models and can outperform them on images. In International Conference on Learning Representations, May 2020.
  19. Auto-encoding variational {{\{{Bayes}}\}}. In International Conference on Learning Representations, Apr. 2014.
  20. NVAE: A deep hierarchical variational autoencoder. In Advances in Neural Information Processing Systems, volume 33, pages 19667–19679, Dec. 2020.
  21. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851, Dec. 2020.
  22. Generating diverse high-fidelity images with VQ-VAE-2. In Advances in Neural Information Processing Systems, volume 32, Jun. 2019.
  23. Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Security Symposium, pages 5253–5270, Jan. 2023.
  24. Data forensics in diffusion models: A systematic analysis of membership privacy. arXiv preprint arXiv:2302.07801, 2023b.
  25. The creation and detection of deepfakes: A survey. ACM Computing Surveys, 54(1):1–41, 2021.
  26. De-fake: Detection and attribution of fake images generated by text-to-image diffusion models. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 3418–3432, 2022.
  27. CNN-generated images are surprisingly easy to spot… for now. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8695–8704, Jun. 2020.
  28. Fusing global and local features for generalized ai-synthesized image detection. In Proceedings of the IEEE International Conference on Image Processing, pages 3465–3469, Oct. 2022.
  29. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In Proceedings of European Conference on Computer Vision, pages 86–103, Aug. 2020.
  30. ADD: Frequency attention and multi-view based knowledge distillation to detect low-quality compressed deepfake images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 122–130, Feb. 2022.
  31. Exploiting fine-grained face forgery clues via progressive enhancement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 735–743, Feb. 2022b.
  32. Fingerprintnet: Synthesized fingerprints for generated image detection. In Proceedings of European Conference on Computer Vision, pages 76–94, Oct. 2022.
  33. Detecting images generated by deep diffusion models using their local intrinsic dimensionality. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 448–459, Oct. 2023.
  34. DIRE for diffusion-generated image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22445–22455, Oct. 2023a.
  35. Parents and children: Distinguishing multimodal deepfakes from natural images. arXiv preprint arXiv:2304.00500, 2023.
  36. Generalizable synthetic image detection via language-guided contrastive learning. arXiv preprint arXiv:2305.13800, 2023.
  37. On the detection of synthetic images generated by diffusion models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5, Jun. 2023a.
  38. Towards the detection of diffusion model deepfakes. In International Conference on Learning Representations, May 2023.
  39. What makes fake images detectable? Understanding properties that generalize. In Proceedings of European Conference on Computer Vision, pages 103–120, Aug. 2020.
  40. Cnn detection of GAN-generated face images based on cross-band co-occurrences analysis. In IEEE International Workshop on Information Forensics and Security, pages 1–6, Dec. 2020.
  41. Representative forgery mining for fake face detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14923–14932, Jun. 2021.
  42. An information theoretic approach for attention-driven face forgery detection. In Proceedings of European Conference on Computer Vision, pages 111–127, Oct. 2022.
  43. Detecting GAN-generated images by orthogonal training of multiple CNNs. In Proceedings of the IEEE International Conference on Image Processing, pages 3091–3095, Oct. 2022.
  44. End-to-end reconstruction-classification learning for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4113–4122, Jun. 2022.
  45. Improving GAN-generated image detection generalization using unsupervised domain adaptation. In Proceedings of the IEEE International Conference on Multimedia and Expo, pages 1–6, Jul. 2022.
  46. Learning patch-channel correspondence for interpretable face forgery detection. IEEE Transactions on Image Processing, 32:1668–1680, 2023.
  47. Artifacts-disentangled adversarial learning for deepfake detection. IEEE Transactions on Circuits and Systems for Video Technology, 33(4):1658–1670, 2022.
  48. AI-generated image detection using a cross-attention enhanced dual-stream network. arXiv preprint arXiv:2306.07005, 2023.
  49. Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security, 7(3):868–882, 2012.
  50. Generalizing face forgery detection with high-frequency features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16317–16326, Jun. 2021.
  51. Exposing the fake: Effective diffusion-generated images detection. In Proceedings of International Conference on Machine Learning Workshops, Jul. 2023.
  52. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770–778, Jun. 2016.
  53. Training data-efficient image transformers & distillation through attention. In Proceedings of International Conference on Machine Learning, pages 10347–10357, Jul. 2021.
  54. Swin Transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, Mar. 2021.
  55. Learning second order local anomaly for general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20270–20280, Jun. 2022.
  56. Intriguing properties of synthetic images: From generative adversarial networks to diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 973–982, Jun. 2023b.
  57. Exploring frequency adversarial attacks for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4103–4112, Jun. 2022.
  58. Leveraging frequency analysis for deep fake image recognition. In Proceedings of International Conference on Machine Learning, pages 3247–3258, Jul. 2020.
  59. Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6458–6467, Jun. 2021.
  60. Local relation learning for face forgery detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1081–1088, Feb. 2021.
  61. CLUB: A contrastive log-ratio upper bound of mutual information. In Proceedings of International Conference on Machine Learning, pages 1779–1788, Jul. 2020.
  62. Detecting and simulating artifacts in GAN fake images. In IEEE International Workshop on Information Forensics and Security, pages 1–6, Dec. 2019.
  63. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8060–8069, Jun. 2020.
  64. Frequency-aware attentional feature fusion for deepfake detection. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 1–5, Jun. 2023.
  65. Midjourney. Midjourney. https://www.midjourney.com/home/, 2022.
  66. Wukong. Wukong. https://xihe.mindspore.cn/modelzoo/wukong, 2022.
  67. LayerCam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing, 30:5875–5888, 2021.
  68. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, Apr. 2022a.
  69. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, Dec. 2017.
  70. Forgerynir: Deep face forgery and detection in near-infrared scenario. IEEE Transactions on Information Forensics and Security, 17:500–515, 2022.
  71. F2Trans: High-frequency fine-grained transformer for face forgery detection. IEEE Transactions on Information Forensics and Security, 18:1039–1051, 2023.
  72. Dynamic difference learning with spatio-temporal correlation for deepfake video detection. IEEE Transactions on Information Forensics and Security, 18, 2023.
  73. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24480–24489, 2023.
  74. Learning on gradients: Generalized artifacts representation for gan-generated images detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12105–12114, 2023.
  75. Detecting generated images by real images. In European Conference on Computer Vision, pages 95–110, 2022b.
  76. Learning transferable visual models from natural language supervision. In Proceedings of International Conference on Machine Learning, pages 8748–8763, 2021.
  77. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  78. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022.
  79. Projected gans converge faster. Advances in Neural Information Processing Systems, 34:17480–17492, 2021.
  80. Diffusion-gan: Training gans with diffusion. In International Conference on Learning Representations, 2023b.
  81. A data augmentation framework by mining structured features for fake face image detection. Computer Vision and Image Understanding, 226:103587, 2023.
  82. Glitch in the matrix: A large scale benchmark for content driven audio–visual forgery detection and localization. Computer Vision and Image Understanding, 236:103818, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up