Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 148 tok/s
Gemini 2.5 Pro 47 tok/s Pro
GPT-5 Medium 26 tok/s Pro
GPT-5 High 28 tok/s Pro
GPT-4o 94 tok/s Pro
Kimi K2 214 tok/s Pro
GPT OSS 120B 429 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection (2403.06803v1)

Published 11 Mar 2024 in cs.CV

Abstract: Recently, the proliferation of increasingly realistic synthetic images generated by various generative adversarial networks has increased the risk of misuse. Consequently, there is a pressing need to develop a generalizable detector for accurately recognizing fake images. The conventional methods rely on generating diverse training sources or large pretrained models. In this work, we show that, on the contrary, the small and training-free filter is sufficient to capture more general artifact representations. Due to its unbias towards both the training and test sources, we define it as Data-Independent Operator (DIO) to achieve appealing improvements on unseen sources. In our framework, handcrafted filters and the randomly-initialized convolutional layer can be used as the training-free artifact representations extractor with excellent results. With the data-independent operator of a popular classifier, such as Resnet50, one could already reach a new state-of-the-art without bells and whistles. We evaluate the effectiveness of the DIO on 33 generation models, even DALLE and Midjourney. Our detector achieves a remarkable improvement of $13.3\%$, establishing a new state-of-the-art performance. The DIO and its extension can serve as strong baselines for future methods. The code is available at \url{https://github.com/chuangchuangtan/Data-Independent-Operator}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. I. J. Goodfellow et al., “Generative adversarial nets,” in NIPS, 2014.
  2. T. Karras et al., “Progressive growing of gans for improved quality, stability, and variation,” in International Conference on Learning Representations, 2018.
  3. ——, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the CVPR, 2019, pp. 4401–4410.
  4. J. Ho et al., “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
  5. R. Rombach et al., “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695.
  6. S.-Y. Wang et al., “Cnn-generated images are surprisingly easy to spot… for now,” in Proceedings of the CVPR, 2020, pp. 8695–8704.
  7. J. Frank et al., “Leveraging frequency analysis for deep fake image recognition,” in International conference on machine learning.   PMLR, 2020, pp. 3247–3258.
  8. ——, “Leveraging frequency analysis for deep fake image recognition,” in International conference on machine learning.   PMLR, 2020, pp. 3247–3258.
  9. Y. Jeong et al., “Frepgan: robust deepfake detection using frequency-level perturbations,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, 2022, pp. 1060–1068.
  10. ——, “Fingerprintnet: Synthesized fingerprints for generated image detection,” in European Conference on Computer Vision.   Springer, 2022, pp. 76–94.
  11. L. Chen et al., “Ost: Improving generalization of deepfake detection via one-shot test-time training,” in Advances in Neural Information Processing Systems, 2022.
  12. U. Ojha et al., “Towards universal fake image detectors that generalize across generative models,” in Proceedings of the CVPR, 2023, pp. 24 480–24 489.
  13. C. Tan et al., “Learning on gradients: Generalized artifacts representation for gan-generated images detection,” in Proceedings of the CVPR (CVPR), June 2023, pp. 12 105–12 114.
  14. D. Ulyanov et al., “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9446–9454.
  15. A. Rossler et al., “Faceforensics++: Learning to detect manipulated facial images,” in Proceedings of the ICCV, 2019, pp. 1–11.
  16. F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258.
  17. Y. Li et al., “In ictu oculi: Exposing ai created fake videos by detecting eye blinking,” in 2018 IEEE International workshop on information forensics and security (WIFS).   IEEE, 2018, pp. 1–7.
  18. A. Haliassos et al., “Lips don’t lie: A generalisable and robust approach to face forgery detection,” in Proceedings of the CVPR, 2021, pp. 5039–5049.
  19. S. Ye et al., “Detecting digital image forgeries by measuring inconsistencies of blocking artifact,” in 2007 IEEE ICME.   Ieee, 2007, pp. 12–15.
  20. T. Zhao et al., “Learning self-consistency for deepfake detection,” in Proceedings of the ICCV, 2021, pp. 15 023–15 033.
  21. X. Dong et al., “Protecting celebrities from deepfake with identity consistency transformer,” in Proceedings of the CVPR, 2022, pp. 9468–9478.
  22. L. Chai et al., “What makes fake images detectable? understanding properties that generalize,” in European conference on computer vision.   Springer, 2020, pp. 103–120.
  23. C. Wang et al., “Representative forgery mining for fake face detection,” in Proceedings of the CVPR, 2021, pp. 14 923–14 932.
  24. L. Chen et al., “Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection,” in Proceedings of the CVPR, 2022, pp. 18 710–18 719.
  25. J. Cao et al., “End-to-end reconstruction-classification learning for face forgery detection,” in Proceedings of the CVPR, 2022, pp. 4113–4122.
  26. Y. He et al., “Beyond the spectrum: Detecting deepfakes via re-synthesis,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21.   International Joint Conferences on Artificial Intelligence Organization, 2021, pp. 2534–2541.
  27. K. Shiohara et al., “Detecting deepfakes with self-blended images,” in Proceedings of the CVPR, 2022, pp. 18 720–18 729.
  28. Y. Ju et al., “Fusing global and local features for generalized ai-synthesized image detection,” in 2022 IEEE International Conference on Image Processing (ICIP).   IEEE, 2022, pp. 3465–3469.
  29. R. Durall et al., “Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions,” in Proceedings of the CVPR, 2020, pp. 7890–7899.
  30. I. Masi et al., “Two-branch recurrent network for isolating deepfakes in videos,” in European conference on computer vision.   Springer, 2020, pp. 667–684.
  31. Y. Qian et al., “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in European conference on computer vision.   Springer, 2020, pp. 86–103.
  32. Y. Luo et al., “Generalizing face forgery detection with high-frequency features,” in Proceedings of the CVPR, 2021, pp. 16 317–16 326.
  33. S. Woo et al., “Add: Frequency attention and multi-view based knowledge distillation to detect low-quality compressed deepfake images,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, 2022, pp. 122–130.
  34. Y. Jeong et al., “Bihpf: Bilateral high-pass filters for robust deepfake detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 48–57.
  35. T. Karras et al., “Analyzing and improving the image quality of stylegan,” in Proceedings of the CVPR, 2020, pp. 8110–8119.
  36. A. Brock et al., “Large scale gan training for high fidelity natural image synthesis,” in International Conference on Learning Representations, 2018.
  37. J.-Y. Zhu et al., “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232.
  38. Y. Choi et al., “Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8789–8797.
  39. T. Park et al., “Semantic image synthesis with spatially-adaptive normalization,” in Proceedings of the CVPR, 2019, pp. 2337–2346.
  40. F. Yu et al., “Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop,” arXiv preprint arXiv:1506.03365, 2015.
  41. O. Russakovsky et al., “Imagenet large scale visual recognition challenge,” International journal of computer vision, vol. 115, no. 3, pp. 211–252, 2015.
  42. Z. Liu et al., “Deep learning face attributes in the wild,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730–3738.
  43. T.-Y. Lin et al., “Microsoft coco: Common objects in context,” in European conference on computer vision.   Springer, 2014, pp. 740–755.
  44. H. L. Chuangchuang Tan, Renshuai Tao, “Gangen-detection: A dataset generated by gans for generalizable deepfake detection,” https://github.com/chuangchuangtan/GANGen-Detection, 2024.
  45. Z. He et al., “Attgan: Facial attribute editing by only changing what you want,” IEEE Transactions on Image Processing, vol. 28, no. 11, pp. 5464–5478, 2019.
  46. D. Berthelot et al., “Began: Boundary equilibrium generative adversarial networks,” arXiv preprint arXiv:1703.10717, 2017.
  47. M. G. Bellemare et al., “The cramer distance as a solution to biased wasserstein gradients,” arXiv preprint arXiv:1705.10743, 2017.
  48. K. S. Lee et al., “Infomax-gan: Improved adversarial image generation via information maximization and contrastive learning,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 3942–3952.
  49. C.-L. Li et al., “Mmd gan: Towards deeper understanding of moment matching network,” Advances in neural information processing systems, vol. 30, 2017.
  50. W. Nie et al., “Relgan: Relational generative adversarial networks for text generation,” in International conference on learning representations, 2019.
  51. M. Lučić et al., “High-fidelity image generation with fewer labels,” in International conference on machine learning.   PMLR, 2019, pp. 4183–4192.
  52. T. Miyato et al., “Spectral normalization for generative adversarial networks,” arXiv preprint arXiv:1802.05957, 2018.
  53. M. Liu et al., “Stgan: A unified selective transfer network for arbitrary image attribute editing,” in Proceedings of the CVPR, 2019, pp. 3673–3682.
  54. Z. Wang et al., “Dire for diffusion-generated image detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 22 445–22 455.
  55. P. Dhariwal et al., “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems, vol. 34, pp. 8780–8794, 2021.
  56. A. Q. Nichol et al., “Improved denoising diffusion probabilistic models,” in International Conference on Machine Learning.   PMLR, 2021, pp. 8162–8171.
  57. L. Liu et al., “Pseudo numerical methods for diffusion models on manifolds,” in ICLR, 2022.
  58. S. Gu et al., “Vector quantized diffusion model for text-to-image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 696–10 706.
  59. A. Nichol et al., “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741, 2021.
  60. A. Ramesh et al., “Zero-shot text-to-image generation,” in ICML.   PMLR, 2021, pp. 8821–8831.
  61. C. Schuhmann et al., “Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,” in NeurIPS Workshop Datacentric AI, no. FZJ-2022-00923.   Jülich Supercomputing Center, 2021.
  62. N. Zhong, Y. Xu, Z. Qian, and X. Zhang, “Rich and poor texture contrast: A simple yet effective approach for ai-generated image detection,” arXiv preprint arXiv:2311.12397, 2023.
  63. “Midjourney,” https://www.midjourney.com/home/, 2023.
  64. “wukong,” https://xihe.mindspore.cn/modelzoo/wukong, 2023.
  65. M. Zhu, H. Chen, Q. Yan, X. Huang, G. Lin, W. Li, Z. Tu, H. Hu, J. Hu, and Y. Wang, “Genimage: A million-scale benchmark for detecting ai-generated image,” 2023.
  66. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022.
  67. K. He et al., “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  68. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in ICLR (Poster), 2015.
  69. K. Simonyan et al., “Very deep convolutional networks for large-scale image recognition,” International Conference on Learning Representations, 2015.
  70. C. Szegedy et al., “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818–2826.
  71. A. Radford et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning.   PMLR, 2021, pp. 8748–8763.
  72. L.-C. Chen et al., “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587, 2017.
  73. S. Mandelli, N. Bonettini, P. Bestagini, and S. Tubaro, “Detecting gan-generated images by orthogonal training of multiple cnns,” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 3091–3095.
  74. Z. Liu et al., “Global texture enhancement for fake face detection in the wild,” in Proceedings of the CVPR, 2020, pp. 8060–8069.
  75. B. Liu, F. Yang, X. Bi, B. Xiao, W. Li, and X. Gao, “Detecting generated images by real images,” in European Conference on Computer Vision.   Springer, 2022, pp. 95–110.
  76. Z. Wang, J. Bao, W. Zhou, W. Wang, H. Hu, H. Chen, and H. Li, “Dire for diffusion-generated image detection,” arXiv preprint arXiv:2303.09295, 2023.
  77. L. Van Der Maaten, “Accelerating t-sne using tree-based algorithms,” The journal of machine learning research, vol. 15, no. 1, pp. 3221–3245, 2014.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up