Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Individualized Deepfake Detection Exploiting Traces Due to Double Neural-Network Operations (2312.08034v1)

Published 13 Dec 2023 in eess.IV, cs.CR, cs.CV, and cs.LG

Abstract: In today's digital landscape, journalists urgently require tools to verify the authenticity of facial images and videos depicting specific public figures before incorporating them into news stories. Existing deepfake detectors are not optimized for this detection task when an image is associated with a specific and identifiable individual. This study focuses on the deepfake detection of facial images of individual public figures. We propose to condition the proposed detector on the identity of the identified individual given the advantages revealed by our theory-driven simulations. While most detectors in the literature rely on perceptible or imperceptible artifacts present in deepfake facial images, we demonstrate that the detection performance can be improved by exploiting the idempotency property of neural networks. In our approach, the training process involves double neural-network operations where we pass an authentic image through a deepfake simulating network twice. Experimental results show that the proposed method improves the area under the curve (AUC) from 0.92 to 0.94 and reduces its standard deviation by 17\%. For evaluating the detection performance of individual public figures, a facial image dataset with individuals' names is required, a criterion not met by the current deepfake datasets. To address this, we curated a dataset comprising 32k images featuring 45 public figures, which we intend to release to the public after the paper is published.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Deepfakes. https://github.com/deepfakes/faceswap Accessed on: June, 2023.
  2. Deepfake. https://github.com/dfaker/df Accessed on: June, 2023.
  3. Deep fake detection dataset. https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html Accessed on: June, 2023.
  4. Deepfacelab. https://github.com/iperov/DeepFaceLab/ Accessed on: June, 2023.
  5. FaceSwap. https://faceswap.dev Accessed on: June, 2023.
  6. FaceSwap-GAN. https://github.com/shaoanlu/faceswap-GAN Accessed on: June, 2023.
  7. Protecting world leaders against deep fakes. In IEEE/CVF Conf. Comput. Vision Pattern Recog. Workshops, Long Beach, CA, 2019.
  8. Towards open-set identity preserving face synthesis. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 6713–6722, 2018.
  9. Video codec identification. In IEEE Int. Conf. Acoust., Speech, Signal Process., pages 2257–2260, 2012.
  10. Face swapping: Automatically replacing faces in photographs. In ACM SIGGRAPH, pages 1–8. 2008.
  11. Video face manipulation detection through ensemble of CNNs. In IEEE Int. Conf. Learn. Pattern, 2020.
  12. Signature verification using a “siamese” time delay neural network. Adv. Neural Infor. Process. Syst., 6, 1993.
  13. Stanley L Brue. Retrospectives: The law of diminishing returns. Journal of Economic Perspectives, 7(3):185–192, 1993.
  14. Pre-training strategies and datasets for facial representation learning. In European Conference on Computer Vision, pages 107–125. Springer, 2022.
  15. Forensics of high-quality JPEG images with color subsampling. In IEEE Int. Workshop Informat. Forensics Security, pages 1–6, 2015.
  16. Cross-age reference coding for age-invariant face recognition and retrieval. In European Conf. Comput. Vision, 2014.
  17. Self-supervised learning of adversarial example: Towards good generalizations for deepfake detection. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 18710–18719, 2022.
  18. Big self-supervised models are strong semi-supervised learners. Adv. Neural Infor. Process. Syst., 33:22243–22255, 2020.
  19. 3d-model-based face replacement in video. In SIGGRAPH. 2009.
  20. François Chollet. Xception: Deep learning with depthwise separable convolutions. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 1251–1258, 2017.
  21. Id-reveal: Identity-aware deepfake video detection. In IEEE/CVF Int. Conf. Comput. Vision, pages 15108–15117, 2021.
  22. On the detection of digital face manipulation. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 5781–5790, 2020.
  23. Source camera identification using auto-white balance approximation. In IEEE/CVF Int. Conf. Comput. Vision, pages 57–64, 2011.
  24. The deepfake detection challenge dataset. arXiv preprint arXiv:2006.07397, 2020.
  25. Protecting celebrities from deepfake with identity consistency transformer. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 9468–9478, 2022.
  26. Watch your up-convolution: CNN based generative deep neural networks are failing to reproduce spectral distributions. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 7890–7899, 2020.
  27. Leveraging frequency analysis for deep fake image recognition. In Int. Conf. Mach. Learn., 2020.
  28. Deepfake detection by analyzing convolutional traces. In IEEE/CVF Conf. Comput. Vision Pattern Recog. Workshops, pages 666–667, 2020.
  29. Deepfake video detection using recurrent neural networks. In IEEE Int. Conf. Advanced Video Signal Based Surveillance, Auckland, New Zealand, 2018.
  30. Controllable guide-space for generalizable face forgery detection. In IEEE/CVF Int. Conf. Comput. Vision, pages 20818–20827, 2023.
  31. Dimensionality reduction by learning an invariant mapping. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 1735–1742, New York, NY, 2006.
  32. Detecting double JPEG compression with the same quantization matrix. IEEE Trans. Inf. Forensics Security, 5(4):848–856, 2010.
  33. DeeperForensics-1.0: A large-scale dataset for real-world face forgery detection. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 2889–2898, 2020.
  34. A style-based generator architecture for generative adversarial networks. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 4401–4410, 2019.
  35. Block convergence in repeated transform coding: JPEG-100 forensics, carbon dating, and tamper detection. In IEEE Int. Conf. Acoust., Speech, Signal Process., pages 3028–3032, 2013.
  36. Detection of deep network generated images using disparities in color components. arXiv preprint arXiv:1808.07276, 2018a.
  37. Faceshifter: Towards high fidelity and occlusion aware face swapping. arXiv preprint arXiv:1912.13457, 2019.
  38. In Ictu Oculi: Exposing AI created fake videos by detecting eye blinking. In IEEE Int. Workshop Informat. Forensics Security, Hong Kong, 2018b.
  39. Celeb-DF: A large-scale challenging dataset for deepfake forensics. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 3207–3216, 2020.
  40. Demosaicing strategy identification via eigenalgorithms. In IEEE Int. Conf. Acoust., Speech, Signal Process., pages 2659–2663, 2014.
  41. The creation and detection of deepfakes: A survey. ACM Comput. Surveys, 54(1):1–41, 2021.
  42. FsNet: An identity-aware generative model for image-based face swapping. In Asian Conf. Comput. Vision, Perth, Australia, Dec. 2–6, 2018.
  43. RSGAN: Face swapping and editing using face and hair representation in latent spaces. arXiv preprint arXiv:1804.03447, 2018.
  44. How useful is self-supervised pretraining for visual tasks? In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 7345–7354, 2020.
  45. FSGAN: Subject agnostic face swapping and reenactment. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 7184–7193, 2019.
  46. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  47. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 10684–10695, 2022.
  48. Faceforensics++: Learning to detect manipulated facial images. In IEEE/CVF Int. Conf. Comput. Vision, pages 1–11, 2019.
  49. Photorealistic text-to-image diffusion models with deep language understanding. Adv. Neural Infor. Process. Syst., 35:36479–36494, 2022.
  50. WJ Spillman. Application of the law of diminishing returns to some fertilizer and feed data. Journal of Farm Economics, 5(1):36–52, 1923.
  51. CNN-generated images are surprisingly easy to spot… for now. In IEEE/CVF Conf. Comput. Vision Pattern Recog., 2020.
  52. UCF: Uncovering common features for generalizable deepfake detection. In IEEE/CVF Int. Conf. Comput. Vision, pages 22412–22423, 2023.
  53. An effective method for detecting double JPEG compression with the same quantization matrix. IEEE Trans. Inf. Forensics Security, 9(11):1933–1942, 2014.
  54. Exposing deep fakes using inconsistent head poses. In IEEE Int. Conf. Acoust., Speech, Signal Process., pages 8261–8265, Brighton, UK, 2019.
  55. Few-shot adversarial learning of realistic neural talking head models. In IEEE/CVF Int. Conf. Comput. Vision, pages 9459–9468, 2019.
  56. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Letters, 23(10):1499–1503, 2016.
  57. Multi-attentional deepfake detection. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 2185–2194, 2021.
  58. General facial representation learning in a visual-linguistic manner. In IEEE/CVF Conf. Comput. Vision Pattern Recog., pages 18697–18709, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Mushfiqur Rahman (6 papers)
  2. Runze Liu (22 papers)
  3. Chau-Wai Wong (33 papers)
  4. Huaiyu Dai (102 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now