Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Siamese-based Verification System for Open-set Architecture Attribution of Synthetic Images (2307.09822v2)

Published 19 Jul 2023 in cs.CV

Abstract: Despite the wide variety of methods developed for synthetic image attribution, most of them can only attribute images generated by models or architectures included in the training set and do not work with unknown architectures, hindering their applicability in real-world scenarios. In this paper, we propose a verification framework that relies on a Siamese Network to address the problem of open-set attribution of synthetic images to the architecture that generated them. We consider two different settings. In the first setting, the system determines whether two images have been produced by the same generative architecture or not. In the second setting, the system verifies a claim about the architecture used to generate a synthetic image, utilizing one or multiple reference images generated by the claimed architecture. The main strength of the proposed system is its ability to operate in both closed and open-set scenarios so that the input images, either the query and reference images, can belong to the architectures considered during training or not. Experimental evaluations encompassing various generative architectures such as GANs, diffusion models, and transformers, focusing on synthetic face image generation, confirm the excellent performance of our method in both closed and open-set settings, as well as its strong generalization capabilities.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Do gans leave artificial fingerprints?, in: 2019 IEEE conference on multimedia information processing and retrieval (MIPR), IEEE. pp. 506–511.
  2. Responsible disclosure of generative models using scalable fingerprinting, in: International Conference on Learning Representations.
  3. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data, in: Proceedings of the IEEE/CVF International conference on computer vision, pp. 14448–14457.
  4. Deepfake network architecture attribution, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4662–4670.
  5. Repmix: Representation mixing for robust attribution of synthesized images, in: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIV, Springer. pp. 146–163.
  6. Towards discovery and attribution of open-world gan generated images, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14094–14103.
  7. Progressive open space expansion for open-set model attribution, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15856–15865.
  8. Open set classification of gan-based image manipulations via a vit-based hybrid architecture, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 953–962.
  9. Learned forensic source similarity for unknown camera models, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 2012–2016.
  10. Noiseprint: A cnn-based camera model fingerprint. IEEE Transactions on Information Forensics and Security 15, 144–159.
  11. An eyes-based siamese neural network for the detection of gan-generated face images. Frontiers in Signal Processing 2, 918725.
  12. Fighting fake news: Image splice detection via learned self-consistency, in: Proceedings of the European Conference on Computer Vision (ECCV).
  13. BEGAN: boundary equilibrium generative adversarial networks. CoRR abs/1703.10717.
  14. Large scale GAN training for high fidelity natural image synthesis, in: ICLR, OpenReview.net.
  15. Progressive growing of gans for improved quality, stability, and variation, in: ICLR, OpenReview.net.
  16. Analyzing and improving the image quality of stylegan, in: CVPR, Computer Vision Foundation / IEEE. pp. 8107–8116.
  17. Training generative adversarial networks with limited data. CoRR abs/2006.06676. 2006.06676.
  18. Alias-free generative adversarial networks, in: NeurIPS, pp. 852–863.
  19. Taming transformers for high-resolution image synthesis, in: CVPR, Computer Vision Foundation / IEEE. pp. 12873–12883.
  20. Denoising diffusion probabilistic models, in: NeurIPS.
  21. Score-based generative modeling in latent space, in: NeurIPS, pp. 11287–11302.
  22. High-resolution image synthesis with latent diffusion models, in: CVPR, IEEE. pp. 10674–10685.
  23. Attributing fake images to gans: Learning and analyzing gan fingerprints, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 7556–7566.
  24. Leveraging frequency analysis for deep fake image recognition, in: International conference on machine learning, PMLR. pp. 3247–3258.
  25. Attributing and detecting fake images generated by known gans, in: 2020 IEEE Security and Privacy Workshops (SPW), IEEE. pp. 8–14.
  26. Learning to disentangle gan fingerprint for fake image attribution. arXiv preprint arXiv:2106.08749 .
  27. Open set source camera attribution and device linking. Pattern Recognition Letters 39, 92–101. Advances in Pattern Recognition and Computer Vision.
  28. Identifying common source digital camera from image pairs, in: 2007 IEEE International Conference on Image Processing, IEEE. pp. VI–125.
  29. Towards generalizable detection of face forgery via self-guided model-agnostic learning. Pattern Recognition Letters 160, 98–104.
  30. On improving cross-dataset generalization of deepfake detectors, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 91–99.
  31. Efficientnet: Rethinking model scaling for convolutional neural networks, in: International conference on machine learning, PMLR. pp. 6105–6114.
  32. Deep residual learning for image recognition, in: CVPR, IEEE Computer Society. pp. 770–778.
  33. Swin transformer: Hierarchical vision transformer using shifted windows, in: ICCV, IEEE. pp. 9992–10002.
  34. Stargan v2: Diverse image synthesis for multiple domains, in: CVPR, Computer Vision Foundation / IEEE. pp. 8185–8194.
  35. A style-based generator architecture for generative adversarial networks, in: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, Computer Vision Foundation / IEEE. pp. 4401–4410.
  36. Deep learning face attributes in the wild, in: Proceedings of International Conference on Computer Vision (ICCV).
  37. Dimensionality reduction by learning an invariant mapping, in: CVPR (2), IEEE Computer Society. pp. 1735–1742.
  38. A baseline for detecting misclassified and out-of-distribution examples in neural networks, in: International Conference on Learning Representations.
  39. Class-specific semantic reconstruction for open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence .
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Lydia Abady (3 papers)
  2. Jun Wang (991 papers)
  3. Benedetta Tondi (43 papers)
  4. Mauro Barni (56 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now