Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hide and Seek: How Does Watermarking Impact Face Recognition? (2404.18890v1)

Published 29 Apr 2024 in cs.CV

Abstract: The recent progress in generative models has revolutionized the synthesis of highly realistic images, including face images. This technological development has undoubtedly helped face recognition, such as training data augmentation for higher recognition accuracy and data privacy. However, it has also introduced novel challenges concerning the responsible use and proper attribution of computer generated images. We investigate the impact of digital watermarking, a technique for embedding ownership signatures into images, on the effectiveness of face recognition models. We propose a comprehensive pipeline that integrates face image generation, watermarking, and face recognition to systematically examine this question. The proposed watermarking scheme, based on an encoder-decoder architecture, successfully embeds and recovers signatures from both real and synthetic face images while preserving their visual fidelity. Through extensive experiments, we unveil that while watermarking enables robust image attribution, it results in a slight decline in face recognition accuracy, particularly evident for face images with challenging poses and expressions. Additionally, we find that directly training face recognition models on watermarked images offers only a limited alleviation of this performance decline. Our findings underscore the intricate trade off between watermarking and face recognition accuracy. This work represents a pivotal step towards the responsible utilization of generative models in face recognition and serves to initiate discussions regarding the broader implications of watermarking in biometrics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022.
  2. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  3. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
  4. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531, 2023.
  5. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now. arXiv preprint arXiv:2310.11868, 2023.
  6. Protecting world leaders against deep fakes. In CVPR workshops, volume 1, page 38, 2019.
  7. Deep fakes: A looming challenge for privacy, democracy, and national security. Calif. L. Rev., 107:1753, 2019.
  8. Shake to leak: Fine-tuning diffusion models can amplify the generative privacy risk. arXiv preprint arXiv:2403.09450, 2024.
  9. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pages 5253–5270, 2023.
  10. Understanding and mitigating copying in diffusion models. Advances in Neural Information Processing Systems, 36:47783–47803, 2023.
  11. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6048–6058, 2023.
  12. Unlearncanvas: A stylized image dataset to benchmark machine unlearning for diffusion models. arXiv preprint arXiv:2402.11846, 2024.
  13. Stable bias: Evaluating societal representations in diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
  14. Rs-corrector: Correcting the racial stereotypes in latent diffusion models. arXiv preprint arXiv:2312.04810, 2023.
  15. On the trustworthiness landscape of state-of-the-art generative models: A comprehensive survey. arXiv preprint arXiv:2307.16680, 2023.
  16. Hiding biometric data. IEEE transactions on pattern analysis and machine intelligence, 25(11):1494–1498, 2003.
  17. Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pages 657–672, 2018.
  18. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22466–22477, 2023.
  19. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030, 2023.
  20. Dcface: Synthetic face generation with dual condition diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12715–12725, 2023.
  21. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
  22. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10492–10502, 2021.
  23. Warpgan: Automatic caricature generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10762–10771, 2019.
  24. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023.
  25. Photomaker: Customizing realistic human photos via stacked id embedding. arXiv preprint arXiv:2312.04461, 2023.
  26. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024.
  27. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
  28. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019.
  29. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8789–8797, 2018.
  30. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
  31. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  32. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pages 67–74. IEEE, 2018.
  33. Training generative adversarial networks with limited data. Advances in neural information processing systems, 33:12104–12114, 2020.
  34. Semi-adversarial networks: Convolutional autoencoders for imparting privacy to face images. In 2018 International Conference on Biometrics (ICB), pages 82–89. IEEE, 2018.
  35. Mika Westerlund. The emergence of deepfake technology: A review. Technology innovation management review, 9(11), 2019.
  36. Digital watermarking and steganography. Morgan kaufmann, 2007.
  37. Informed embedding: exploiting image and detector information during watermark insertion. In Proceedings 2000 International Conference on Image Processing (Cat. No. 00CH37101), volume 3, pages 1–4. IEEE, 2000.
  38. Attacks on digital watermarks: classification, estimation based attacks, and benchmarks. IEEE communications Magazine, 39(8):118–126, 2001.
  39. Jessica Fridrich. Visual hash for oblivious watermarking. In Security and Watermarking of Multimedia Contents II, volume 3971, pages 286–294. SPIE, 2000.
  40. Dct-based watermark recovering without resorting to the uncorrupted original image. In Proceedings of international conference on image processing, volume 1, pages 520–523. IEEE, 1997.
  41. A dct-domain system for robust image watermarking. Signal processing, 66(3):357–372, 1998.
  42. A multiresolution watermark for digital images. In Proceedings of international conference on image processing, volume 1, pages 548–551. IEEE, 1997.
  43. An svd-based watermarking scheme for protecting rightful ownership. IEEE transactions on multimedia, 4(1):121–128, 2002.
  44. Steganogan: High capacity image steganography with gans. arXiv preprint arXiv:1901.03892, 2019.
  45. Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In Proceedings of the IEEE/CVF International conference on computer vision, pages 14448–14457, 2021.
  46. Deep model intellectual property protection via deep watermarking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4005–4020, 2021.
  47. Vlasislav Skripniuk. Watermarking for generative adversarial networks. PhD thesis, Universität des Saarlandes Saarbrücken, 2020.
  48. Watermarking neural networks with watermarked images. IEEE Transactions on Circuits and Systems for Video Technology, 31(7):2591–2601, 2020.
  49. Supervised gan watermarking for intellectual property protection. In 2022 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1–6. IEEE, 2022.
  50. Eigenfaces for recognition. Journal of cognitive neuroscience, 3(1):71–86, 1991.
  51. Face recognition using local binary patterns (lbp). Global Journal of Computer Science and Technology, 13(4):1–8, 2013.
  52. Face recognition using hog–ebgm. Pattern Recognition Letters, 29(10):1537–1543, 2008.
  53. Improved face recognition rate using hog features and svm classifier. IOSR Journal of Electronics and Communication Engineering, 11(04):34–44, 2016.
  54. Face recognition using multiple classifiers. In 2006 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’06), pages 179–186. IEEE, 2006.
  55. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701–1708, 2014.
  56. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212–220, 2017.
  57. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019.
  58. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  59. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  60. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  61. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 87–102. Springer, 2016.
  62. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  63. Adaface: Quality adaptive margin for face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18750–18759, 2022.
  64. Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices. In Biometric Recognition: 13th Chinese Conference, CCBR 2018, Urumqi, China, August 11-12, 2018, Proceedings 13, pages 428–438. Springer, 2018.
  65. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5265–5274, 2018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now