Leveraging Diffusion For Strong and High Quality Face Morphing Attacks (2301.04218v4)
Abstract: Face morphing attacks seek to deceive a Face Recognition (FR) system by presenting a morphed image consisting of the biometric qualities from two different identities with the aim of triggering a false acceptance with one of the two identities, thereby presenting a significant threat to biometric systems. The success of a morphing attack is dependent on the ability of the morphed image to represent the biometric characteristics of both identities that were used to create the image. We present a novel morphing attack that uses a Diffusion-based architecture to improve the visual fidelity of the image and the ability of the morphing attack to represent characteristics from both identities. We demonstrate the effectiveness of the proposed attack by evaluating its visual fidelity via the Frechet Inception Distance (FID). Also, extensive experiments are conducted to measure the vulnerability of FR systems to the proposed attack. The ability of a morphing attack detector to detect the proposed attack is measured and compared against two state-of-the-art GAN-based morphing attacks along with two Landmark-based attacks. Additionally, a novel metric to measure the relative strength between different morphing attacks is introduced and evaluated.
- L. J. Spreeuwers, A. J. Hendrikse, and K. J. Gerritsen, “Evaluation of automatic face recognition for automatic border control on actual data recorded of travellers at schiphol airport,” in Proc. of the Int’l Conf. of Biometrics Special Interest Group (BIOSIG), 2012, pp. 1–6.
- D. J. Robertson, R. S. S. Kramer, and A. M. Burton, “Fraudulent id using face morphs: Experiments on human and automatic recognition,” PLOS ONE, vol. 12, no. 3, pp. 1–12, 03 2017. [Online]. Available: https://doi.org/10.1371/journal.pone.0173319
- R. Raghavendra, K. B. Raja, and C. Busch, “Detecting morphed face images,” in IEEE 8th Int’l Conf. on Biometrics Theory, Applications and Systems (BTAS), 2016, pp. 1–7.
- J. T. A. Andrews, T. Tanay, and L. D. Griffin, “Multiple-identity image attacks against face-based identity verification,” CoRR, vol. abs/1906.08507, 2019. [Online]. Available: http://arxiv.org/abs/1906.08507
- R. Raghavendra, K. Raja, S. Venkatesh, and C. Busch, “Face morphing versus face averaging: Vulnerability and detection,” IEEE Int’l Joint Conf. on Biometrics (IJCB), pp. 555–563, 2017.
- C. Burt, “Face morphing threat to biometric identity credentials’ trustworthiness a growing problem,” BiometricUpdate.com.
- S. Venkatesh, R. Ramachandra, K. Raja, L. Spreeuwers, R. Veldhuis, and C. Busch, “Morphed face detection based on deep color residual noise,” in the 9th Int’l Conf. on Image Pr. Theory, Tools and Applications (IPTA), 2019, pp. 1–6.
- U. Scherhag, R. Raghavendra, K. B. Raja, M. Gomez-Barrero, C. Rathgeb, and C. Busch, “On the vulnerability of face recognition systems towards morphed face attacks,” in 2017 5th International Workshop on Biometrics and Forensics (IWBF), 2017, pp. 1–6.
- U. Scherhag, L. Debiasi, C. Rathgeb, C. Busch, and A. Uhl, “Detection of face morphing attacks based on prnu analysis,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 1, no. 4, pp. 302–317, 2019.
- Z. Blasingame and C. Liu, “Leveraging adversarial learning for the detection of morphing attacks,” 2021 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–8, 2021.
- S. Venkatesh, R. Ramachandra, K. Raja, and C. Busch, “Face Morphing Attack Generation & Detection: A Comprehensive Survey,” arXiv e-prints arXiv:2011.02045, Nov. 2020.
- T. Ojala, M. Pietikäinen, and D. Harwood, “A comparative study of texture measures with classification based on featured distributions,” Pattern Recognit., vol. 29, pp. 51–59, 1996.
- J. Kannala and E. Rahtu, “Bsif: Binarized statistical image features,” in Proceedings of the 21st Int’l Conf. on Pattern Recognition (ICPR2012), 2012, pp. 1363–1366.
- C. Seibold, W. Samek, A. Hilsmann, and P. Eisert, “Detection of face morphing attacks by deep learning,” in Digital Forensics and Watermarking, C. Kraetzer, Y.-Q. Shi, J. Dittmann, and H. J. Kim, Eds., 2017, pp. 107–120.
- U. Scherhag, C. Rathgeb, J. Merkle, and C. Busch, “Deep face representations for differential morphing attack detection,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3625–3639, 2020.
- M. Ngan, P. Grother, K. Hanaoka, and J. Kuo, “Face recognition vendor test (frvt) part 4: Morph - performance of automated face morph detection,” 2020-03-06 2020.
- E. Sarkar, P. Korshunov, L. Colbois, and S. Marcel, “Vulnerability analysis of face morphing attacks from landmarks and generative adversarial networks,” ArXiv, vol. abs/2012.05344, 2020.
- K. O’Haire, S. Soleymani, B. Chaudhary, P. Aghdaie, J. Dawson, and N. M. Nasrabadi, “Adversarially perturbed wavelet-based morphed face generation,” in 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021, pp. 01–05.
- N. Damer, A. M. Saladié, A. Braun, and A. Kuijper, “Morgan: Recognition vulnerability and attack detectability of face morphing attacks created by generative adversarial network,” in 2018 IEEE 9th Int’l Conf. on Biometrics Theory, Applications and Systems (BTAS), 2018, pp. 1–10.
- S. Venkatesh, H. Zhang, R. Ramachandra, K. Raja, N. Damer, and C. Busch, “Can gan generated morphs threaten face recognition systems equally as landmark based morphs? - vulnerability and detection,” in 2020 8th International Workshop on Biometrics and Forensics (IWBF), 2020, pp. 1–6.
- T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 8107–8116.
- H. Zhang, S. Venkatesh, R. Ramachandra, K. Raja, N. Damer, and C. Busch, “Mipgan—generating strong and high quality morphing attacks using identity prior driven gan,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 3, no. 3, pp. 365–383, 2021.
- P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates, Inc., 2021, pp. 8780–8794. [Online]. Available: https://proceedings.neurips.cc/paper/2021/file/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf
- L. DeBruine and B. Jones, “Face research lab london set.”
- S. Venkatesh, H. Zhang, R. Ramachandra, K. Raja, N. Damer, and C. Busch, “Can gan generated morphs threaten face recognition systems equally as landmark based morphs? – vulnerability and detection,” 2020.
- E. Sarkar, P. Korshunov, L. Colbois, and S. Marcel, “Are gan-based morphs threatening face recognition?” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 2959–2963.
- A. Quek, “Facemorpher,” https://github.com/alyssaq/face_morpher, 2019.
- S. Milborrow and F. Nicolls, “Active shape models with sift descriptors and mars,” in 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, 2014, pp. 380–387.
- U. Scherhag, J. Kunze, C. Rathgeb, and C. Busch, “Face morph detection for unknown morphing algorithms and image sources: a multi-scale block local binary pattern fusion approach,” IET Biometrics, vol. 9, pp. 278–289, 11 2020.
- D. E. King, “Dlib-ml: A machine learning toolkit,” J. Mach. Learn. Res., vol. 10, p. 1755–1758, Dec. 2009.
- L. Colbois and S. Marcel, “On the detection of morphing attacks generated by gans,” in 2022 International Conference of the Biometrics Special Interest Group (BIOSIG), 2022, pp. 1–5.
- D. Roich, R. Mokady, A. H. Bermano, and D. Cohen-Or, “Pivotal tuning for latent-based editing of real images,” ACM Transactions on Graphics (TOG), 2021.
- T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4396–4405.
- A. Brock, J. Donahue, and K. Simonyan, “Large scale GAN training for high fidelity natural image synthesis,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=B1xsqj09Fm
- J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 6840–6851. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
- O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds. Cham: Springer International Publishing, 2015, pp. 234–241.
- J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=St1giarCHLP
- K. Preechakul, N. Chatthee, S. Wizadwongsa, and S. Suwajanakorn, “Diffusion autoencoders: Toward a meaningful and decodable representation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 10 619–10 629.
- A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
- Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, “Vggface2: A dataset for recognising faces across pose and age,” in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 67–74.
- F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 815–823.
- J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 4690–4699.
- O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in BMVC, 2015.
- J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141.
- I. C. Duta, L. Liu, F. Zhu, and L. Shao, “Improved residual networks for image and video recognition,” arXiv preprint arXiv:2004.04989, 2020.
- X. An, X. Zhu, Y. Gao, Y. Xiao, Y. Zhao, Z. Feng, L. Wu, B. Qin, M. Zhang, D. Zhang, and Y. Fu, “Partial fc: Training 10 million identities on a single machine,” in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 1445–1449.
- K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016.
- P. Phillips, H. Wechsler, J. Huang, and P. J. Rauss, “The feret database and evaluation procedure for face-recognition algorithms,” Image Vis. Comput., vol. 16, pp. 295–306, 1998.
- P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, “Overview of the face recognition grand challenge,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, 2005, pp. 947–954 vol. 1.
- M. Lucic, K. Kurach, M. Michalski, O. Bousquet, and S. Gelly, “Are gans created equal? a large-scale study,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, ser. NIPS’18. Red Hook, NY, USA: Curran Associates Inc., 2018, p. 698–707.
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
- M. Seitzer, “pytorch-fid: FID Score for PyTorch,” https://github.com/mseitzer/pytorch-fid, August 2020, version 0.2.1.
- FRONTEX, “Best practice technical guidelines for automated border control abc systems,” Frontex, the European Border and Coast Guard Agency, Plac Europejski 6, 00-844 Warsaw, Poland, Tech. Rep., 2015.
- U. Scherhag, A. Nautsch, C. Rathgeb, M. Gomez-Barrero, R. N. J. Veldhuis, L. Spreeuwers, M. Schils, D. Maltoni, P. Grother, S. Marcel, R. Breithaupt, R. Ramachandra, and C. Busch, “Biometric systems under morphing attacks: Assessment of morphing techniques and vulnerability reporting,” in 2017 International Conference of the Biometrics Special Interest Group (BIOSIG), 2017, pp. 1–7.
- T. Karras, M. Aittala, S. Laine, E. Härkönen, J. Hellsten, J. Lehtinen, and T. Aila, “Alias-free generative adversarial networks,” in Proc. NeurIPS, 2021.
- S. Xie, R. B. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5987–5995, 2017.