Towards minimizing efforts for Morphing Attacks -- Deep embeddings for morphing pair selection and improved Morphing Attack Detection (2305.18216v2)
Abstract: Face Morphing Attacks pose a threat to the security of identity documents, especially with respect to a subsequent access control process, because it enables both individuals involved to exploit the same document. In this study, face embeddings serve two purposes: pre-selecting images for large-scale Morphing Attack generation and detecting potential Morphing Attacks. We build upon previous embedding studies in both use cases using the MagFace model. For the first objective, we employ an pre-selection algorithm that pairs individuals based on face embedding similarity. We quantify the attack potential of differently morphed face images to compare the usability of pre-selection in automatically generating numerous successful Morphing Attacks. Regarding the second objective, we compare embeddings from two state-of-the-art face recognition systems in terms of their ability to detect Morphing Attacks. Our findings demonstrate that ArcFace and MagFace provide valuable face embeddings for image pre-selection. Both open-source and COTS face recognition systems are susceptible to generated attacks, particularly when pre-selection is based on embeddings rather than random pairing which was only constrained by soft biometrics. More accurate face recognition systems exhibit greater vulnerability to attacks, with COTS systems being the most susceptible. Additionally, MagFace embeddings serve as a robust alternative for detecting morphed face images compared to the previously used ArcFace embeddings. The results endorse the advantages of face embeddings in more effective image pre-selection for face morphing and accurate detection of morphed face images. This is supported by extensive analysis of various designed attacks. The MagFace model proves to be a powerful alternative to the commonly used ArcFace model for both objectives, pre-selection and attack detection.
- Face Recognition Systems: A Survey. Sensors. 2020;20(2):342. doi:10.3390/s20020342.
- Automated border control e-gates and facial recognition systems. Computers & Security. 2016;62:49–72. doi:10.1016/j.cose.2016.07.001.
- Regulation (EU) 2017/2226 of the European Parliament and of the Council of 30 November 2017 establishing an Entry/Exit System (EES) to register entry and exit data and refusal of entry data of third-country nationals crossing the external borders of the Member States and determining the conditions for access to the EES for law enforcement purposes, and amending the Convention implementing the Schengen Agreement and Regulations (EC) No 767/2008 and (EU) No 1077/2011; 2017.
- Face Recognition Systems Under Morphing Attacks: A Survey. IEEE Access. 2016;7:23012–23026. doi:10.1109/ACCESS.2019.2899367.
- On the effects of image alterations on face recognition accuracy. In: Face recognition across the imaging spectrum; 2016. p. 195–222. Available from: https://doi.org/10.1007/978-3-319-28501-6₉.
- UNCW. MORPH Non-Commercial Release Whitepaper.;. Available from: http://people.uncw.edu/vetterr/MORPH-NonCommercial-Stats.pdf.
- Torkar M. Morphing Cases in Slovenia; 2021.Available from: https://eab.org/events/program/220.
- Face Morphing Attack Generation & Detection: A Comprehensive Survey. IEEE Transactions on Technology and Society. 2021; p. 1–1. doi:10.1109/TTS.2021.3066254.
- Face morphing detection in the presence of printing/scanning and heterogeneous image sources. IET Biometrics. 2021;10(3):290–303. doi:10.1049/bme2.12021.
- Tapia JE, Busch C. Single morphing attack detection using feature selection and visualization based on mutual information. IEEE access : practical innovations, open solutions. 2021;9:167628–167641. doi:10.1109/ACCESS.2021.3136485.
- Biometric Systems under Morphing Attacks: Assessment of Morphing Techniques and Vulnerability Reporting. In: 2017 International Conference of the Biometrics Special Interest Group (BIOSIG). Darmstadt, Germany: IEEE; 2017. p. 1–7. Available from: http://ieeexplore.ieee.org/document/8053499/.
- Finding the Suitable Doppelgänger for a Face Morphing Attack. In: 2020 IEEE International Joint Conference on Biometrics (IJCB). Houston, TX, USA: IEEE; 2020. p. 1–7. Available from: https://ieeexplore.ieee.org/document/9304878/.
- Face morphing versus face averaging: Vulnerability and detection. In: 2017 IEEE International Joint Conference on Biometrics (IJCB). Denver, CO: IEEE; 2017. p. 555–563. Available from: http://ieeexplore.ieee.org/document/8272742/.
- Morphing Attack Detection – Database, Evaluation Platform and Benchmarking. arXiv:200606458 [cs]. 2020;.
- OpenFace: A general-purpose face recognition library with mobile applications. CMU School of Computer Science; p. 20.
- Will Koehrsen. Neural Network Embeddings Explained.; 2018. Available from: https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526.
- Shi Y, Jain A. Probabilistic Face Embeddings. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE; 2019. p. 6901–6910. Available from: https://ieeexplore.ieee.org/document/9008376/.
- FaceNet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015; p. 815–823. doi:10.1109/CVPR.2015.7298682.
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition. arXiv:180107698 [cs]. 2019;.
- MagFace: A Universal Representation for Face Recognition and Quality Assessment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021; p. 10.
- Is your biometric system robust to morphing attacks? In: 2017 5th International Workshop on Biometrics and Forensics (IWBF). Coventry: IEEE; 2017. p. 1–6. Available from: https://ieeexplore.ieee.org/document/7935079/.
- Morph Attack Potential. proceedings of the IEEE International Workshop on Biometrics and Forensics (IWBF), Salzburg, Austria. 2022;.
- ISO/IEC. International Standard ISO/IEC 20059: Methodologies to evaluate the resistance of biometric recognition systems to morphing attacks; 2023. Available from: https://www.iso.org/standard/86084.html.
- On the Influence of Ageing on Face Morph Attacks: Vulnerability and Detection. In: 2020 IEEE International Joint Conference on Biometrics (IJCB). Houston, TX, USA: IEEE; 2020. p. 1–10. Available from: https://ieeexplore.ieee.org/document/9304856/.
- Deep Face Representations for Differential Morphing Attack Detection. arXiv:200101202 [cs]. 2020;.
- Deep Face Recognition. In: Procedings of the British Machine Vision Conference 2015. Swansea: British Machine Vision Association; 2015. p. 41.1–41.12. Available from: http://www.bmva.org/bmvc/2015/papers/paper041/index.html.
- DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE; 2014. p. 1701–1708. Available from: https://ieeexplore.ieee.org/document/6909616.
- Serengil SI, Ozpinar A. LightFace: A hybrid deep face recognition framework. In: 2020 innovations in intelligent systems and applications conference (ASYU); 2020. p. 23–27.
- Nguyen HV, Bai L. Cosine similarity metric learning for face verification. In: Asian conference on computer vision; 2010. p. 709–720.
- Ricanek K, Tesafaye T. Morph: A longitudinal image database of normal adult age-progression. In: 7th international conference on automatic face and gesture recognition (FGR06). IEEE; 2006. p. 341–345.
- Quek A. face morpher;. Available from: https://github.com/alyssaq/face_morpher.
- The magic passport. In: IEEE international joint conference on biometrics; 2014. p. 1–7.
- Face demorphing. IEEE Transactions on Information Forensics and Security. 2018;13(4):1008–1017. doi:10.1109/TIFS.2017.2777340.
- Decoupling texture blending and shape warping in face morphing. In: 2019 international conference of the biometrics special interest group (BIOSIG); 2019. p. 1–5.
- Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH: IEEE; 2014. p. 1867–1874. Available from: https://ieeexplore.ieee.org/document/6909637.
- MIPGAN - Generating Strong and High Quality Morphing Attacks Using Identity Prior Driven GAN. IEEE Transactions on Biometrics, Behavior, and Identity Science. 2021;3(3):365–383. doi:10.1109/TBIOM.2021.3072349.
- Can GAN generated morphs threaten face recognition systems equally as landmark based morphs?-vulnerability and detection. In: 2020 8th International Workshop on Biometrics and Forensics (IWBF). IEEE; 2020. p. 1–6.
- Martinez MA. PyEER;. Available from: https://github.com/manuelaguadomtz/pyeer.
- Frontex. Best practice technical guidelines for automated border control (ABC) systems; 2015.
- Available from: http://arxiv.org/abs/2211.11039.
- Detecting Morphed Face Attacks Using Residual Noise from Deep Multi-Scale Context Aggregation Network. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2020; p. 10.
- ISO/IEC. International Standard ISO/IEC 30107-3 (First edition) Information technology - Biometric presentation attack detection - Part 3: Testing and reporting.; 2017. Available from: https://www.iso.org/standard/67381.html.
- Available from: https://eab.org/cgi-bin/dl.pl?/upload/documents/2026/Ngan-FRVT-MORPH-NBLAW-210303.pdf.
- The effect of face morphing on face image quality. In: 2021 international conference of the biometrics special interest group (BIOSIG); 2021. p. 1–5.
- Available from: https://github.com/usnistgov/frvt/blob/nist-pages/reports/morph/frvt_morph_report_2022_07_14.pdf.
- Fingerprint and on-line signature verification competitions at ICB 2009. In: Advances in Biometrics: Third International Conference, ICB 2009, Alghero, Italy, June 2-5, 2009. Proceedings 3. Springer; 2009. p. 725–732.
- FVC-onGoing Report on Differential Morph Attack Detection; 2023. Available from: https://biolab.csr.unibo.it/FvcOnGoing/UI/Form/AlgResult.aspx?algId=8570.
- To detect or not to detect: The right faces to morph. In: 2019 international conference on biometrics (ICB); 2019. p. 1–8.
- Overview of the Face Recognition Grand Challenge. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). vol. 1. San Diego, CA, USA: IEEE; 2005. p. 947–954. Available from: http://ieeexplore.ieee.org/document/1467368/.
- Information theoretic measures for clusterings comparison: is a correction for chance necessary? In: Proceedings of the 26th Annual International Conference on Machine Learning - ICML ‘09; 2009. p. 8.
- Constrained k-means clustering. Microsoft Research, Redmond. 2000;20(0):0.