Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking (2405.05900v1)

Published 9 May 2024 in cs.CV

Abstract: Masked face recognition (MFR) has emerged as a critical domain in biometric identification, especially by the global COVID-19 pandemic, which introduced widespread face masks. This survey paper presents a comprehensive analysis of the challenges and advancements in recognising and detecting individuals with masked faces, which has seen innovative shifts due to the necessity of adapting to new societal norms. Advanced through deep learning techniques, MFR, along with Face Mask Recognition (FMR) and Face Unmasking (FU), represent significant areas of focus. These methods address unique challenges posed by obscured facial features, from fully to partially covered faces. Our comprehensive review delves into the various deep learning-based methodologies developed for MFR, FMR, and FU, highlighting their distinctive challenges and the solutions proposed to overcome them. Additionally, we explore benchmark datasets and evaluation metrics specifically tailored for assessing performance in MFR research. The survey also discusses the substantial obstacles still facing researchers in this field and proposes future directions for the ongoing development of more robust and effective masked face recognition systems. This paper serves as an invaluable resource for researchers and practitioners, offering insights into the evolving landscape of face recognition technologies in the face of global health crises and beyond.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (128)
  1. Refineface: Refinement neural network for high performance face detection. IEEE transactions on pattern analysis and machine intelligence, 43(11):4008–4020, 2020.
  2. Ganmasker: A two-stage generative adversarial network for high-quality face mask removal. Sensors, 23(16):7094, 2023.
  3. A novel gan-based network for unmasking of masked face. IEEE Access, 8:44276–44287, 2020.
  4. Kohtd: Kazakh offline handwritten text dataset. Signal Processing: Image Communication, 108:116827, 2022.
  5. Handwritten kazakh and russian (hkr) database for text recognition. Multimedia Tools and Applications, 80:33075–33097, 2021.
  6. Classification of handwritten names of cities and handwritten text recognition using various deep learning models. arXiv preprint arXiv:2102.04816, 2021.
  7. Attention-based fully gated cnn-bgru for russian handwritten text. Journal of Imaging, 6(12):141, 2020.
  8. Advancements and challenges in arabic optical character recognition: A comprehensive survey. arXiv preprint arXiv:2312.11812, 2023.
  9. Ae-lstm: Autoencoder with lstm-based intrusion detection in iot. In 2022 International Telecommunications Conference (ITC-Egypt), pages 1–6. IEEE, 2022.
  10. Lkd-stnn: A lightweight malicious traffic detection method for internet of things based on knowledge distillation. IEEE Internet of Things Journal, 2023.
  11. Conv3d-based video violence detection network using optical flow and rgb data. Sensors, 24(2):317, 2024.
  12. A novel two-stage deep learning model for network intrusion detection: Lstm-ae. IEEE Access, 2023.
  13. Deep learning for table detection and structure recognition: A survey. ACM Computing Surveys, 2022.
  14. Tncr: Table net detection and classification dataset. Neurocomputing, 473:79–97, 2022.
  15. Automated question-answer medical model based on deep learning technology. In Proceedings of the 6th International Conference on Engineering & MIS 2020, pages 1–8, 2020.
  16. Transformers and language models in form understanding: A comprehensive review of scanned document analysis. arXiv preprint arXiv:2403.04080, 2024.
  17. Exploring the state of the art in legal qa systems. Journal of Big Data, 10(1):127, 2023.
  18. Amurd: annotated multilingual receipts dataset for cross-lingual key information extraction and classification. arXiv preprint arXiv:2309.09800, 2023.
  19. Generator-retriever-generator: A novel approach to open-domain question answering. arXiv preprint arXiv:2307.11278, 2023.
  20. Arabicaqa: A comprehensive dataset for arabic question answering. arXiv preprint arXiv:2403.17848, 2024.
  21. Neural network estimation model to optimize timing and schedule of software projects. In 2021 IEEE International Conference on Smart Information Systems and Technologies (SIST), pages 1–7. IEEE, 2021.
  22. Customer profiling, segmentation, and sales prediction using ai in direct marketing. Neural Computing and Applications, 36(9):4995–5005, 2024.
  23. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
  24. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  25. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008.
  26. Masked face recognition for secure authentication. arXiv preprint arXiv:2008.11104, 2020.
  27. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.
  28. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  29. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
  30. A survey on techniques to handle face recognition challenges: occlusion, single sample per subject and expression. Artificial Intelligence Review, 52:949–979, 2019.
  31. A survey on occluded face recognition. In Proceedings of the 2020 9th International Conference on Networks, Communication and Computing, pages 40–49, 2020.
  32. A survey of face recognition techniques under occlusion. IET biometrics, 10(6):581–606, 2021.
  33. Masked face recognition using deep learning: A review. Electronics, 10(21):2666, 2021.
  34. A survey on masked facial detection methods and datasets for fighting against covid-19. IEEE Transactions on Artificial Intelligence, 3(3):323–343, 2021.
  35. Comprehensive review on facemask detection techniques in the context of covid-19. IEEE access, 9:106839–106864, 2021.
  36. Masked face recognition dataset and application. arXiv:2003.09093, 2020.
  37. Masked face recognition with generative data augmentation and domain constrained ranking. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2246–2254, 2020.
  38. Face masks detection dataset. https://trainingdata.pro/data-market/face-masks-detection/#header-form.
  39. Masked face recognition with latent part detection. In Proceedings of the 28th ACM international Conference on multimedia, pages 2281–2289, 2020.
  40. Masked face recognition with convolutional neural networks and local binary patterns. Applied Intelligence, 52(5):5497–5512, 2022.
  41. A novel deepmasknet model for face mask detection and masked facial recognition. Journal of King Saud University-Computer and Information Sciences, 34(10):9905–9914, 2022.
  42. Masked face recognition dataset and application. IEEE Transactions on Biometrics, Behavior, and Identity Science, 5(2):298–304, 2023.
  43. Aizootech-facemasksdetection. https://github.com/AIZOOTech/FaceMaskDetection/tree/master.
  44. Detecting masked faces in the wild with lle-cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2682–2690, 2017.
  45. How to correctly detect face-masks for covid-19 from visual information? Applied Sciences, 11(5), 2021.
  46. Face mask detection using yolov3 and faster r-cnn models: Covid-19 environment. Multimedia Tools and Applications, 80:19753–19768, 2021.
  47. A novel detection framework about conditions of wearing face mask for helping control the spread of covid-19. Ieee Access, 9:42975–42984, 2021.
  48. Unconstrained face mask and face-hand interaction datasets: building a computer vision system to help prevent the transmission of covid-19. Signal, Image and Video Processing, pages 1–8, 2022.
  49. Mask dataset. https://www.kaggle.com/datasets/andrewmvd/face-mask-detection/data.
  50. Real-time face mask detection method based on yolov3. Electronics, 10(7):837, 2021.
  51. Moxa: A deep learning based unmanned approach for real-time monitoring of people wearing medical masks. Transactions of the Indian National Academy of Engineering, 5:509–518, 2020.
  52. Tfm a dataset for detection and recognition of masked faces in the wild. In Proceedings of the 4th ACM International Conference on Multimedia in Asia, pages 1–7, 2022.
  53. Hybrid transfer learning and broad learning system for wearing mask detection in the covid-19 era. IEEE Transactions on Instrumentation and Measurement, 70:1–12, 2021.
  54. Bias-aware face mask detection dataset. arXiv preprint arXiv:2211.01207, 2022.
  55. Wider face: A face detection benchmark. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5525–5533, 2016.
  56. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019.
  57. Frontal to profile face verification in the wild. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–9. IEEE, 2016.
  58. Agedb: the first manually collected, in-the-wild age database. In proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 51–59, 2017.
  59. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pages 67–74. IEEE, 2018.
  60. Maskedface-net–a dataset of correctly/incorrectly masked face images in the context of covid-19. Smart Health, 19:100144, 2021.
  61. Smfd-github. https://github.com/prajnasb/observations/tree/master/experiements/data.
  62. Fmdd-kaggle. https://www.kaggle.com/datasets/omkargurav/face-mask-dataset.
  63. Self-restrained triplet loss for accurate masked face recognition. Pattern Recognition, 124:108473, 2022.
  64. Iarpa janus benchmark-c: Face dataset and protocol. In 2018 international conference on biometrics (ICB), pages 158–165. IEEE, 2018.
  65. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
  66. Mfcosface: a masked-face recognition algorithm based on large margin cosine loss. Applied sciences, 11(16):7310, 2021.
  67. Effective attention-based mechanism for masked face recognition. Applied Sciences, 12(11):5590, 2022.
  68. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019.
  69. Davis E King. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research, 10:1755–1758, 2009.
  70. An asian face dataset and how race influences face recognition. In Advances in Multimedia Information Processing–PCM 2018: 19th Pacific-Rim Conference on Multimedia, Hefei, China, September 21-22, 2018, Proceedings, Part II 19, pages 372–383. Springer, 2018.
  71. Receiver-operating characteristic (roc) plots: a fundamental evaluation tool in clinical medicine. Clinical chemistry, 39(4):561–577, 1993.
  72. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  73. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
  74. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3):209–212, 2012.
  75. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12):4695–4708, 2012.
  76. An efficient and effective deep learning-based model for real-time face mask detection. Sensors, 22(7):2602, 2022.
  77. A deep learning-based approach for real-time facemask detection. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 1478–1481. IEEE, 2021.
  78. Face mask detection using convolutional neural network (cnn) to reduce the spread of covid-19. In 2021 5th international conference on trends in electronics and informatics (ICOEI), pages 1231–1237. IEEE, 2021.
  79. A real time face mask detection system using convolutional neural network. Multimedia Tools and Applications, 81(11):14999–15015, 2022.
  80. Face mask detection using deep learning: An approach to reduce risk of coronavirus spread. Journal of biomedical informatics, 120:103848, 2021.
  81. Multi-stage cnn architecture for face mask detection. In 2021 6th International Conference for Convergence in Technology (i2ct), pages 1–8. IEEE, 2021.
  82. Face mask detection using deep convolutional neural network and multi-stage image processing. Image and Vision Computing, 133:104657, 2023.
  83. Autonomous face mask detection using single shot multibox detector, and resnet-50 with identity retrieval through face matching using deep siamese neural network. Journal of Ambient Intelligence and Humanized Computing, 14(8):11195–11205, 2023.
  84. Burhan ul haque Sheikh and Aasim Zafar. Rrfmds: Rapid real-time face mask detection system for effective covid-19 monitoring. SN Computer Science, 4(3):288, 2023.
  85. Ssdmnv2: A real time dnn-based face mask detection system using single shot multibox detector and mobilenetv2. Sustainable cities and society, 66:102692, 2021.
  86. Face mask detection using transfer learning of inceptionv3. In Big Data Analytics: 8th International Conference, BDA 2020, Sonepat, India, December 15–18, 2020, Proceedings 8, pages 81–90. Springer, 2020.
  87. Control the covid-19 pandemic: Face mask detection using transfer learning. In 2020 IEEE 2nd International Conference on Electronics, Control, Optimization and Computer Science (ICECOCS), pages 1–5. IEEE, 2020.
  88. A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the covid-19 pandemic. Measurement, 167:108288, 2021.
  89. Face mask detection and classification via deep transfer learning. Multimedia Tools and Applications, pages 1–20, 2022.
  90. Face mask detection on photo and real-time video images using caffe-mobilenetv2 transfer learning. Applied Sciences, 13(2):935, 2023.
  91. Face mask wearing detection algorithm based on improved yolo-v4. Sensors, 21(9):3263, 2021.
  92. Deep learning-based face mask detection using yolov5. In 2021 9th International Electrical Engineering Congress (iEECON), pages 428–431. IEEE, 2021.
  93. Integration of improved yolov5 for face mask detector and auto-labeling to generate dataset for fighting against covid-19. The Journal of Supercomputing, 79(8):8966–8992, 2023.
  94. Fmd-yolo: An efficient face mask detection method for covid-19 prevention and control in public. Image and vision computing, 117:104341, 2022.
  95. A novel attention-based enhancement framework for face mask detection in complicated scenarios. Signal Processing: Image Communication, 116:116985, 2023.
  96. Enhancing covid-19 safety: Exploring yolov8 object detection for accurate face mask classification. International Journal of Intelligent Systems and Applications in Engineering, 11(2):892–897, 2023.
  97. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536–2544, 2016.
  98. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4471–4480, 2019.
  99. Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212, 2019.
  100. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV), pages 85–100, 2018.
  101. R-mnet: A perceptual adversarial network for image inpainting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2714–2723, 2021.
  102. Interactive removal of microphone object in facial images. Electronics, 8(10):1115, 2019.
  103. Generative face completion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3911–3919, 2017.
  104. Effective removal of user-selected foreground object from facial images using a novel gan-based network. IEEE Access, 8:109648–109661, 2020.
  105. Globally and locally consistent image completion. ACM Transactions on Graphics (ToG), 36(4):1–14, 2017.
  106. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022.
  107. Towards coherent image inpainting using denoising diffusion implicit models. International Conference on Machine Learning, 2023.
  108. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593–23606, 2022.
  109. Zero-shot image restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490, 2022.
  110. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  111. Look through masks: Towards masked face recognition with de-occlusion distillation. In Proceedings of the 28th ACM International Conference on Multimedia, pages 3016–3024, 2020.
  112. Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5505–5514, 2018.
  113. Cropping and attention based approach for masked face recognition. Applied Intelligence, 51:3012–3025, 2021.
  114. Walid Hariri. Efficient masked face recognition method during the covid-19 pandemic. Signal, image and video processing, 16(3):605–612, 2022.
  115. Masked face recognition using deep learning based on unmasked area. In 2022 Second International Conference on Power, Control and Computing Technologies (ICPC2T), pages 1–6. IEEE, 2022.
  116. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  117. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017.
  118. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  119. Learning bag-of-features pooling for deep convolutional neural networks. In Proceedings of the IEEE international conference on computer vision, pages 5755–5763, 2017.
  120. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020.
  121. Boosting masked face recognition with multi-task arcface. In 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pages 184–189. IEEE, 2022.
  122. GuiLing Wu. Masked face recognition algorithm for a contactless distribution cabinet. Mathematical problems in engineering, 2021:1–11, 2021.
  123. Masked-face recognition using deep metric learning and facemasknet-21. Applied Intelligence, 52(11):13268–13279, 2022.
  124. Masked face recognition using deep learning model. In 2021 3rd International Conference on Advances in Computing, Communication Control and Networking (ICAC3N), pages 428–432. IEEE, 2021.
  125. A real-time cnn-based lightweight mobile masked face recognition system. Ieee Access, 10:63496–63507, 2022.
  126. Advanced masked face recognition using robust and light weight deep learning model. International Journal of Computer Applications, 975:8887, 2024.
  127. An optimized machine learning and deep learning framework for facial and masked facial recognition. Emerging Science Journal, 7(4):1173–1187, 2023.
  128. Single camera masked face identification. In 2020 19th IEEE international conference on machine learning and applications (ICMLA), pages 57–60. IEEE, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Mohamed Mahmoud (24 papers)
  2. Mahmoud SalahEldin Kasem (4 papers)
  3. Hyun-Soo Kang (2 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets