Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainability (2401.13641v2)

Published 24 Jan 2024 in cs.CV, cs.AI, cs.CY, and cs.LG

Abstract: LLMs such as GPT developed by OpenAI, have already shown astonishing results, introducing quick changes in our society. This has been intensified by the release of ChatGPT which allows anyone to interact in a simple conversational way with LLMs, without any experience in the field needed. As a result, ChatGPT has been rapidly applied to many different tasks such as code- and song-writer, education, virtual assistants, etc., showing impressive results for tasks for which it was not trained (zero-shot learning). The present study aims to explore the ability of ChatGPT, based on the recent GPT-4 multimodal LLM, for the task of face biometrics. In particular, we analyze the ability of ChatGPT to perform tasks such as face verification, soft-biometrics estimation, and explainability of the results. ChatGPT could be very valuable to further increase the explainability and transparency of automatic decisions in human scenarios. Experiments are carried out in order to evaluate the performance and robustness of ChatGPT, using popular public benchmarks and comparing the results with state-of-the-art methods in the field. The results achieved in this study show the potential of LLMs such as ChatGPT for face biometrics, especially to enhance explainability. For reproducibility reasons, we release all the code in GitHub.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. K. Hu, “ChatGPT Sets Record for Fastest-Growing User Base - Analyst Note,” 2023, Reuters Technology. [Online]. Available: https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/
  2. S. S. Biswas, “Role of Chat GPT in Public Health,” Annals of Biomedical Engineering, pp. 868–869, 2023.
  3. E. Kasneci, K. Seßler, S. Küchemann et al., “ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education,” Learning and Individual Differences, vol. 103, p. 102274, 2023.
  4. P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing,” ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023.
  5. W. X. Zhao, K. Zhou, J. Li et al., “A Survey of Large Language Models,” arXiv preprint arXiv:2303.18223, 2023.
  6. H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, “Capabilities of GPT-4 on Medical Challenge Problems,” arXiv preprint arXiv:2303.13375, 2023.
  7. Z. Liu, X. Yu, L. Zhang et al., “DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4,” arXiv preprint arXiv:2303.11032, 2023.
  8. A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,” Nature Medicine, vol. 29, no. 8, pp. 1930–1940, 2023.
  9. A. Singla, “Evaluating ChatGPT and GPT-4 for Visual Programming,” in Proc. Conference on International Computing Education Research, 2023.
  10. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving Language Understanding by Generative Pre-Training,” 2018. [Online]. Available: https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
  11. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All You Need,” in Proc. Advances in Neural Information Processing Systems, 2017.
  12. Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun, “Transformers in Time Series: A Survey,” arXiv preprint arXiv:2202.07125, 2022.
  13. P. Delgado-Santos, R. Tolosana, R. Guest, F. Deravi, and R. Vera-Rodriguez, “Exploring transformers for behavioural biometrics: A case study in gait recognition,” Pattern Recognition, vol. 143, p. 109798, 2023.
  14. Y. Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, and S. Fidler, “Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books,” in Proc. IEEE/CVF International Conference on Computer Vision, 2015.
  15. T. Brown, B. Mann, N. Ryder et al., “Language Models are Few-Shot Learners,” in Proc. Advances in Neural Information Processing Systems, 2020.
  16. A. Chowdhery, S. Narang, J. Devlin et al., “PaLM: Scaling Language Modeling with Pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023. [Online]. Available: http://jmlr.org/papers/v24/22-1144.html
  17. R. Anil, A. M. Dai, O. Firat et al., “PaLM 2 Technical Report,” arXiv preprint arXiv:2305.10403, 2023.
  18. H. Touvron, T. Lavril, G. Izacard et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv preprint arXiv:2302.13971, 2023.
  19. J. Achiam, S. Adler, S. Agarwal et al., “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774, 2023.
  20. G. Guo and N. Zhang, “A Survey on Deep Learning based Face Recognition,” Computer Vision and Image Understanding, vol. 189, p. 102805, 2019.
  21. P. Melzi, R. Tolosana, R. Vera-Rodriguez et al., “FRCSyn Challenge at WACV 2024: Face Recognition Challenge in the Era of Synthetic Data,” in Proc. IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, 2024.
  22. P. Terhorst, D. Fahrmann, J. N. Kolf, N. Damer, F. Kirchbuchner, and A. Kuijper, “MAAD-Face: A Massively Annotated Attribute Dataset for Face Images,” IEEE Transactions on Information Forensics and Security, vol. 16, pp. 3942–3957, 2021.
  23. G. B. Huang, M. Mattar, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database forstudying face recognition in unconstrained environments,” in Proc. Workshop on faces in ‘Real-Life’ Images: detection, alignment, and recognition, 2008.
  24. J. Neves, J. Moreno, and H. Proença, “QUIS‐CAMPI: an annotated multi‐biometrics data feed from surveillance scenarios,” IET Biometrics, vol. 7, no. 4, pp. 371–379, 2017.
  25. Z. Cheng, X. Zhu, and S. Gong, “Low-resolution face recognition,” in Proc. Asian Conference on Computer Vision, 2019.
  26. M. Wang, Y. Zhang, and W. Deng, “Meta Balanced Network for Fair Face Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2021.
  27. S. Sengupta, J.-C. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs, “Frontal to profile face verification in the wild,” in Proc. IEEE/CVF Winter Conference on Applications of Computer Vision, 2016.
  28. S. Moschoglou, A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, and S. Zafeiriou, “AgeDB: The First Manually Collected, In-the-Wild Age Database,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2017.
  29. M. E. Erakin, U. Demir, and H. K. Ekenel, “On Recognizing Occluded Faces in the Wild,” in Proc. International Conference of the Biometrics Special Interest Group, 2021.
  30. J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia, and S. Zafeiriou, “ArcFace: Additive Angular Margin Loss for Deep Face Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 5962–5979, 2022.
  31. M. Kim, A. K. Jain, and X. Liu, “AdaFace: Quality Adaptive Margin for Face Recognition,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  32. Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, “VGGFace2: A Dataset for Recognising Faces across Pose and Age,” in Proc. 13th IEEE International Conference on Automatic Face & Gesture Recognition, 2018.
  33. E. Gonzalez-Sosa, J. Fierrez, R. Vera-Rodriguez, and F. Alonso-Fernandez, “Facial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluation,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 8, pp. 2001–2014, 2018.
  34. I. C. Duta, L. Liu, F. Zhu, and L. Shao, “Improved Residual Networks for Image and Video Recognition,” in Proc. 25th International Conference on Pattern Recognition, 2021.
  35. J. Deng, J. Guo, D. Zhang, Y. Deng, X. Lu, and S. Shi, “Lightweight Face Recognition Challenge,” in Proc. IEEE/CVF International Conference on Computer Vision Workshop, 2019.
  36. Z. Zhu, G. Huang, J. Deng, Y. Ye, J. Huang, X. Chen, J. Zhu, T. Yang, J. Lu, D. Du, and J. Zhou, “WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
  37. K. Karkkainen and J. Joo, “FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation,” in Proc. IEEE/CVF Winter Conference on Applications of Computer Vision, 2021.
  38. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
  39. L. Lucy and D. Bamman, “Gender and Representation Bias in GPT-3 Generated Stories,” in Proc. of the Third Workshop on Narrative Understanding, 2021. [Online]. Available: https://aclanthology.org/2021.nuse-1.5
  40. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. EEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Ivan DeAndres-Tame (8 papers)
  2. Ruben Tolosana (79 papers)
  3. Ruben Vera-Rodriguez (66 papers)
  4. Aythami Morales (93 papers)
  5. Julian Fierrez (131 papers)
  6. Javier Ortega-Garcia (51 papers)
Citations (15)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets