Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CLIPC8: Face liveness detection algorithm based on image-text pairs and contrastive learning (2311.17583v1)

Published 29 Nov 2023 in cs.CV

Abstract: Face recognition technology is widely used in the financial field, and various types of liveness attack behaviors need to be addressed. Existing liveness detection algorithms are trained on specific training datasets and tested on testing datasets, but their performance and robustness in transferring to unseen datasets are relatively poor. To tackle this issue, we propose a face liveness detection method based on image-text pairs and contrastive learning, dividing liveness attack problems in the financial field into eight categories and using text information to describe the images of these eight types of attacks. The text encoder and image encoder are used to extract feature vector representations for the classification description text and face images, respectively. By maximizing the similarity of positive samples and minimizing the similarity of negative samples, the model learns shared representations between images and texts. The proposed method is capable of effectively detecting specific liveness attack behaviors in certain scenarios, such as those occurring in dark environments or involving the tampering of ID card photos. Additionally, it is also effective in detecting traditional liveness attack methods, such as printing photo attacks and screen remake attacks. The zero-shot capabilities of face liveness detection on five public datasets, including NUAA, CASIA-FASD, Replay-Attack, OULU-NPU and MSU-MFSD also reaches the level of commercial algorithms. The detection capability of proposed algorithm was verified on 5 types of testing datasets, and the results show that the method outperformed commercial algorithms, and the detection rates reached 100% on multiple datasets. Demonstrating the effectiveness and robustness of introducing image-text pairs and contrastive learning into liveness detection tasks as proposed in this paper.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Deep face liveness detection based on nonlinear diffusion using convolution neural network. Signal, Image and Video Processing, 11:713–720, 2017.
  2. Face anti-spoofing using patch and depth-based cnns. In 2017 IEEE International Joint Conference on Biometrics (IJCB), pages 319–328. IEEE, 2017.
  3. Face anti-spoofing based on color texture analysis. In 2015 IEEE international conference on image processing (ICIP), pages 2636–2640. IEEE, 2015.
  4. Oulu-npu: A mobile face presentation attack database with real-world variations. In 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017), pages 612–618. IEEE, 2017.
  5. Transfas: Transformer-based network for face anti-spoofing using token guided inspection. In 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), pages 1–7. IEEE, 2023.
  6. A cascade face spoofing detector based on face anti-spoofing r-cnn and improved retinex lbp. IEEE Access, 7:170116–170133, 2019.
  7. On the effectiveness of local binary patterns in face anti-spoofing. In 2012 BIOSIG-proceedings of the international conference of biometrics special interest group (BIOSIG), pages 1–7. IEEE, 2012.
  8. On the effectiveness of vision transformers for zero-shot face anti-spoofing. In 2021 IEEE International Joint Conference on Biometrics (IJCB), pages 1–8. IEEE, 2021.
  9. Deep learning in object detection and recognition. 2019.
  10. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  11. Domain invariant vision transformer learning for face anti-spoofing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6098–6107, 2023.
  12. Ma-vit: Modality-agnostic vision transformers for face anti-spoofing. arXiv preprint arXiv:2304.07549, 2023.
  13. Fm-vit: Flexible modal vision transformers for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 2023.
  14. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
  15. Fgdnet: Fine-grained detection network towards face anti-spoofing. IEEE Transactions on Multimedia, 2022.
  16. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  17. Multi-modal face anti-spoofing transformer (mfast). In 2022 19th International Bhurban Conference on Applied Sciences and Technology (IBCAST), pages 494–501. IEEE, 2022.
  18. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10023–10031, 2019.
  19. Face spoofing detection by fusing binocular depth and spatial pyramid coding micro-texture features. In 2017 IEEE International Conference on Image Processing (ICIP), pages 96–100. IEEE, 2017.
  20. Face liveness detection from a single image with sparse low rank bilinear discriminative model. In Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part VI 11, pages 504–517. Springer, 2010.
  21. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  22. Robustness evaluation of commercial liveness detection platform. Chinese Journal of Network and Information Security, pages 180–189.
  23. Robustness evaluation of commercial liveness detection platform. Chinese Journal of Network and Information Security, pages 180–189, 2022.
  24. Face anti-spoofing using transformers with relation-aware mechanism. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(3):439–450, 2022.
  25. Learning multi-granularity temporal characteristics for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 17:1254–1269, 2022.
  26. Face spoof detection with image distortion analysis. IEEE Transactions on Information Forensics and Security, 10(4):746–761, 2015.
  27. Face liveness detection based on inceptionv3 and feature fusion. Journal of Computer Applications, pages 2037–2042.
  28. Face anti-spoofing: Model matters, so does data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3507–3516, 2019.
  29. Can hierarchical transformers learn facial geometry? Sensors, 23(2):929, 2023.
  30. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022.
  31. Transrppg: Remote photoplethysmography transformer for 3d mask face presentation attack detection. IEEE Signal Processing Letters, 28:1290–1294, 2021.
  32. A face antispoofing database with diverse attacks. In 2012 5th IAPR international conference on Biometrics (ICB), pages 26–31. IEEE, 2012.
  33. Attention-based spatial-temporal multi-scale network for face anti-spoofing. IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3):296–307, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now