Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofing (2403.14333v1)

Published 21 Mar 2024 in cs.CV

Abstract: Domain generalization (DG) based Face Anti-Spoofing (FAS) aims to improve the model's performance on unseen domains. Existing methods either rely on domain labels to align domain-invariant feature spaces, or disentangle generalizable features from the whole sample, which inevitably lead to the distortion of semantic feature structures and achieve limited generalization. In this work, we make use of large-scale VLMs like CLIP and leverage the textual feature to dynamically adjust the classifier's weights for exploring generalizable visual features. Specifically, we propose a novel Class Free Prompt Learning (CFPL) paradigm for DG FAS, which utilizes two lightweight transformers, namely Content Q-Former (CQF) and Style Q-Former (SQF), to learn the different semantic prompts conditioned on content and style features by using a set of learnable query vectors, respectively. Thus, the generalizable prompt can be learned by two improvements: (1) A Prompt-Text Matched (PTM) supervision is introduced to ensure CQF learns visual representation that is most informative of the content description. (2) A Diversified Style Prompt (DSP) technology is proposed to diversify the learning of style prompts by mixing feature statistics between instance-specific styles. Finally, the learned text features modulate visual features to generalization through the designed Prompt Modulation (PM). Extensive experiments show that the CFPL is effective and outperforms the state-of-the-art methods on several cross-domain datasets.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Oulu-npu: A mobile face presentation attack database with real-world variations. In FGR, pages 612–618, 2017.
  2. Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 397–406, 2021.
  3. Generalizable representation learning for mixture domain face anti-spoofing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1132–1139, 2021.
  4. On the effectiveness of local binary patterns in face anti-spoofing. In BIOSIG, 2012.
  5. Surveillance face presentation attack detection challenge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 6360–6370, 2023a.
  6. Surveillance face anti-spoofing. IEEE Transactions on Information Forensics and Security, 2023b.
  7. Deep pixel-wise binary supervision for face presentation attack detection. In ICB, 2019.
  8. Cross modal focal loss for rgbd face anti-spoofing. In CVPR, pages 7882–7891, 2021.
  9. Biometric face presentation attack detection with multi-channel convolutional neural network. TIFS, 2019.
  10. Adaptive transformers for robust few-shot cross-domain face anti-spoofing. arXiv preprint arXiv:2203.12175, 2022.
  11. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision, pages 1501–1510, 2017.
  12. Single-side domain generalization for face anti-spoofing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8484–8493, 2020.
  13. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694–9705, 2021.
  14. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023.
  15. Ma-vit: Modality-agnostic vision transformers for face anti-spoofing. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 1180–1186. International Joint Conferences on Artificial Intelligence Organization, 2022.
  16. Casia-surf cefa: A benchmark for multi-modal cross-ethnicity face anti-spoofing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1179–1187, 2021a.
  17. Face anti-spoofing via adversarial cross-modality translation. IEEE Transactions on Information Forensics and Security, 16:2759–2772, 2021b.
  18. 3d high-fidelity mask face presentation attack detection challenge. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 814–823, 2021c.
  19. Disentangling facial pose and appearance information for face anti-spoofing. In 2022 26th International Conference on Pattern Recognition (ICPR), pages 4537–4543. IEEE, 2022a.
  20. Contrastive context-aware learning for 3d high-fidelity mask face presentation attack detection. IEEE Transactions on Information Forensics and Security, 17:2497–2507, 2022b.
  21. Attack-agnostic deep face anti-spoofing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 6335–6344, 2023a.
  22. Fm-vit: Flexible modal vision transformers for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 18:4775–4786, 2023b.
  23. Unified frequency-assisted transformer framework for detecting and grounding multi-modal manipulation. arXiv preprint arXiv:2309.09667, 2023c.
  24. Forgery-aware adaptive transformer for generalizable synthetic image detection. arXiv preprint arXiv:2312.16649, 2023d.
  25. Padvg: A simple baseline of active protection for audio-driven video generation. ACM Transactions on Multimedia Computing, Communications and Applications, 20(6), 2024a.
  26. A 3d mask face anti-spoofing database with real world variations. In CVPRW, 2016.
  27. Adaptive normalized representation learning for generalizable face anti-spoofing. In Proceedings of the 29th ACM International Conference on Multimedia, pages 1469–1477, 2021d.
  28. Dual reweighting domain generalization for face presentation attack detection. arXiv preprint arXiv:2106.16128, 2021e.
  29. Spoof trace disentanglement for generic face anti-spoofing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3813–3830, 2022.
  30. Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In CVPR, 2018.
  31. Deep tree learning for zero-shot face anti-spoofing. In CVPR, 2019.
  32. On disentangling spoof trace for generic face anti-spoofing. In ECCV, pages 406–422. Springer, 2020.
  33. Source-free domain adaptation with contrastive domain alignment and self-supervised exploration for face anti-spoofing. In ECCV, 2022c.
  34. Causal intervention for generalizable face anti-spoofing. In ICME, 2022d.
  35. Towards unsupervised domain generalization for face anti-spoofing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023e.
  36. Source-free domain adaptation with domain generalized pretraining for face anti-spoofing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024b.
  37. The high-quality wide multi-channel attack (hq-wmca) database, 2020.
  38. Meta-teacher for face anti-spoofing. IEEE transactions on pattern analysis and machine intelligence, 2021.
  39. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  40. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In CVPR, 2019.
  41. Regularized fine-grained meta face anti-spoofing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11974–11981, 2020.
  42. Flip: Cross-domain face anti-spoofing with language guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19685–19696, 2023.
  43. Rethinking domain generalization for face anti-spoofing: Separability and alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24563–24574, 2023.
  44. Compound text-guided prompt tuning via image-adaptive cues. arXiv preprint arXiv:2312.06401, 2023.
  45. Patchnet: A simple face anti-spoofing framework via fine-grained patch recognition. pages 20281–20290, 2022a.
  46. Improving cross-database face presentation attack detection via adversarial domain adaptation. In 2019 International Conference on Biometrics (ICB), pages 1–8. IEEE, 2019.
  47. Unsupervised adversarial domain adaptation for cross-domain face presentation attack detection. TIFS, 2020a.
  48. Self-domain adaptation for face anti-spoofing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2746–2754, 2021.
  49. Deep spatial gradient and temporal depth learning for face anti-spoofing. In CVPR, 2020b.
  50. Domain generalization via shuffled style assembly for face anti-spoofing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4123–4133, 2022b.
  51. Face spoof detection with image distortion analysis. IEEE TIFS, 2015.
  52. Nas-fas: Static-dynamic central difference network search for face anti-spoofing. In TPAMI, 2020a.
  53. Searching central difference convolutional networks for face anti-spoofing. In CVPR, 2020b.
  54. Flexible-modal face anti-spoofing: A benchmark, 2023.
  55. Face anti-spoofing via disentangled representation learning. In ECCV, 2020a.
  56. Casia-surf: A large-scale multi-modal benchmark for face anti-spoofing. TBMIO, 2(2):182–193, 2020b.
  57. Celeba-spoof: Large-scale face anti-spoofing dataset with rich annotations. In ECCV, 2020c.
  58. A face antispoofing database with diverse attacks. In ICB, 2012.
  59. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008, 2021.
  60. Conditional prompt learning for vision-language models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022a.
  61. Learning to prompt for vision-language models. International Journal of Computer Vision (IJCV), 2022b.
  62. Adaptive mixture of experts learning for generalizable face anti-spoofing. In Proceedings of the 30th ACM International Conference on Multimedia, pages 6009–6018, 2022c.
  63. Instance-aware domain generalization for face anti-spoofing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20453–20463, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Ajian Liu (31 papers)
  2. Shuai Xue (1 paper)
  3. Jianwen Gan (4 papers)
  4. Jun Wan (79 papers)
  5. Yanyan Liang (29 papers)
  6. Jiankang Deng (96 papers)
  7. Sergio Escalera (127 papers)
  8. Zhen Lei (205 papers)
Citations (14)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets