Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing (2402.19298v2)

Published 29 Feb 2024 in cs.CV

Abstract: Face Anti-Spoofing (FAS) is crucial for securing face recognition systems against presentation attacks. With advancements in sensor manufacture and multi-modal learning techniques, many multi-modal FAS approaches have emerged. However, they face challenges in generalizing to unseen attacks and deployment conditions. These challenges arise from (1) modality unreliability, where some modality sensors like depth and infrared undergo significant domain shifts in varying environments, leading to the spread of unreliable information during cross-modal feature fusion, and (2) modality imbalance, where training overly relies on a dominant modality hinders the convergence of others, reducing effectiveness against attack types that are indistinguishable sorely using the dominant modality. To address modality unreliability, we propose the Uncertainty-Guided Cross-Adapter (U-Adapter) to recognize unreliably detected regions within each modality and suppress the impact of unreliable regions on other modalities. For modality imbalance, we propose a Rebalanced Modality Gradient Modulation (ReGrad) strategy to rebalance the convergence speed of all modalities by adaptively adjusting their gradients. Besides, we provide the first large-scale benchmark for evaluating multi-modal FAS performance under domain generalization scenarios. Extensive experiments demonstrate that our method outperforms state-of-the-art methods. Source code and protocols will be released on https://github.com/OMGGGGG/mmdg.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Improving language models by retrieving from trillions of tokens. In Proceedings of the International Conference on Machine Learning, pages 2206–2240, 2022.
  2. Drl-fas: A novel framework based on deep reinforcement learning for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 16:937–951, 2021.
  3. Learning meta pattern for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 17:1201–1213, 2022a.
  4. Learning meta pattern for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 17:1201–1213, 2022b.
  5. Learning meta pattern for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 17:1201–1213, 2022c.
  6. Rehearsal-free domain continual face anti-spoofing: Generalize more and forget less. In Proceedings of the IEEE International Conference on Computer Vision, 2023a.
  7. S-adapter: Generalizing vision transformer for face anti-spoofing with statistical tokens. arXiv, abs/2309.04038, 2023b.
  8. Adapting neural models with sequential monte carlo dropout. In Conference on Robot Learning, pages 1542–1552, 2022.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations, 2021.
  10. Energy-based domain generalization for face anti-spoofing. In Proceedings of the ACM International Conference on Multimedia, pages 1749–1757, 2022.
  11. PMR: prototypical modal rebalance for multimodal learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 20029–20038, 2023.
  12. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the International Conference on Machine Learning, pages 1050–1059, 2016.
  13. Cross modal focal loss for RGBD face anti-spoofing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7882–7891, 2021a.
  14. Learning one class representations for face presentation attack detection using multi-channel convolutional neural networks. IEEE Transactions on Information Forensics and Security, 16:361–375, 2021b.
  15. Biometric face presentation attack detection with multi-channel convolutional neural network. IEEE Transactions on Information Forensics and Security, 15:42–55, 2020.
  16. Uncertainty-guided probabilistic transformer for complex action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 20020–20029, 2022.
  17. Adaptive transformers for robust few-shot cross-domain face anti-spoofing. In Proceedings of the European Conference on Computer Vision, pages 37–54, 2022.
  18. Uncertainty-guided learning for improving image manipulation detection. In Proceedings of the IEEE International Conference on Computer Vision, pages 22456–22465, 2023.
  19. Single-side domain generalization for face anti-spoofing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8481–8490, 2020.
  20. Dual-branch meta-learning network with distribution alignment for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 17:138–151, 2022.
  21. Adversarial learning domain-invariant conditional features for robust face anti-spoofing. International Journal of Computer Vision, 131(7):1680–1703, 2023.
  22. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In Proceedings of the British Machine Vision Conference, 2017.
  23. Segment anything. In Proceedings of the IEEE International Conference on Computer Vision, pages 4015–4026, 2023.
  24. Beyond the pixel world: a novel acoustic-based face anti-spoofing system for smartphones. IEEE Transactions on Information Forensics and Security, 17:3238–3253, 2022.
  25. M3fas: An accurate and robust multimodal mobile face anti-spoofing system. arXiv preprint arXiv:2301.12831, 2023.
  26. Learning polysemantic spoof trace: A multi-modal disentanglement network for face anti-spoofing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1351–1359, 2023a.
  27. Asymmetric modality translation for face presentation attack detection. IEEE Transactions on Multimedia, 25:62–76, 2023b.
  28. Domain invariant vision transformer learning for face anti-spoofing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6087–6096, 2023.
  29. Biomedical image splicing detection using uncertainty-guided refinement, 2023a.
  30. Image manipulation detection by multiple tampering traces and edge artifact enhancement. Pattern Recognition, 133:109026, 2023b.
  31. Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7640–7650, 2023.
  32. Ma-vit: Modality-agnostic vision transformers for face anti-spoofing. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1180–1186, 2022.
  33. CASIA-SURF cefa: A benchmark for multi-modal cross-ethnicity face anti-spoofing. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1178–1186, 2021a.
  34. Face anti-spoofing via adversarial cross-modality translation. IEEE Transactions on Information Forensics and Security, 16:2759–2772, 2021b.
  35. FM-ViT: Flexible modal vision transformers for face anti-spoofing. IEEE Transactions on Information Forensics and Security, 18:4775–4786, 2023a.
  36. Spoof trace disentanglement for generic face anti-spoofing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3813–3830, 2023.
  37. Towards unsupervised domain generalization for face anti-spoofing. In Proceedings of the IEEE International Conference on Computer Vision, pages 20654–20664, 2023b.
  38. Uncertainty propagation for dropout-based bayesian neural networks. Neural Networks, 144:394–406, 2021.
  39. Domain generalization via gradient surgery. In Proceedings of the IEEE International Conference on Computer Vision, pages 6610–6618, 2021.
  40. Domain adaptation in multi-channel autoencoder based features for robust face anti-spoofing. In Proceedings of the International Conference on Biometrics, pages 1–8, 2019.
  41. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8228–8237, 2022.
  42. Meta-teacher for face anti-spoofing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):6311–6326, 2022.
  43. Detection and continual learning of novel face presentation attacks. In Proceedings of the IEEE International Conference on Computer Vision, pages 14831–14840, 2021.
  44. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10023–10031, 2019.
  45. Facebagnet: Bag-of-local-features model for multi-modal face anti-spoofing. In IEEE Conference on Computer Vision and Pattern Recognition Workshop, pages 1611–1616, 2019.
  46. Flip: Cross-domain face anti-spoofing with language guidance. In Proceedings of the IEEE International Conference on Computer Vision, pages 19685–19696, 2023.
  47. Rethinking domain generalization for face anti-spoofing: Separability and alignment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 24563–24574, 2023.
  48. Domain generalization via shuffled style assembly for face anti-spoofing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4113–4123, 2022.
  49. Uncertainty-aware physically-guided proxy tasks for unseen domain face anti-spoofing. arXiv, abs/2011.14054, 2020.
  50. pmbqa: Projection-based blind point cloud quality assessment via multimodal learning. In Proceedings of the ACM International Conference on Multimedia, page 3250–3258, 2023.
  51. Uncertainty quantification in intelligent-based electrical resistivity tomography image reconstruction with monte carlo dropout strategy. IEEE Transactions on Geoscience and Remote Sensing, 61:1–16, 2023.
  52. Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  53. Gradient surgery for multi-task learning. In Proceedings of the Neural Information Processing Systems, 2020a.
  54. Towards robust rain removal against adversarial attacks: A comprehensive benchmark analysis and beyond. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6013–6022, 2022.
  55. Backdoor attacks against deep image compression via adaptive frequency trigger. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12250–12259, 2023a.
  56. Multi-modal face anti-spoofing based on central difference networks. In IEEE Conference on Computer Vision and Pattern Recognition Workshop, pages 2766–2774, 2020b.
  57. Searching central difference convolutional networks for face anti-spoofing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5294–5304, 2020c.
  58. Visual prompt flexible-modal face anti-spoofing. arXiv, abs/2307.13958, 2023b.
  59. Rethinking vision transformer and masked autoencoder in multimodal face anti-spoofing. arXiv, abs/2302.05744, 2023c.
  60. Flexible-modal face anti-spoofing: A benchmark. In IEEE Conference on Computer Vision and Pattern Recognition Workshop, pages 6346–6351, 2023d.
  61. Deep learning for face anti-spoofing: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5609–5631, 2023e.
  62. Cyclically disentangled feature translation for face anti-spoofing. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3358–3366, 2023.
  63. Provable dynamic fusion for low-quality multimodal data. In Proceedings of the International Conference on Machine Learning, pages 41753–41769, 2023.
  64. CASIA-SURF: A large-scale multi-modal benchmark for face anti-spoofing. IEEE transactions on biometrics, behavior, and identity science, 2(2):182–193, 2020.
  65. Adaptive mixture of experts learning for generalizable face anti-spoofing. In Proceedings of the ACM International Conference on Multimedia, pages 6009–6018, 2022a.
  66. Generative domain adaptation for face anti-spoofing. In Proceedings of the European Conference on Computer Vision, pages 335–356, 2022b.
  67. Instance-aware domain generalization for face anti-spoofing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 20453–20463, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Xun Lin (25 papers)
  2. Shuai Wang (466 papers)
  3. Rizhao Cai (20 papers)
  4. Yizhong Liu (7 papers)
  5. Ying Fu (98 papers)
  6. Zitong Yu (119 papers)
  7. Wenzhong Tang (4 papers)
  8. Alex Kot (31 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now