Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adaptive Hybrid Masking Strategy for Privacy-Preserving Face Recognition Against Model Inversion Attack (2403.10558v2)

Published 14 Mar 2024 in cs.CV, cs.CR, and cs.LG

Abstract: The utilization of personal sensitive data in training face recognition (FR) models poses significant privacy concerns, as adversaries can employ model inversion attacks (MIA) to infer the original training data. Existing defense methods, such as data augmentation and differential privacy, have been employed to mitigate this issue. However, these methods often fail to strike an optimal balance between privacy and accuracy. To address this limitation, this paper introduces an adaptive hybrid masking algorithm against MIA. Specifically, face images are masked in the frequency domain using an adaptive MixUp strategy. Unlike the traditional MixUp algorithm, which is predominantly used for data augmentation, our modified approach incorporates frequency domain mixing. Previous studies have shown that increasing the number of images mixed in MixUp can enhance privacy preservation but at the expense of reduced face recognition accuracy. To overcome this trade-off, we develop an enhanced adaptive MixUp strategy based on reinforcement learning, which enables us to mix a larger number of images while maintaining satisfactory recognition accuracy. To optimize privacy protection, we propose maximizing the reward function (i.e., the loss function of the FR system) during the training of the strategy network. While the loss function of the FR network is minimized in the phase of training the FR network. The strategy network and the face recognition network can be viewed as antagonistic entities in the training process, ultimately reaching a more balanced trade-off. Experimental results demonstrate that our proposed hybrid masking scheme outperforms existing defense algorithms in terms of privacy preservation and recognition accuracy against MIA.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (114)
  1. A survey on homomorphic encryption schemes: Theory and implementation. arXiv, pages 1–35.
  2. A survey on homomorphic encryption schemes: Theory and implementation. ACM Comput. Surv., 51(4).
  3. Fedface: Collaborative learning of face recognition model. In 2021 IEEE International Joint Conference on Biometrics (IJCB), pages 1–8. IEEE.
  4. Quotient: two-party secure neural network training and prediction. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 1231–1247.
  5. Secure Online Payment with Facial Recognition using MTCNN. International Journal of Applied Engineering Research, 15(3):249–252.
  6. Federated face recognition. arXiv preprint arXiv:2105.02501.
  7. Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390.
  8. Boddeti, V. N. (2018). Secure Face Matching Using Fully Homomorphic Encryption. arXiv.
  9. VGGFace2: A dataset for recognising faces across pose and age. Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018, pages 67–74.
  10. An attack on instahide: Is private learning possible with instance encoding?
  11. Privacy Preserving Face Recognition Utilizing Differential Privacy. Computers and Security, 97.
  12. Knowledge-enriched distributional model inversion attacks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16178–16187.
  13. Secure Evaluation of Quantized Neural Networks. (x):1–18.
  14. Revealing and protecting labels in distributed training. Advances in Neural Information Processing Systems, 34.
  15. Spact: Self-supervised privacy preservation for action recognition. arXiv preprint arXiv:2203.15205.
  16. ArcFace: Additive angular margin loss for deep face recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June:4685–4694.
  17. Deep residual learning in the JPEG transform domain. Proceedings of the IEEE International Conference on Computer Vision, 2019-October:3483–3492.
  18. Ergun, O. O. (2015). Privacy preserving face recognition in encrypted domain. IEEE Asia-Pacific Conference on Circuits and Systems, Proceedings, APCCAS, 2015-Febru(February):643–646.
  19. Efficient and Privacy-preserving Online Face Recognition over Encrypted Outsourced Data. 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pages 1349–1354.
  20. Rethinking privacy preserving deep learning: How to evaluate and thwart privacy attacks. In Federated Learning, pages 32–50. Springer.
  21. Robbing the fed: Directly obtaining private data in federated learning with modified models. arXiv preprint arXiv:2110.13057.
  22. Inverting gradients-how easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems, 33:16937–16947.
  23. Shuffled model of differential privacy in federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2521–2529. PMLR.
  24. A Survey on Differentially Private Machine Learning [Review Article]. IEEE Computational Intelligence Magazine, 15(2):49–64.
  25. Gonzalez, T. F. (2007). ImageNet Classification with Deep Convolutional Neural Networks. Handbook of Approximation Algorithms and Metaheuristics, pages 1–1432.
  26. Generative adversarial networks. Communications of the ACM, 63(11):139–144.
  27. Faster neural networks straight from JPEG. 6th International Conference on Learning Representations, ICLR 2018 - Workshop Track Proceedings, (NeurIPS):1–12.
  28. Towards efficient privacy-preserving face recognition in the cloud. Signal Processing, 164:320–328.
  29. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision, pages 87–102. Springer.
  30. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009.
  31. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December:770–778.
  32. Deep models under the gan: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 603–618.
  33. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8):2011–2023.
  34. Evaluating gradient inversion attacks and defenses in federated learning. Advances in Neural Information Processing Systems, 34.
  35. Instahide: Instance-hiding schemes for private distributed learning. In International Conference on Machine Learning, pages 4507–4518. PMLR.
  36. Instahide: Instance-hiding schemes for private distributed learning.
  37. Image-to-image translation with conditional adversarial networks. CVPR.
  38. Gradient inversion with generative image prior. Advances in Neural Information Processing Systems, 34:29898–29908.
  39. Privacy-preserving face recognition with learnable privacy budgets in frequency domain. arXiv preprint arXiv:2207.07316.
  40. A style-based generator architecture for generative adversarial networks.
  41. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR.
  42. Auditing privacy defenses in federated learning via generative gradient leakage. arXiv preprint arXiv:2203.15696.
  43. Fedfr: Joint optimization federated framework for generic and personalized face recognition. arXiv preprint arXiv:2112.12496.
  44. Better accuracy with quantified privacy: representations learned via reconstructive adversarial network.
  45. SphereFace: Deep hypersphere embedding for face recognition. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January:6738–6746.
  46. Fediris: Towards more accurate and privacy-preserving iris recognition via federated template communication. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3357–3366.
  47. Lightweight privacy-preserving ensemble classification for face recognition. IEEE Internet of Things Journal, 6(3):5778–5790.
  48. Block-wise Scrambled Image Recognition Using Adaptation Network. arXiv, (Lowe 1999).
  49. EPIC: Efficient Private Image Classification (or: Learning from the Masters). Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11405 LNCS:473–492.
  50. IARPA janus benchmark-C: Face dataset and protocol. Proceedings - 2018 International Conference on Biometrics, ICB 2018, pages 158–165.
  51. Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (SP), pages 691–706. IEEE.
  52. Improving federated learning face recognition via privacy-agnostic clusters. arXiv preprint arXiv:2201.12467.
  53. Shredder: Learning noise to protect privacy with partial DNN inference on the edge. CoRR, abs/1905.11814.
  54. Poster: Towards characterizing and limiting information exposure in dnn layers. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS ’19, page 2653–2655, New York, NY, USA. Association for Computing Machinery.
  55. Towards characterizing and limiting information exposure in DNN layers. arXiv, (2):2653–2655.
  56. Darknetz: towards model privacy at the edge using trusted execution environments. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pages 161–174.
  57. DarkneTZ: Towards model privacy at the edge using trusted execution environments. MobiSys 2020 - Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pages 161–174.
  58. ABY3: A mixed protocol framework for machine learning. Proceedings of the ACM Conference on Computer and Communications Security, pages 35–52.
  59. SecureML: A System for Scalable Privacy-Preserving Machine Learning. Proceedings - IEEE Symposium on Security and Privacy, pages 19–38.
  60. AgeDB: The First Manually Collected, In-the-Wild Age Database. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2017-July:1997–2005.
  61. Biometric template protection: Bridging the performance gap between theory and practice. IEEE Signal Processing Magazine, 32(5):88–100.
  62. Naresh Boddeti, V. (2018). Secure face matching using fully homomorphic encryption. In 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–10.
  63. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pages 739–753. IEEE.
  64. Federated learning for face recognition with gradient correction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1999–2007.
  65. Theory-oriented deep leakage from gradients via linear equation solver. arXiv preprint arXiv:2010.13356.
  66. Privacy-preserving deep learning: Revisited and enhanced. In International Conference on Applications and Techniques in Information Security, pages 100–110. Springer.
  67. What can we learn from gradients?
  68. Presentation attack detection methods for face recognition systems: A comprehensive survey. ACM Computing Surveys, 50(1).
  69. A method for obtaining digital signatures and public-key cryptosystems. Communications of the ACM, 21(2):120–126.
  70. Mobilenetv2: Inverted residuals and linear bottlenecks.
  71. FaceNet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07-12-June-2015:815–823.
  72. Frontal to profile face verification in the wild. 2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016.
  73. Towards efficient and secure delivery of data for deep learning with privacy-preserving. arXiv.
  74. Pixel-Based Image Encryption without Key Management for Privacy-Preserving Deep Neural Networks. IEEE Access, 7(Ml):177844–177855.
  75. Plug & play attacks: Towards robust and flexible model inversion attacks. arXiv preprint arXiv:2201.12179.
  76. Tanaka, M. (2018). Learnable image encryption. In 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pages 1–2. IEEE.
  77. Compressive privacy generative adversarial network. IEEE Transactions on Information Forensics and Security, 15:2499–2513.
  78. Authentic Face Detection and Encryption for Security Assurance. Proceedings of the 4th International Conference on Applied and Theoretical Computing and Communication Technology, iCATccT 2018, pages 131–135.
  79. SecureNN: 3-Party Secure Computation for Neural Network Training. Proceedings on Privacy Enhancing Technologies, 2019(3):26–49.
  80. FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learning.
  81. F: Honest-majority maliciously secure framework for private deep learning. Proceedings on Privacy Enhancing Technologies, 2021(1):188–208.
  82. User label leakage from gradients in federated learning. arXiv preprint arXiv:2105.09369.
  83. CosFace: Large Margin Cosine Loss for Deep Face Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 5265–5274.
  84. High-frequency component helps explain the generalization of convolutional neural networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8681–8691.
  85. Facemae: Privacy-preserving face recognition via masked autoencoders. arXiv preprint arXiv:2205.11090.
  86. Variational model inversion attacks. Advances in Neural Information Processing Systems, 34:9706–9719.
  87. Mitigating bias in face recognition using skewness-aware reinforcement learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9322–9331.
  88. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Proceedings of the ieee/cvf international conference on computer vision, pages 692–702.
  89. Privacy-preserving face recognition in the frequency domain.
  90. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pages 2512–2520. IEEE.
  91. Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256.
  92. Compressed Video Action Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 6026–6035.
  93. A face based fuzzy vault scheme for secure online authentication. Proceedings - 2nd International Symposium on Data, Privacy, and E-Commerce, ISDPE 2010, pages 45–49.
  94. Complex-valued neural networks for privacy protection. arXiv preprint arXiv:1901.09546.
  95. Complex-valued neural networks for privacy protection. arXiv.
  96. Learning in the frequency domain. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1737–1746.
  97. LAPRAN: A scalable laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11214 LNCS:491–507.
  98. Deep Pyramidal Residual Networks with Separated Stochastic Depth. pages 8–9.
  99. Learning face representation from scratch.
  100. See through gradients: Image batch recovery via gradinversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16337–16346.
  101. How transferable are features in deep neural networks? Advances in Neural Information Processing Systems, 4(January):3320–3328.
  102. Rotation-equivariant neural networks for privacy protection. arXiv preprint arXiv:2006.13016.
  103. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412.
  104. Rotation-equivariant neural networks for privacy protection. arXiv.
  105. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. 2016 International Conference on Biometrics, ICB 2016, pages 1–11.
  106. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595.
  107. Accelerating Very Deep Convolutional Networks for Classification and Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(10):1943–1955.
  108. A Complex-Valued CNN for Different Activation Functions in Polarsar Image Classification. International Geoscience and Remote Sensing Symposium (IGARSS), pages 10023–10026.
  109. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 253–261.
  110. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610.
  111. Local differential privacy-based federated learning for internet of things. IEEE Internet of Things Journal, 8(11):8836–8853.
  112. SQUEEZE-AND-EXCITATION WIDE RESIDUAL NETWORKS IN IMAGE CLASSIFICATION School of Computer Science and Technology , Wuhan University of Technology , Wuhan , P . R . China Hubei Province Key Laboratory of Transportation Internet of Things , Wuhan , P . R . China School of Computer Science and Technology , Wuhan University , Wuhan , P . R . China. pages 395–399.
  113. R-gap: Recursive gradient attack on privacy. arXiv preprint arXiv:2010.07733.
  114. Deep leakage from gradients. Advances in Neural Information Processing Systems, 32.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now