Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Network Pruning: A Comparative Study on CNNs in Face Recognition (2405.18302v1)

Published 28 May 2024 in cs.CV

Abstract: The widespread use of mobile devices for all kind of transactions makes necessary reliable and real-time identity authentication, leading to the adoption of face recognition (FR) via the cameras embedded in such devices. Progress of deep Convolutional Neural Networks (CNNs) has provided substantial advances in FR. Nonetheless, the size of state-of-the-art architectures is unsuitable for mobile deployment, since they often encompass hundreds of megabytes and millions of parameters. We address this by studying methods for deep network compression applied to FR. In particular, we apply network pruning based on Taylor scores, where less important filters are removed iteratively. The method is tested on three networks based on the small SqueezeNet (1.24M parameters) and the popular MobileNetv2 (3.5M) and ResNet50 (23.5M) architectures. These have been selected to showcase the method on CNNs with different complexities and sizes. We observe that a substantial percentage of filters can be removed with minimal performance loss. Also, filters with the highest amount of output channels tend to be removed first, suggesting that high-dimensional spaces within popular CNNs are over-dimensionated.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. A. Almadan and A. Rattani. Benchmarking neural network compression techniques for ocular-based user authentication on smartphones. IEEE Access, 11:36550–36565, 2023.
  2. Squeezefaceposenet: Lightweight face verification across different poses for mobile platforms. In Proc. IAPR TC4 Workshop on Mobile and Wearable Biometrics, WMWB, in conjunction with ICPR, 2020.
  3. Squeezerfacenet: Reducing a small face recognition cnn even more via filter pruning. In Proc. International Workshop on Artificial Intelligence and Pattern Recognition, IWAIPR, pages 349–361, 2023.
  4. Mixfacenets: Extremely efficient face recognition networks. In 2021 IEEE International Joint Conference on Biometrics, IJCB, pages 1–8, 2021.
  5. Pocketnet: Extreme lightweight face recognition network using neural architecture search and multistep knowledge distillation. IEEE Access, 10:46823–46833, 2022.
  6. Model compression techniques in biometrics applications: A survey. CoRR, abs/2401.10139, 2024. URL https://arxiv.org/abs/2401.10139.
  7. Vggface2: A dataset for recognising faces across pose and age. In 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG, pages 67–74, 2018.
  8. Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices. CoRR, abs/1804.07573, 2018. URL http://arxiv.org/abs/1804.07573.
  9. Arcface: Additive angular margin loss for deep face recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4685–4694, 2019a.
  10. Retinaface: Single-stage dense face localisation in the wild. CoRR, abs/1905.00641, 2019b. URL http://arxiv.org/abs/1905.00641.
  11. Mobiface: A lightweight deep learning face recognition on mobile devices. In Proc IEEE 10th International Conference on Biometrics Theory, Applications and Systems, BTAS, Sep. 2019.
  12. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In 14th European Conference on Computer Vision, ECCV, pages 87–102, Cham, 2016. ISBN 978-3-319-46487-9.
  13. Deep residual learning for image recognition. In Proc IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 770–778, June 2016.
  14. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size. CoRR, abs/1602.07360, 2016. URL http://arxiv.org/abs/1602.07360.
  15. Prunefacedet: Pruning lightweight face detection network by sparsity training. Cognitive Computation and Systems, 4(4):391–399, 2022.
  16. Do better imagenet models transfer better? In Proc IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2656–2666, 2019.
  17. Discrimination-aware network pruning for deep model compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4035–4051, 2022. doi: 10.1109/TPAMI.2021.3066410.
  18. Shufflefacenet: A lightweight face architecture for efficient and highly-accurate face recognition. In Proc IEEE/CVF International Conference on Computer Vision Workshop, ICCVW, pages 2721–2728, 2019.
  19. Common cnn-based face embedding spaces are (almost) equivalent. CoRR, abs/2010.02323, 2020. URL https://arxiv.org/abs/2010.02323.
  20. Importance estimation for neural network pruning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11256–11264, 2019.
  21. Face space representations in deep convolutional neural networks. Trends in Cognitive Sciences, 22(9):794–809, 2018. ISSN 1364-6613. doi: 10.1016/j.tics.2018.06.006.
  22. Channel-level acceleration of deep face representations. IEEE Access, 3:2163–2175, 2015. doi: 10.1109/ACCESS.2015.2494536.
  23. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, Dec 2015. ISSN 1573-1405.
  24. Mobilenetv2: Inverted residuals and linear bottlenecks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pages 4510–4520, 2018.
  25. K. Sundararajan and D. L. Woodard. Deep learning for biometrics: A survey. ACM Comput. Surv., 51(3), May 2018. ISSN 0360-0300.
  26. M. Tan and Q. V. Le. Mixconv: Mixed depthwise convolutional kernels. In 30th British Machine Vision Conference, BMVC, 2019.
  27. Vargfacenet: An efficient variable group convolutional neural network for lightweight face recognition. In IEEE/CVF International Conference on Computer Vision Workshops, ICCVW), pages 2647–2654, 2019.
  28. Vargnet: Variable group convolutional neural network for efficient embedded computing. CoRR, abs/1907.05653, 2020. URL https://arxiv.org/abs/1907.05653.
  29. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proc IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets