Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Lightweight Facial Attractiveness Prediction Using Dual Label Distribution (2212.01742v2)

Published 4 Dec 2022 in cs.CV

Abstract: Facial attractiveness prediction (FAP) aims to assess facial attractiveness automatically based on human aesthetic perception. Previous methods using deep convolutional neural networks have improved the performance, but their large-scale models have led to a deficiency in flexibility. In addition, most methods fail to take full advantage of the dataset. In this paper, we present a novel end-to-end FAP approach that integrates dual label distribution and lightweight design. The manual ratings, attractiveness score, and standard deviation are aggregated explicitly to construct a dual-label distribution to make the best use of the dataset, including the attractiveness distribution and the rating distribution. Such distributions, as well as the attractiveness score, are optimized under a joint learning framework based on the label distribution learning (LDL) paradigm. The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone. Extensive experiments are conducted on two benchmark datasets, where our approach achieves promising results and succeeds in balancing performance and efficiency. Ablation studies demonstrate that our delicately designed learning modules are indispensable and correlated. Additionally, the visualization indicates that our approach can perceive facial attractiveness and capture attractive facial regions to facilitate semantic predictions. The code is available at https://github.com/enquan/2D_FAP.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. M. Bashour, “History and current concepts in the analysis of facial attractiveness,” Plastic and Reconstructive Surgery, vol. 118, no. 3, pp. 741–756, 2006.
  2. M. Ibáñez-Berganza, A. Amico, and V. Loreto, “Subjectivity and complexity of facial attractiveness,” Scientific Reports, vol. 9, no. 1, p. 8364, 2019.
  3. S. Liu, Y.-Y. Fan, A. Samal et al., “Advances in computational facial attractiveness methods,” Multimedia Tools and Applications, vol. 75, no. 23, pp. 16 633–16 663, 2016.
  4. F. Chen, X. Xiao, and D. Zhang, “Data-driven facial beauty analysis: Prediction, retrieval and manipulation,” IEEE Transactions on Affective Computing, vol. 9, no. 2, pp. 205–216, 2018.
  5. X. Ning, S. Xu, F. Nan, Q. Zeng, C. Wang, W. Cai, W. Li, and Y. Jiang, “Face editing based on facial recognition features,” IEEE Transactions on Cognitive and Developmental Systems, vol. 15, no. 2, pp. 774–783, 2023.
  6. R. Rothe, R. Timofte, and L. Van Gool, “Some like it hot-visual guidance for preference prediction,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5553–5561.
  7. F. S. Abousaleh, W.-H. Cheng, N.-H. Yu, and Y. Tsao, “Multimodal deep learning framework for image popularity prediction on social media,” IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 3, pp. 679–692, 2021.
  8. A. Bottino, M. De Simone, A. Laurentini et al., “A new 3-d tool for planning plastic surgery,” IEEE Transactions on Biomedical Engineering, vol. 59, no. 12, pp. 3439–3449, 2012.
  9. P. Aarabi, D. Hughes, K. Mohajer et al., “The automatic measurement of facial beauty,” in IEEE International Conference on Systems, Man and Cybernetics, vol. 4, 2001, pp. 2644–2647.
  10. F. Chen and D. Zhang, “Combining a causal effect criterion for evaluation of facial attractiveness models,” Neurocomputing, vol. 177, pp. 98–109, 2016.
  11. A. Kagian, G. Dror, T. Leyvand et al., “A humanlike predictor of facial attractiveness,” in International Conference on Neural Information Processing Systems, 2006, pp. 649–656.
  12. D. Gray, K. Yu, W. Xu et al., “Predicting facial beauty without landmarks,” in European Conference on Computer Vision, 2010, pp. 434–447.
  13. L. Lin, L. Liang, and L. Jin, “Regression guided by relative ranking using convolutional neural network (r3cnn) for facial beauty prediction,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 122–134, 2022.
  14. L. Lin, L. Liang, L. Jin et al., “Attribute-aware convolutional neural networks for facial beauty prediction.” in International Joint Conference on Artificial Intelligence, 2019, pp. 847–853.
  15. S. Shi, F. Gao, X. Meng et al., “Improving facial attractiveness prediction via co-attention learning,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 4045–4049.
  16. Y.-Y. Fan, S. Liu, B. Li et al., “Label distribution-based facial attractiveness computation by deep residual learning,” IEEE Transactions on Multimedia, vol. 20, no. 8, pp. 2196–2208, 2018.
  17. X. Geng, “Label distribution learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 7, pp. 1734–1748, 2016.
  18. B.-B. Gao, H.-Y. Zhou, J. Wu et al., “Age estimation using expectation of label distribution learning,” in International Joint Conference on Artificial Intelligence, 2018, pp. 712–718.
  19. Y. Zhou, H. Xue, and X. Geng, “Emotion distribution recognition from facial expressions,” in ACM International Conference on Multimedia, 2015, pp. 1247–1250.
  20. K. Su and X. Geng, “Soft facial landmark detection by label distribution learning,” in AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 5008–5015.
  21. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural information processing systems, vol. 25, 2012.
  22. O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference, 2015.
  23. D. Xie, L. Liang, L. Jin et al., “Scut-fbp: A benchmark dataset for facial beauty perception,” in IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 1821–1826.
  24. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  25. L. Xu, J. Xiang, and X. Yuan, “Transferring rich deep features for facial beauty prediction,” arXiv preprint arXiv:1803.07253, 2018.
  26. J. Xu, L. Jin, L. Liang, Z. Feng, D. Xie, and H. Mao, “Facial attractiveness prediction using psychologically inspired convolutional neural network (pi-cnn),” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2017, pp. 1657–1661.
  27. F. Bougourzi, F. Dornaika, and A. Taleb-Ahmed, “Deep learning based face beauty prediction via dynamic robust losses and ensemble regression,” Knowledge-Based Systems, vol. 242, p. 108246, 2022.
  28. L. Xu, H. Fan, and J. Xiang, “Hierarchical multi-task network for race, gender and facial attractiveness recognition,” in IEEE International Conference on Image Processing, 2019, pp. 3861–3865.
  29. J. Xu, “Mt-resnet: a multi-task deep network for facial attractiveness prediction,” in IEEE 2nd International Conference on Computing and Data Science, 2021, pp. 44–48.
  30. M. Sandler, A. Howard, M. Zhu et al., “Mobilenetv2: Inverted residuals and linear bottlenecks,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520.
  31. F. N. Iandola, S. Han, M. W. Moskewicz et al., “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <<<0.5 mb model size,” arXiv preprint arXiv:1602.07360, 2016.
  32. M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International Conference on Machine Learning, 2019, pp. 6105–6114.
  33. A. G. Howard, M. Zhu, B. Chen et al., “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017.
  34. A. Howard, M. Sandler, G. Chu et al., “Searching for mobilenetv3,” in IEEE/CVF International Conference on Computer Vision, 2019, pp. 1314–1324.
  35. D. Zhou, Q. Hou, Y. Chen, J. Feng, and S. Yan, “Rethinking bottleneck structure for efficient mobile network design,” in European Conference on Computer Vision, 2020, pp. 680–697.
  36. Q. Xie, M.-T. Luong, E. Hovy et al., “Self-training with noisy student improves imagenet classification,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10 687–10 698.
  37. M. Tan, R. Pang, and Q. V. Le, “Efficientdet: Scalable and efficient object detection,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 10 781–10 790.
  38. Y.-F. Song, Z. Zhang, C. Shan, and L. Wang, “Constructing stronger and faster baselines for skeleton-based action recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  39. X. Sun, C. Chen, X. Wang, J. Dong, H. Zhou, and S. Chen, “Gaussian dynamic convolution for efficient single-image segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 2937–2948, 2022.
  40. Y. Ren and X. Geng, “Sense beauty by label distribution learning,” in International Joint Conference on Artificial Intelligence, 2017, pp. 2648–2654.
  41. L. Chen and W. Deng, “Facial attractiveness prediction by deep adaptive label distribution learning,” in Chinese Conference on Biometric Recognition, 2019, pp. 198–206.
  42. B.-B. Gao, X.-X. Liu, H.-Y. Zhou, J. Wu, and X. Geng, “Learning expectation of label distribution for facial age and attractiveness estimation,” arXiv preprint arXiv:2007.01771v2, 2021.
  43. A. Paszke, S. Gross, F. Massa et al., “Pytorch: An imperative style, high-performance deep learning library,” in International Conference on Neural Information Processing Systems, 2019, pp. 8026–8037.
  44. L. Liang, L. Lin, L. Jin et al., “Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction,” in International Conference on Pattern Recognition, 2018, pp. 1598–1603.
  45. K. Zhang, Z. Zhang, Z. Li et al., “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016.
  46. J. Deng, W. Dong, R. Socher et al., “Imagenet: A large-scale hierarchical image database,” in IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
  47. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019.
  48. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2921–2929.
  49. H. J. Richards, V. Benson, N. Donnelly, and J. A. Hadwin, “Exploring the function of selective attention and hypervigilance for threat in anxiety,” Clinical Psychology Review, vol. 34, no. 1, pp. 1–13, 2014.
  50. L. Zhu, H. Zhou, X. Wang, X. Ma, and Q. Liu, “Preference for ugly faces? —A cognitive study of attentional and memorial biases toward facial information among young females with facial dissatisfaction,” Frontiers in Psychology, vol. 13, p. 1024197, 2022.
  51. H. Kou, Y. Su, T. Bi, X. Gao, and H. Chen, “Attentional biases toward face-related stimuli among face dissatisfied women: Orienting and maintenance of attention revealed by eye-movement,” Frontiers in Psychology, vol. 7, p. 919, 2016.
  52. A. J. Rubenstein, “Variation in perceived attractiveness: Differences between dynamic and static faces,” Psychological science, vol. 16, no. 10, pp. 759–762, 2005.
  53. A. J. O’Toole, D. A. Roark, and H. Abdi, “Recognizing moving faces: A psychological and neural synthesis,” Trends in cognitive sciences, vol. 6, no. 6, pp. 261–266, 2002.
  54. S. Kalayci, H. K. Ekenel, and H. Gunes, “Automatic analysis of facial attractiveness from video,” in IEEE International Conference on Image Processing, 2014, pp. 4191–4195.
  55. N. Weng, J. Wang, A. Li, and Y. Wang, “Two-stream temporal convolutional network for dynamic facial attractiveness prediction,” in International Conference on Pattern Recognition, 2021, pp. 10 026–10 033.
  56. Q. Liao, X. Jin, and W. Zeng, “Enhancing the symmetry and proportion of 3d face geometry,” IEEE transactions on visualization and computer graphics, vol. 18, no. 10, pp. 1704–1716, 2012.
  57. S. Liu, Y.-Y. Fan, Z. Guo, A. Samal, and A. Ali, “A landmark-based data-driven approach on 2.5 d facial attractiveness computation,” Neurocomputing, vol. 238, pp. 168–178, 2017.
  58. Q. Xiao, Y. Wu, D. Wang, Y.-L. Yang, and X. Jin, “Beauty3dfacenet: deep geometry and texture fusion for 3d facial attractiveness prediction,” Computers & Graphics, vol. 98, pp. 11–18, 2021.
  59. S. Liu, E. Huang, Y. Xu, K. Wang, and D. K. Jain, “Computation of facial attractiveness from 3d geometry,” Soft Computing, vol. 26, no. 19, pp. 10 401–10 407, 2022.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now