Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Learning-Based Quasi-Conformal Surface Registration for Partial 3D Faces Applied to Facial Recognition (2405.09880v1)

Published 16 May 2024 in cs.CV

Abstract: 3D face registration is an important process in which a 3D face model is aligned and mapped to a template face. However, the task of 3D face registration becomes particularly challenging when dealing with partial face data, where only limited facial information is available. To address this challenge, this paper presents a novel deep learning-based approach that combines quasi-conformal geometry with deep neural networks for partial face registration. The proposed framework begins with a Landmark Detection Network that utilizes curvature information to detect the presence of facial features and estimate their corresponding coordinates. These facial landmark features serve as essential guidance for the registration process. To establish a dense correspondence between the partial face and the template surface, a registration network based on quasiconformal theories is employed. The registration network establishes a bijective quasiconformal surface mapping aligning corresponding partial faces based on detected landmarks and curvature values. It consists of the Coefficients Prediction Network, which outputs the optimal Beltrami coefficient representing the surface mapping. The Beltrami coefficient quantifies the local geometric distortion of the mapping. By controlling the magnitude of the Beltrami coefficient through a suitable activation function, the bijectivity and geometric distortion of the mapping can be controlled. The Beltrami coefficient is then fed into the Beltrami solver network to reconstruct the corresponding mapping. The surface registration enables the acquisition of corresponding regions and the establishment of point-wise correspondence between different partial faces, facilitating precise shape comparison through the evaluation of point-wise geometric differences at these corresponding regions. Experimental results demonstrate the effectiveness of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Optimal step nonrigid icp algorithms for surface registration. In 2007 IEEE conference on computer vision and pattern recognition, pages 1–8. IEEE, 2007.
  2. Elliptic partial differential equations and quasiconformal mappings in the plane (pms-48). In Elliptic Partial Differential Equations and Quasiconformal Mappings in the Plane (PMS-48). Princeton University Press, 2008.
  3. On partial shape correspondence and functional maps. arXiv preprint arXiv:2310.14692, 2023.
  4. Review of statistical shape spaces for 3d data with comparative analysis for human faces. Computer Vision and Image Understanding, 128:1–17, 2014.
  5. Robust face landmark estimation under occlusion. In Proceedings of the IEEE international conference on computer vision, pages 1513–1520, 2013.
  6. Unsupervised learning of robust spectral shape matching. arXiv preprint arXiv:2304.14419, 2023.
  7. Quasi-conformal geometry based local deformation analysis of lateral cephalogram for childhood osa classification. arXiv preprint arXiv:2006.11408, 2020.
  8. A deep learning framework for diffeomorphic mapping problems via quasi-conformal geometry applied to imaging. arXiv preprint arXiv:2110.10580, 2021.
  9. Fernando De la Torre and Jeffrey F Cohn. Facial expression analysis. Visual analysis of humans: Looking at people, pages 377–409, 2011.
  10. Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0, 2019.
  11. Intrinsic parameterizations of surface meshes. Comput. Graph. Forum, 21(3):209–218, 2002.
  12. Supervision by registration and triangulation for landmark detection. 2020. doi:10.1109/TPAMI.2020.2983935.
  13. Supervision-by-Registration: An unsupervised approach to improve the precision of facial landmark detectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 360–368, 2018.
  14. Wing loss for robust facial landmark localisation with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2235–2245, 2018.
  15. Surface parameterization: a tutorial and survey. Advances in Multiresolution for Geometric Modelling, pages 157–186, 2005.
  16. Quasiconformal teichmuller theory. Number 76. American Mathematical Soc., 2000.
  17. Pfld: A practical facial landmark detector. arXiv preprint arXiv:1902.10859, 2019.
  18. Automatic landmark detection and registration of brain cortical surfaces via quasi-conformal geometry and convolutional neural networks. arXiv preprint arXiv:2208.07010, 2022.
  19. Quasiconformal maps in metric spaces with controlled geometry. Acta Mathematica, 181(1):1–61, 1998.
  20. Face super-resolution guided by 3d facial priors. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16, pages 763–780. Springer, 2020.
  21. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007.
  22. Fast facial landmark detection and applications: A survey. Journal of Computer Science and Technology, 22(1):e02–e02, 2022.
  23. Deep alignment network: A convolutional neural network for robust face alignment. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 88–97, 2017.
  24. Avatarme: Realistically renderable 3d facial reconstruction” in-the-wild”. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 760–769, 2020.
  25. Quasiconformal mappings in the plane, volume 126. Citeseer, 1973.
  26. Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph., 21(3):362–371, 2002.
  27. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
  28. Locating facial landmarks using probabilistic random forest. IEEE Signal Processing Letters, 22(12):2324–2328, 2015.
  29. Spectral shape recovery and analysis via data-driven connections. International journal of computer vision, 129:2745–2760, 2021.
  30. Robust learning from normals for 3d face recognition. In Computer Vision–ECCV 2012. Workshops and Demonstrations: Florence, Italy, October 7-13, 2012, Proceedings, Part II 12, pages 230–239. Springer, 2012.
  31. Magface: A universal representation for face recognition and quality assessment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14225–14234, 2021.
  32. José Augusto Cadena Moreano and NBLS Palomino. Global facial recognition using gabor wavelet, support vector machines and 3d face models. Journal of Advances in Information Technology, 11(3), 2020.
  33. Spectral conformal parameterization. Comput. Graph. Forum, 27(5):1487–1494, 2008.
  34. Computing quasi-conformal folds. SIAM Journal on Imaging Sciences, 12(3):1392–1424, 2019.
  35. Di Qiu and Lok Ming Lui. Inconsistent surface registration via optimization of mapping distortions. Journal of Scientific Computing, 83:1–31, 2020.
  36. Automatic analysis of facial affect: A survey of registration, representation, and recognition. IEEE transactions on pattern analysis and machine intelligence, 37(6):1113–1133, 2014.
  37. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
  38. Mesh parameterization methods and their applications. Found. Trends Comput. Graph. Vis., 2(2):105–171, 2007.
  39. Deepid3: Face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873, 2015.
  40. Deeply learned face representations are sparse, selective, and robust. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2892–2900, 2015.
  41. Face recognition: Past, present and future (a review). Digital Signal Processing, 106:102809, 2020.
  42. Nonlinear 3d face morphable model. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7346–7355, 2018.
  43. Detector of facial landmarks learned by the structured output svm. VIsAPP, 12:547–556, 2012.
  44. Adaptive wing loss for robust face alignment via heatmap regression. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6971–6981, 2019.
  45. Facial landmark detection: A literature survey. International Journal of Computer Vision, 127(2):115–143, 2019.
  46. i3dmm: Deep implicit 3d morphable model of human heads. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12803–12813, 2021.
  47. An efficient energy minimization for conformal parameterizations. J. Sci. Comput., 73(1):203–227, 2017.
  48. Surface registration by optimization in constrained diffeomorphism space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4169–4176, 2014.
  49. Facial landmark detection by deep multi-task learning. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pages 94–108. Springer, 2014.
  50. Learning deep representation for face alignment with auxiliary attributes. IEEE transactions on pattern analysis and machine intelligence, 38(5):918–930, 2015.
  51. Mobilefan: Transferring deep hidden representation for face alignment. Pattern Recognition, 100:107114, 2020.
  52. Parallelizable global quasi-conformal parameterization of multiply connected surfaces via partial welding. SIAM Journal on Imaging Sciences, 15(4):1765–1807, 2022.
  53. Learning robust facial landmark detection via hierarchical structured ensemble. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 141–150, 2019.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets