UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition (1712.04695v1)

Abstract: Recently proposed robust 3D face alignment methods establish either dense or sparse correspondence between a 3D face model and a 2D facial image. The use of these methods presents new challenges as well as opportunities for facial texture analysis. In particular, by sampling the image using the fitted model, a facial UV can be created. Unfortunately, due to self-occlusion, such a UV map is always incomplete. In this paper, we propose a framework for training Deep Convolutional Neural Network (DCNN) to complete the facial UV map extracted from in-the-wild images. To this end, we first gather complete UV maps by fitting a 3D Morphable Model (3DMM) to various multiview image and video datasets, as well as leveraging on a new 3D dataset with over 3,000 identities. Second, we devise a meticulously designed architecture that combines local and global adversarial DCNNs to learn an identity-preserving facial UV completion model. We demonstrate that by attaching the completed UV to the fitted mesh and generating instances of arbitrary poses, we can increase pose variations for training deep face recognition/verification models, and minimise pose discrepancy during testing, which lead to better performance. Experiments on both controlled and in-the-wild UV datasets prove the effectiveness of our adversarial UV completion model. We achieve state-of-the-art verification accuracy, $94.05\%$, under the CFP frontal-profile protocol only by combining pose augmentation during training and pose discrepancy reduction during testing. We will release the first in-the-wild UV dataset (we refer as WildUV) that comprises of complete facial UV maps from 1,892 identities for research purposes.

• The paper introduces UV-GAN, an adversarial framework that completes occluded facial UV maps to enable robust pose-invariant face recognition.
• The method leverages both local and global discriminators within deep CNNs to significantly improve face verification accuracy, achieving 94.05% accuracy on the CFP protocol.
• This innovation enhances security and identity verification applications while setting the stage for further biometric research and advanced texture synthesis techniques.

Adversarial Facial UV Map Completion for Enhanced Pose-Invariant Face Recognition

The paper "UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition" addresses a significant challenge in computer vision, specifically in the domain of face recognition: the completion of facial UV maps derived from 3D models. The methodology proposed leverages Deep Convolutional Neural Networks (DCNN), augmented by adversarial networks, to achieve facial UV map completion that preserves identity across images and poses. The authors present a robust framework through which face recognition accuracy is improved by overcoming pose variation challenges that typically impair verification performance.

Face recognition systems are predominantly affected by variations in pose, leading to substantial drops in verification accuracy, especially between frontal and profile views. The approach adopted in this paper involves completing facial UV maps, 2D projections of facial texture derived from 3D models, addressing self-occlusion issues that lead to incomplete UV maps. By synthesizing these UV maps, the researchers can train face recognition models that exhibit improved performance irrespective of pose variations.

A major contribution of this research is the development of the UV-GAN framework, which utilizes local and global adversarial networks to synthesize UV maps that are both realistic and identity-preserving, even when the occluded regions constitute significant portions of the UV map. The adversarial networks, consisting of a global discriminator focusing on the entirety of the UV texture and a local discriminator targeting key facial regions, provide a dual mechanism for verifying the authenticity and identity compliance of the generated textures.

The paper reports notable achievements in verification accuracy, such as a

94.05% accuracy in the CFP frontal-profile evaluation protocol. This improvement is attributed to augmenting the training dataset to contain larger pose variations without extensive data labeling. Furthermore, during testing, the paper illustrates the efficacy of synthesizing face images in different poses to reduce pose discrepancy between verification pairs.

The implications of this research are both practical and theoretical. Practically, the method enhances face recognition systems used in security, identity verification, and other applications requiring high reliability against varied poses. Theoretically, it opens avenues for further exploration in adversarial network applications for texture synthesis and identity-preserving generation frameworks, paving the way for more nuanced model-fitting techniques that could extend beyond facial recognition to other biometric modalities.

Future developments might explore integrating additional discriminative features and expanding datasets to further challenge the model under varying environmental and sensor conditions. The release of a large-scale in-the-wild facial UV dataset further contributes to the research community's ability to test and iterate on these findings, fostering advancements in AI-driven recognition technologies.

Overall, the paper provides substantial contributions to the development of pose-invariant face recognition methods through innovative solutions for UV map completion, backed by strong empirical evidence and methodological robustness.