FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction (2211.13874v2)

Abstract: We present a large-scale facial UV-texture dataset that contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions, which are desired characteristics for rendering realistic 3D face models under different lighting conditions. The dataset is derived from a large-scale face image dataset namely FFHQ, with the help of our fully automatic and robust UV-texture production pipeline. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image inputs. An elaborated UV-texture extraction, correction, and completion procedure is then applied to produce high-quality UV-maps from the normalized face images. Compared with existing UV-texture datasets, our dataset has more diverse and higher-quality texture maps. We further train a GAN-based texture decoder as the nonlinear texture basis for parametric fitting based 3D face reconstruction. Experiments show that our method improves the reconstruction accuracy over state-of-the-art approaches, and more importantly, produces high-quality texture maps that are ready for realistic renderings. The dataset, code, and pre-trained texture decoder are publicly available at https://github.com/csbhr/FFHQ-UV.

• The paper introduces a fully automatic pipeline to extract over 50K normalized facial UV textures that improve 3D face reconstruction.
• It employs StyleGAN-based editing, precise UV-texture extraction from 3D shape estimations, and meticulous texture correction for high fidelity.
• Experiments demonstrate enhanced identity preservation and uniform illumination, boosting applications in VR, gaming, and biometric recognition.

FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction

The paper introduces FFHQ-UV, a comprehensive dataset designed to enhance the realism and fidelity of 3D face reconstruction. The dataset contains over 50,000 high-quality facial texture UV-maps, meticulously derived from the existing FFHQ dataset. FFHQ-UV maintains the original data diversity while offering enhanced texture quality crucial for realistic digital renderings under various lighting conditions.

Contributions and Methodology

The primary contribution of the paper is the development of a fully automatic pipeline that generates normalized facial UV-textures from large-scale, in-the-wild face image datasets. The authors employ three significant stages in their pipeline:

1. StyleGAN-Based Image Editing: Utilizing recent advancements in StyleGAN-based facial image editing, the authors generate multi-view normalized face images. This step ensures that facial attributes such as illumination, expression, and occlusions are standardized to facilitate the extraction of quality textures.
2. UV-Texture Extraction: From these preprocessed images, the pipeline extracts UV-textures using 3D face shape estimations. The process mitigates potential artifacts through a meticulous blending strategy, ensuring that only the most accurate textures are included.
3. Texture Correction and Completion: Any imperfections, particularly around complex facial regions like the eyes and mouth, are corrected using precise masking techniques and template UV-map insertions. This step is vital for maintaining the integrity of the facial representation.

The paper details quantitative analyses showcasing FFHQ-UV’s superiority in terms of identity diversity and illumination quality compared to current datasets.

Results and Implications

The introduction of a GAN-based texture decoder trained on FFHQ-UV demonstrates substantial improvements in both texture fidelity and the accuracy of 3D facial reconstructions. Experiments against state-of-the-art texture recovery methods reveal that FFHQ-UV enables more expressive, realistic reconstructions.

Notably, the authors’ evaluations include:

• Identity Preservation: FFHQ-UV inherits the identity variability of the FFHQ dataset, achieving a high degree of fidelity in reconstructed 3D models.
• Even Illumination Quality: The proposed texture maps exhibit uniform illumination characteristics, aligning closely with datasets captured under controlled conditions.

The practical applications of FFHQ-UV extend to enhancing digital human renditions in diverse scenarios, given its public availability. This accessibility is positioned to propel advancements in related fields such as virtual reality, gaming, and biometric recognition.

Future Directions

The authors acknowledge limitations, particularly the dataset's reliance on FFHQ, potentially mirroring its biases. Future work could explore integrating other datasets, although the inherent dependency on StyleGAN’s latent space remains a challenge. Additionally, developing quantitative metrics for texture quality evaluation remains an open area of exploration.

In summary, FFHQ-UV stands as a pivotal contribution, offering a public resource that addresses key challenges in realistic 3D face reconstruction. The insights and methodologies presented in this paper lay foundational work for future research and practical advancements in the domain of computer vision and graphics.