A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Abstract: Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.

• The paper provides a comprehensive survey outlining deep face restoration techniques that leverage facial priors and advanced network architectures.
• It categorizes methods into geometric, reference, generative, and non-prior approaches, emphasizing their unique contributions and performance metrics.
• The survey identifies key challenges in real-world image degradation and suggests future directions for lightweight and video-based restoration research.

Survey of Deep Face Restoration Techniques

The paper "A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal" presents a comprehensive overview of recent advancements in face restoration using deep learning techniques. Face restoration encompasses the enhancement of low-quality facial images by addressing specific degradation factors, such as noise, blur, low resolution, and artifacts.

Key Contributions

This paper offers a structured analysis of face restoration methods, identifying various challenges and classifying existing approaches into categories based on the type of prior information utilized. It also highlights the distinction between traditional statistical models and modern deep learning approaches, emphasizing the latter's superior performance in real-world applications.

Methodological Insights

The authors categorize face restoration techniques into four primary groups:

1. Geometric Prior Based Methods: These methods exploit facial structures like landmarks and parsing maps to enhance restoration accuracy. Representative works include Super-FAN, which integrates face alignment sub-networks, and MTUN, which leverages facial component heatmaps.
2. Reference Prior Based Methods: These approaches use additional high-quality facial images as references to guide restoration. Notable examples include GFRNet and DFDNet, which employ component dictionaries to transfer high-quality details.
3. Generative Prior Based Methods: Leveraging pre-trained face GAN models, these methods, such as GFP-GAN, utilize generative priors to improve restoration quality while maintaining identity consistency.
4. Non-prior Based Methods: Focused on developing powerful architectures like GANs and Transformers, these methods aim to directly learn the restoration mapping. Examples include the attention-based SISN and transformer-based RestoreFormer.

Experimental Evaluation

The paper provides a detailed benchmark evaluation of state-of-the-art methods across multiple datasets, measuring metrics such as PSNR, SSIM, and LPIPS. Transformer-based models like STUNet have shown superior performance, while GAN-based approaches excel in generating visually pleasing results.

Challenges and Future Directions

Despite significant progress, several challenges remain. The ill-posed nature of real-world degradation, the need for effective utilization of facial priors, and the lack of standardized benchmark datasets are critical concerns. The authors suggest future research directions, including the development of lightweight models, integration of novel priors, creation of comprehensive datasets, and exploration of video-based restoration tasks.

Conclusion

This survey paper is pivotal for researchers, offering a thorough understanding of the current landscape in deep face restoration and guiding future explorations. It underscores the importance of leveraging facial priors and strong network architectures to navigate the complexities of real-world applications.