WaveFace: Authentic Face Restoration with Efficient Frequency Recovery (2403.12760v1)
Abstract: Although diffusion models are rising as a powerful solution for blind face restoration, they are criticized for two problems: 1) slow training and inference speed, and 2) failure in preserving identity and recovering fine-grained facial details. In this work, we propose WaveFace to solve the problems in the frequency domain, where low- and high-frequency components decomposed by wavelet transformation are considered individually to maximize authenticity as well as efficiency. The diffusion model is applied to recover the low-frequency component only, which presents general information of the original image but 1/16 in size. To preserve the original identity, the generation is conditioned on the low-frequency component of low-quality images at each denoising step. Meanwhile, high-frequency components at multiple decomposition levels are handled by a unified network, which recovers complex facial details in a single step. Evaluations on four benchmark datasets show that: 1) WaveFace outperforms state-of-the-art methods in authenticity, especially in terms of identity preservation, and 2) authentic images are restored with the efficiency 10x faster than existing diffusion model-based BFR methods.
- To learn image super-resolution, use a gan to learn how to do image degradation first. In ECCV, 2018.
- Progressive semantic-aware style transformation for blind face restoration. In CVPR, 2021.
- FSRNet: End-to-end learning face super-resolution with facial priors. In CVPR, 2018.
- Diffusion posterior sampling for general noisy inverse problems. In ICLR, 2023.
- Arcface: Additive angular margin loss for deep face recognition. In CVPR, 2019a.
- The menpo benchmark for multi-pose 2d and 3d facial landmark localisation and tracking. IJCV, 2019b.
- Diffusion models beat gans on image synthesis. In NeurIPS, 2021.
- Taming transformers for high-resolution image synthesis. In CVPR, 2021.
- VQFR: Blind face restoration with vector-quantized dictionary and parallel decoder. In ECCV, 2022.
- GANs trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
- Denoising diffusion probabilistic models. In NeurIPS, 2020.
- Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. Report, 2008.
- Progressive growing of GANs for improved quality, stability, and variation. In ICLR, 2018.
- A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
- Analyzing and improving the image quality of stylegan. In CVPR, 2020.
- Denoising diffusion restoration models. In NeurIPS, 2022.
- Adam: A method for stochastic optimization. In ICLR, 2015.
- Learning warped guidance for blind face restoration. In ECCV, 2018.
- SwinIR: Image restoration using swin transformer. In ICCV, 2021.
- Learning a no-reference quality metric for single-image super-resolution. CVIU, 2017.
- Making a “completely blind” image quality analyzer. SPL, 2012.
- Improved denoising diffusion probabilistic models. In ICML, 2021.
- Wavelet diffusion models are fast and scalable image generators. In CVPR, 2023.
- Diffusion autoencoders: Toward a meaningful and decodable representation. In CVPR, 2022.
- High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
- Image super-resolution via iterative refinement. TPAMI, 2022.
- Deep semantic face deblurring. In CVPR, 2018.
- Denoising diffusion implicit models. arXiv:2010.02502, 2020.
- Attention is all you need. In NeurIPS, 2017.
- A survey of deep face restoration: Denoise, super-resolution, deblur, artifact removal. arXiv:2211.02831, 2022a.
- Towards real-world blind face restoration with generative facial prior. In CVPR, 2021a.
- Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In ICCV Workshops, 2021b.
- Image quality assessment: from error visibility to structural similarity. TIP, 2004.
- Restoreformer: High-quality blind face restoration from undegraded key-value pairs. In CVPR, 2022b.
- DR2: Diffusion-based robust degradation remover for blind face restoration. In CVPR, 2023.
- Wider face: A face detection benchmark. In CVPR, 2016.
- Gan prior embedded network for blind face restoration in the wild. In CVPR, 2021.
- Face super-resolution guided by facial component heatmaps. In ECCV, 2018.
- Difface: Blind face restoration with diffused error contraction. arXiv:2212.06512, 2022.
- Edface-celeb-1 m: Benchmarking face hallucination with a million-scale dataset. TPAMI, 2022.
- The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
- Decoupled multi-task learning with cyclical self-regulation for face parsing. In CVPR, 2022.
- Towards robust blind face restoration with codebook lookup transformer. In NeurIPS, 2022.
- Yunqi Miao (5 papers)
- Jiankang Deng (96 papers)
- Jungong Han (111 papers)