Self-Supervised Face Image Restoration with a One-Shot Reference
Abstract: For image restoration, methods leveraging priors from generative models have been proposed and demonstrated a promising capacity to robustly restore photorealistic and high-quality results. However, these methods are susceptible to semantic ambiguity, particularly with images that have obviously correct semantics such as facial images. In this paper, we propose a semantic-aware latent space exploration method for image restoration (SAIR). By explicitly modeling semantics information from a given reference image, SAIR is able to reliably restore severely degraded images not only to high-resolution and highly realistic looks but also to correct semantics. Quantitative and qualitative experiments collectively demonstrate the superior performance of the proposed SAIR. Our code is available at https://github.com/Liamkuo/SAIR.
- “Image super-resolution using very deep residual channel attention networks,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 286–301.
- “Swinir: Image restoration using swin transformer,” arXiv preprint arXiv:2108.10257, 2021.
- “Functional neural networks for parametric image restoration problems,” Advances in Neural Information Processing Systems, vol. 34, pp. 6762–6775, 2021.
- “Data acquisition and preparation for dual-reference deep learning of image super-resolution,” IEEE Transactions on Image Processing, vol. 31, pp. 4393–4404, 2022.
- “Pulse: Self-supervised photo upsampling via latent space exploration of generative models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 2437–2445.
- “Encoding in style: a stylegan encoder for image-to-image translation,” pp. 2287–2296, 2021.
- “Gan prior embedded network for blind face restoration in the wild,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
- “Image super-resolution via iterative refinement,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4713–4726, 2022.
- “Dr2: Diffusion-based robust degradation remover for blind face restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1704–1713.
- “Training generative adversarial networks with limited data,” in Proc. NeurIPS, 2020.
- “The information bottleneck method,” Proceedings of the 37th Allerton Conference on Communication, Control and Computation, vol. 49, 07 2001.
- “Image2stylegan++: How to edit the embedded images?,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8293–8302.
- “Image2stylegan: How to embed images into the stylegan latent space?,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4431–4440.
- “Arcface: Additive angular margin loss for deep face recognition,” in CVPR, 2019.
- “Designing an encoder for stylegan image manipulation,” arXiv preprint arXiv:2102.02766, 2021.
- “Deep multi-modality soft-decoding of very low bit-rate face videos,” Proceedings of the 28th ACM International Conference on Multimedia, Oct 2020.
- Microsoft Azure, “Perceived emotion recognition,” [EB/OL], 2021.
- “Differentiable histogram with hard-binning,” CoRR, vol. abs/2012.06311, 2020.
- “Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation,” 2020.
- “Exemplar guided face image super-resolution without facial landmarks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 0–0.
- “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” 2018.
- “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018.
- “Photo-realistic single image super-resolution using a generative adversarial network,” 2017.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.