Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ENTED: Enhanced Neural Texture Extraction and Distribution for Reference-based Blind Face Restoration (2401.06978v1)

Published 13 Jan 2024 in cs.CV

Abstract: We present ENTED, a new framework for blind face restoration that aims to restore high-quality and realistic portrait images. Our method involves repairing a single degraded input image using a high-quality reference image. We utilize a texture extraction and distribution framework to transfer high-quality texture features between the degraded input and reference image. However, the StyleGAN-like architecture in our framework requires high-quality latent codes to generate realistic images. The latent code extracted from the degraded input image often contains corrupted features, making it difficult to align the semantic information from the input with the high-quality textures from the reference. To overcome this challenge, we employ two special techniques. The first technique, inspired by vector quantization, replaces corrupted semantic features with high-quality code words. The second technique generates style codes that carry photorealistic texture information from a more informative latent space developed using the high-quality features in the reference image's manifold. Extensive experiments conducted on synthetic and real-world datasets demonstrate that our method produces results with more realistic contextual details and outperforms state-of-the-art methods. A thorough ablation study confirms the effectiveness of each proposed module.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Progressive semantic-aware style transformation for blind face restoration. In CVPR, pages 11896–11905, 2021.
  2. Blind image super resolution with semantic-aware quantized texture prior. arXiv preprint arXiv:2202.13142, 2022.
  3. Fsrnet: End-to-end learning face super-resolution with facial priors. In CVPR, pages 2492–2501, 2018.
  4. Nafssr: Stereo image super-resolution using nafnet. In CVPR, pages 1239–1248, 2022.
  5. Arcface: Additive angular margin loss for deep face recognition. In CVPR, pages 4690–4699, 2019.
  6. Exemplar guided face image super-resolution without facial landmarks. In CVPRW, pages 0–0, 2019.
  7. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873–12883, 2021.
  8. Image processing using multi-code gan prior. In CVPR, pages 3012–3021, 2020.
  9. Vqfr: Blind face restoration with vector-quantized dictionary and parallel decoder. arXiv preprint arXiv:2205.06803, 2022.
  10. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NIPS, 30, 2017.
  11. Robust reference-based super-resolution via c2-matching. In CVPR, pages 2103–2112, 2021.
  12. Perceptual losses for real-time style transfer and super-resolution. In ECCV, pages 694–711. Springer, 2016.
  13. A style-based generator architecture for generative adversarial networks. In CVPR, pages 4401–4410, 2019.
  14. Analyzing and improving the image quality of stylegan. In CVPR, pages 8110–8119, 2020.
  15. Progressive face super-resolution via attention to facial landmark. arXiv preprint arXiv:1908.08239, 2019.
  16. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  17. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017.
  18. Robust training of vector quantized bottleneck models. In IJCNN, pages 1–7. IEEE, 2020.
  19. Blind face restoration via deep multi-scale component dictionaries. In ECCV, pages 399–415. Springer, 2020.
  20. Enhanced blind face restoration with multi-exemplar images and adaptive spatial feature fusion. In CVPR, June 2020.
  21. Learning warped guidance for blind face restoration. In ECCV, September 2018.
  22. Learning dual memory dictionaries for blind face restoration. TPAMI, 2022.
  23. Masa-sr: Matching acceleration and spatial adaptation for reference-based image super-resolution. In CVPR, pages 6368–6377, 2021.
  24. Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In CVPR, pages 2437–2445, 2020.
  25. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3):209–212, 2012.
  26. Neural texture extraction and distribution for controllable person image synthesis. In CVPR, pages 13535–13544, 2022.
  27. Encoding in style: a stylegan encoder for image-to-image translation. In CVPR, pages 2287–2296, 2021.
  28. Deep semantic face deblurring. In CVPR, pages 8260–8269, 2018.
  29. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  30. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
  31. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  32. Bringing old photos back to life. In CVPR, pages 2747–2757, 2020.
  33. Towards real-world blind face restoration with generative facial prior. In CVPR, pages 9168–9178, 2021.
  34. Restoreformer: High-quality blind face restoration from undegraded key-value pairs. In CVPR, pages 17512–17521, 2022.
  35. Feature representation matters: End-to-end learning for reference-based image super-resolution. In ECCV, pages 230–245. Springer, 2020.
  36. Learning texture transformer network for image super-resolution. In CVPR, pages 5791–5800, 2020.
  37. Hifacegan: Face renovation via collaborative suppression and replenishment. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1551–1560, 2020.
  38. Gan prior embedded network for blind face restoration in the wild. In CVPR, pages 672–681, 2021.
  39. Face super-resolution guided by facial component heatmaps. In ECCV, pages 217–233, 2018.
  40. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pages 586–595, 2018.
  41. Image super-resolution by neural texture transfer. In CVPR, pages 7982–7991, 2019.
  42. Rethinking deep face restoration. In CVPR, pages 7652–7661, 2022.
  43. Towards robust blind face restoration with codebook lookup transformer. arXiv preprint arXiv:2206.11253, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now