Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Super-Resolution through StyleGAN Regularized Latent Search: A Realism-Fidelity Trade-off (2311.16923v1)

Published 28 Nov 2023 in cs.CV

Abstract: This paper addresses the problem of super-resolution: constructing a highly resolved (HR) image from a low resolved (LR) one. Recent unsupervised approaches search the latent space of a StyleGAN pre-trained on HR images, for the image that best downscales to the input LR image. However, they tend to produce out-of-domain images and fail to accurately reconstruct HR images that are far from the original domain. Our contribution is twofold. Firstly, we introduce a new regularizer to constrain the search in the latent space, ensuring that the inverted code lies in the original image manifold. Secondly, we further enhanced the reconstruction through expanding the image prior around the optimal latent code. Our results show that the proposed approach recovers realistic high-quality images for large magnification factors. Furthermore, for low magnification factors, it can still reconstruct details that the generator could not have produced otherwise. Altogether, our approach achieves a good trade-off between fidelity and realism for the super-resolution task.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (28)
  1. Image2stylegan: How to embed images into the stylegan latent space? In Int. Conf. Comput. Vis., pages 4432–4441, 2019.
  2. Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018.
  3. Compressed sensing using generative models. In Int. Conf. Mach. Learn., pages 537–546. PMLR, 2017.
  4. Minimal achievable sufficient statistic learning. In Int. Conf. Mach. Learn., pages 1465–1474. PMLR, 2019.
  5. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
  6. Deep learning. MIT press, 2016.
  7. Gcfsr: a generative and controllable face super resolution method without facial and gan priors. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1889–1898, 2022.
  8. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inform. Process. Syst., 30, 2017.
  9. Curricularface: adaptive curriculum learning loss for deep face recognition. In IEEE Conf. Comput. Vis. Pattern Recog., pages 5901–5910, 2020.
  10. Image-to-image translation with conditional adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1125–1134, 2017.
  11. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
  12. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4401–4410, 2019.
  13. Analyzing and improving the image quality of stylegan. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8110–8119, 2020.
  14. Denoising diffusion restoration models. In Adv. Neural Inform. Process. Syst., 2022.
  15. Snips: Solving noisy inverse problems stochastically. Advances in Neural Information Processing Systems, 34:21757–21769, 2021.
  16. Normalizing flows: An introduction and review of current methods. IEEE Trans. Pattern Anal. Mach. Intell., 43(11):3964–3979, 2020.
  17. Bayesian image reconstruction using deep generative models. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021.
  18. Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2437–2445, 2020.
  19. Making a “completely blind” image quality analyzer. IEEE Sign. Process. Letters, 20(3):209–212, 2012.
  20. Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. Res., 22(57):1–64, 2021.
  21. Masked autoregressive flow for density estimation. NIPS, 30, 2017.
  22. High-resolution image synthesis and semantic manipulation with conditional gans. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8798–8807, 2018.
  23. Towards real-world blind face restoration with generative facial prior. In IEEE Conf. Comput. Vis. Pattern Recog., pages 9168–9178, 2021.
  24. Esrgan: Enhanced super-resolution generative adversarial networks. In Eur. Conf. Comput. Vis., pages 0–0, 2018.
  25. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems and Computers, 2003, volume 2, pages 1398–1402. IEEE, 2003.
  26. Improving inversion and generation diversity in stylegan using a gaussianized latent space. arXiv preprint arXiv:2009.06529, 2020.
  27. Gan prior embedded network for blind face restoration in the wild. In IEEE Conf. Comput. Vis. Pattern Recog., pages 672–681, 2021.
  28. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE Conf. Comput. Vis. Pattern Recog., pages 586–595, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now