Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

StyleRetoucher: Generalized Portrait Image Retouching with GAN Priors (2312.14389v1)

Published 22 Dec 2023 in cs.CV

Abstract: Creating fine-retouched portrait images is tedious and time-consuming even for professional artists. There exist automatic retouching methods, but they either suffer from over-smoothing artifacts or lack generalization ability. To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN's generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details. Harnessing the priors of pretrained StyleGAN, our method shows superior robustness: a). performing stably with fewer training samples and b). generalizing well on the out-domain data. Moreover, by blending the spatial features of the input image and intermediate features of the StyleGAN layers, our method preserves the input characteristics to the largest extent. We further propose a novel blemish-aware feature selection mechanism to effectively identify and remove the skin blemishes, improving the image skin condition. Qualitative and quantitative evaluations validate the great generalization capability of our method. Further experiments show StyleRetoucher's superior performance to the alternative solutions in the image retouching task. We also conduct a user perceptive study to confirm the superior retouching performance of our method over the existing state-of-the-art alternatives.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Image2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE international conference on computer vision, pages 4432–4441, 2019.
  2. Image2stylegan++: How to edit the embedded images? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8296–8305, 2020.
  3. Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows. ACM Transactions on Graphics (ToG), 40(3):1–21, 2021.
  4. Histogan: Controlling colors of gan-generated and real images via color histograms. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7941–7950, 2021.
  5. Carigans: Unpaired photo-to-caricature translation. ACM Transactions on Graphics (Proc. of Siggraph Asia 2018), 2018.
  6. Glean: Generative latent bank for large-factor image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14245–14254, 2021.
  7. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259, 2014.
  8. Jojogan: One shot face stylization. In European Conference on Computer Vision, pages 128–152. Springer, 2022.
  9. Stylegan-nada: Clip-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG), 41(4):1–13, 2022.
  10. Domain transform for edge-aware image and video processing. In ACM SIGGRAPH 2011 papers, pages 1–12. 2011.
  11. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4):1–12, 2017.
  12. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems.
  13. Mutually guided image filtering. In Proceedings of the 25th ACM international conference on Multimedia, pages 1283–1290, 2017.
  14. Ganspace: Discovering interpretable gan controls. Advances in Neural Information Processing Systems, 33:9841–9850, 2020.
  15. Conditional sequential modulation for efficient global image retouching. pages 679–695, 2020.
  16. Progressive color transfer with dense semantic correspondences. ACM Transactions on Graphics (TOG), 38(2):1–18, 2019.
  17. Meitu Inc. Beautycam, 2013. https://meiyan.meipai.com.
  18. Meitu Inc. Meitupic, 2013. http://xiuxiu.meitu.com/.
  19. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019.
  20. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR, 2020.
  21. Abpn: Adaptive blend pyramid network for real-time local retouching of ultra high-resolution photo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2108–2117, 2022.
  22. Exemplar-based freckle retouching and skin tone adjustment. Computers & Graphics, 78:54–63, 2019.
  23. Stgan: A unified selective transfer network for arbitrary image attribute editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3673–3682, 2019.
  24. Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 2437–2445, 2020.
  25. Fast global image smoothing based on weighted least squares. IEEE Transactions on Image Processing, 23(12):5638–5653, 2014.
  26. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3):209–212, 2012.
  27. Styleclip: Text-driven manipulation of stylegan imagery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2085–2094, 2021.
  28. Resolution dependent gan interpolation for controllable image synthesis between domains. arXiv preprint arXiv:2010.05334, 2020.
  29. Encoding in style: a stylegan encoder for image-to-image translation. arXiv preprint arXiv:2008.00951, 2020.
  30. Ola Sevandersson. Pixlr, 2008. https://www.pixlr.com/.
  31. Autoretouch: Automatic professional face retouching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 990–998, January 2021.
  32. Agilegan: stylizing portraits by inversion-consistent transfer learning. ACM Transactions on Graphics (TOG), 40(4):1–13, 2021.
  33. Drawinginstyles: Portrait image generation and editing with spatially conditioned stylegan. IEEE Transactions on Visualization and Computer Graphics, 2022.
  34. Bilateral filtering for gray and color images. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271), pages 839–846. IEEE, 1998.
  35. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
  36. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6849–6857, 2019.
  37. Real-time image enhancer via learnable spatial-aware 3d lookup tables. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2471–2480, 2021.
  38. High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  39. Towards real-world blind face restoration with generative facial prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9168–9178, 2021.
  40. Panini-net: Gan prior based degradation-aware feature interpolation for face restoration. arXiv preprint arXiv:2203.08444, 2022.
  41. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  42. Joint bilateral learning for real-time universal photorealistic style transfer. In European Conference on Computer Vision, pages 327–342. Springer, 2020.
  43. Image smoothing via l 0 gradient minimization. In Proceedings of the 2011 SIGGRAPH Asia conference, pages 1–12, 2011.
  44. Vtoonify: Controllable high-resolution portrait video style transfer. ACM Transactions on Graphics (TOG), 41(6):1–15, 2022.
  45. Gan prior embedded network for blind face restoration in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 672–681, 2021.
  46. Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
  47. Deep exemplar-based video colorization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8052–8061, 2019.
  48. 100+ times faster weighted median filter (wmf). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2830–2837, 2014.
  49. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
  50. In-domain gan inversion for real image editing. arXiv preprint arXiv:2004.00049, 2020.
Citations (1)

Summary

We haven't generated a summary for this paper yet.