Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Face Identity-Aware Disentanglement in StyleGAN (2309.12033v1)

Published 21 Sep 2023 in cs.CV and cs.LG

Abstract: Conditional GANs are frequently used for manipulating the attributes of face images, such as expression, hairstyle, pose, or age. Even though the state-of-the-art models successfully modify the requested attributes, they simultaneously modify other important characteristics of the image, such as a person's identity. In this paper, we focus on solving this problem by introducing PluGeN4Faces, a plugin to StyleGAN, which explicitly disentangles face attributes from a person's identity. Our key idea is to perform training on images retrieved from movie frames, where a given person appears in various poses and with different attributes. By applying a type of contrastive loss, we encourage the model to group images of the same person in similar regions of latent space. Our experiments demonstrate that the modifications of face attributes performed by PluGeN4Faces are significantly less invasive on the remaining characteristics of the image than in the existing state-of-the-art models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Image2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4432–4441, 2019.
  2. Image2stylegan++: How to edit the embedded images?, 2020.
  3. Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows. ACM Transactions on Graphics (TOG), 40(3):1–21, 2021.
  4. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8188–8197, 2020.
  5. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019.
  6. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014.
  7. Nice: Non-linear independent components estimation, 2015.
  8. High-fidelity and arbitrary face editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16115–16124, 2021.
  9. Ffjord: Free-form continuous dynamics for scalable reversible generative models, 2018.
  10. Ganspace: Discovering interpretable gan controls. arXiv preprint arXiv:2004.02546, 2020.
  11. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  12. Attgan: Facial attribute editing by only changing what you want. IEEE Transactions on Image Processing, 28(11):5464–5478, 2019.
  13. On loss functions for deep neural networks in classification. Schedae Informaticae, 25:49–59, 2016.
  14. Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34:852–863, 2021.
  15. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
  16. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR, 2020.
  17. Semi-supervised learning with deep generative models. arXiv preprint arXiv:1406.5298, 2014.
  18. Learning latent subspaces in variational autoencoders. arXiv preprint arXiv:1812.06190, 2018.
  19. On convergence and stability of gans. arXiv preprint arXiv:1705.07215, 2017.
  20. Fader networks: Manipulating images by sliding attributes. arXiv preprint arXiv:1706.00409, 2017.
  21. Deep regression tracking with shrinkage loss. In Proceedings of the European conference on computer vision (ECCV), pages 353–369, 2018.
  22. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
  23. Disentangling in latent space by harnessing a pretrained generator. arXiv preprint arXiv:2005.07728, 2(3), 2020.
  24. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2337–2346, 2019.
  25. Invertible conditional gans for image editing. arXiv preprint arXiv:1611.06355, 2016.
  26. Interfacegan: Interpreting the disentangled face representation learned by gans. IEEE transactions on pattern analysis and machine intelligence, 2020.
  27. Spherical wards clustering and generalized voronoi diagrams. In 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 1–10. IEEE, 2015.
  28. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28:3483–3491, 2015.
  29. Pie: Portrait image embedding for semantic control. ACM Transactions on Graphics (TOG), 39(6):1–14, 2020.
  30. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
  31. Hijack-gan: Unintended-use of pretrained, black-box gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7872–7881, 2021.
  32. Plugen: Multi-label conditional generation from pre-trained models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8647–8656, 2022.
  33. Attribute2image: Conditional image generation from visual attributes. In European Conference on Computer Vision, pages 776–791. Springer, 2016.
  34. In-domain gan inversion for real image editing, 2020.
  35. Improved stylegan embedding: Where are the good latents? arXiv preprint arXiv:2012.09036, 2020.
  36. Improved stylegan embedding: Where are the good latents?, 2021.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now