Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LatentSwap: An Efficient Latent Code Mapping Framework for Face Swapping (2402.18351v2)

Published 28 Feb 2024 in cs.CV

Abstract: We propose LatentSwap, a simple face swapping framework generating a face swap latent code of a given generator. Utilizing randomly sampled latent codes, our framework is light and does not require datasets besides employing the pre-trained models, with the training procedure also being fast and straightforward. The loss objective consists of only three terms, and can effectively control the face swap results between source and target images. By attaching a pre-trained GAN inversion model independent to the model and using the StyleGAN2 generator, our model produces photorealistic and high-resolution images comparable to other competitive face swap models. We show that our framework is applicable to other generators such as StyleNeRF, paving a way to 3D-aware face swapping and is also compatible with other downstream StyleGAN2 generator tasks. The source code and models can be found at \url{https://github.com/usingcolor/LatentSwap}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. 2021. DeepFakes (https://github.com/deepfakes/faceswap).
  2. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? In IEEE/CVF International Conference on Computer Vision (ICCV), 4431–4440.
  3. Image2StyleGAN++: How to Edit the Embedded Images? In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  4. StyleFlow: Attribute-Conditioned Exploration of StyleGAN-Generated Images Using Conditional Continuous Normalizing Flows. ACM Trans. Graph., 40(3).
  5. Restyle: A residual-based stylegan encoder via iterative refinement. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6711–6720.
  6. HyperStyle: StyleGAN Inversion With HyperNetworks for Real Image Editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 18511–18521.
  7. Towards Open-Set Identity Preserving Face Synthesis. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6713–6722.
  8. Exchanging faces in images. In Computer Graphics Forum, volume 23, 3, 669–676. Wiley Online Library.
  9. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, 187–194.
  10. SimSwap: An Efficient Framework For High Fidelity Face Swapping, 2003–2011. New York, NY, USA: Association for Computing Machinery. ISBN 9781450379885.
  11. Arcface: Additive angular margin loss for deep face recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4690–4699.
  12. Generative Adversarial Networks. Advances in Neural Information Processing Systems, 27.
  13. StyleNeRF: A Style-based 3D Aware Generator for High-resolution Image Synthesis. In International Conference on Learning Representations (ICLR).
  14. GANSpace: Discovering Interpretable GAN Controls. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 9841–9850. Curran Associates, Inc.
  15. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 770–778.
  16. Alias-Free Generative Adversarial Networks. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems, volume 34, 852–863. Curran Associates, Inc.
  17. A Style-Based Generator Architecture for Generative Adversarial Networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  18. Analyzing and Improving the Image Quality of StyleGAN. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  19. Smooth-Swap: A Simple Enhancement for Face-Swapping With Smoothness. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10779–10788.
  20. DiffFace: Diffusion-based Face Swapping with Facial Guidance. Arxiv.
  21. Advancing High Fidelity Identity Swapping for Forgery Detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  22. Face swapping under large pose variations: A 3D model based approach. In 2012 IEEE International Conference on Multimedia and Expo, 333–338. IEEE.
  23. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
  24. StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2085–2094.
  25. DeepFaceLab: Integrated, flexible and extensible face-swapping framework. Arxiv preprint.
  26. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 8748–8763. PMLR.
  27. Searching for activation functions. arXiv preprint arXiv:1710.05941.
  28. Encoding in style: a stylegan encoder for image-to-image translation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2287–2296.
  29. Pivotal Tuning for Latent-based Editing of Real Images. ACM Trans. Graph.
  30. Interpreting the Latent Space of GANs for Semantic Face Editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  31. Closed-form factorization of latent semantics in gans. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1532–1540.
  32. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4): 1–14.
  33. Designing an Encoder for StyleGAN Image Manipulation. ACM Trans. Graph., 40(4).
  34. Unsupervised discovery of interpretable directions in the gan latent space. In International Conference on Machine Learning, 9786–9796. PMLR.
  35. A Geometric Analysis of Deep Generative Image Models and Its Applications. In International Conference on Learning Representations.
  36. HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping. In Zhou, Z.-H., ed., Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, 1136–1142. International Joint Conferences on Artificial Intelligence Organization. Main Track.
  37. Region-Aware Face Swapping. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7632–7641.
  38. High-resolution face swapping via latent semantics disentanglement. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7642–7651.
  39. FaceController: Controllable Attribute Editing for Face in the Wild. Proceedings of the AAAI Conference on Artificial Intelligence, 35(4): 3083–3091.
  40. StyleSwap: Style-Based Generator Empowers Robust Face Swapping. In Proceedings of the European Conference on Computer Vision (ECCV).
  41. Exposing deep fakes using inconsistent head poses. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 8261–8265. IEEE.
  42. In-domain GAN Inversion for Real Image Editing. In Proceedings of European Conference on Computer Vision (ECCV).
  43. One Shot Face Swapping on Megapixels. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4834–4844.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now