Papers
Topics
Authors
Recent
Search
2000 character limit reached

Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors

Published 4 Sep 2024 in cs.CV | (2409.02979v4)

Abstract: This paper studies how to synthesize face images of non-existent persons, to create a dataset that allows effective training of face recognition (FR) models. Besides generating realistic face images, two other important goals are: 1) the ability to generate a large number of distinct identities (inter-class separation), and 2) a proper variation in appearance of the images for each identity (intra-class variation). However, existing works 1) are typically limited in how many well-separated identities can be generated and 2) either neglect or use an external model for attribute augmentation. We propose Vec2Face, a holistic model that uses only a sampled vector as input and can flexibly generate and control the identity of face images and their attributes. Composed of a feature masked autoencoder and an image decoder, Vec2Face is supervised by face image reconstruction and can be conveniently used in inference. Using vectors with low similarity among themselves as inputs, Vec2Face generates well-separated identities. Randomly perturbing an input identity vector within a small range allows Vec2Face to generate faces of the same identity with proper variation in face attributes. It is also possible to generate images with designated attributes by adjusting vector values with a gradient descent method. Vec2Face has efficiently synthesized as many as 300K identities, whereas 60K is the largest number of identities created in the previous works. As for performance, FR models trained with the generated HSFace datasets, from 10k to 300k identities, achieve state-of-the-art accuracy, from 92% to 93.52%, on five real-world test sets (\emph{i.e.}, LFW, CFP-FP, AgeDB-30, CALFW, and CPLFW). For the first time, the FR model trained using our synthetic training set achieves higher accuracy than that trained using a same-scale training set of real face images on the CALFW, IJBB, and IJBC test sets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. How does gender balance in training data affect face recognition accuracy? In IJCB, pp.  1–10. IEEE, 2020.
  2. img2pose: Face alignment and detection via 6dof, face pose estimation. In CVPR, pp.  7617–7627, 2021.
  3. Partial FC: training 10 million identities on a single machine. In ICCVW, pp.  1445–1449, 2021.
  4. Digiface-1m: 1 million digital face images for face recognition. In WACV, pp.  3515–3524. IEEE, 2023.
  5. All are worth words: a vit backbone for score-based diffusion models. In NeurIPS, 2022.
  6. Sface: Privacy-friendly and accurate face recognition using synthetic data. In IJCB 2022, pp.  1–11. IEEE, 2022a.
  7. Unsupervised face recognition using unlabeled synthetic data. 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), pp.  1–8, 2022b.
  8. Idiff-face: Synthetic-based face recognition through fizzy identity-conditioned diffusion model. In ICCV, pp.  19650–19661, 2023a.
  9. Exfacegan: Exploring identity directions in gan’s learned latent space for synthetic identity generation. In IJCB, pp.  1–10. IEEE, 2023b.
  10. Sface2: Synthetic-based face recognition with w-space identity-driven sampling. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2024.
  11. Vggface2: A dataset for recognising faces across pose and age. In IEEE F&G, pp.  67–74, 2018.
  12. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023.
  13. Ilvr: Conditioning method for denoising diffusion probabilistic models. ICCV, 2021.
  14. Arcface: Additive angular margin loss for deep face recognition. In CVPR, pp.  4690–4699, 2019.
  15. Arcface: Additive angular margin loss for deep face recognition. TPAMI, 44, 2022.
  16. Fine-grained face verification: Fglfw database, baselines, and human-dcmn partnership. PR, 66:63–73, 2017.
  17. Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  18. Taming transformers for high-resolution image synthesis. In CVPR, pp.  12873–12883, 2021.
  19. Masked diffusion transformer is a strong image synthesizer. In ICCV, pp.  23164–23173, 2023.
  20. Synthetic face datasets generation via latent space exploration from brownian identity diffusion. arXiv preprint arXiv:2405.00228, 2024.
  21. Deep residual learning for image recognition. In CVPR, pp.  770–778, 2016.
  22. Masked autoencoders are scalable vision learners. In CVPR, pp.  16000–16009, 2022.
  23. Toward robust and unconstrained full range of rotation head pose estimation. IEEE Transactions on Image Processing, 33:2377–2387, 2024. doi: 10.1109/TIP.2024.3378180.
  24. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008.
  25. Image-to-image translation with conditional adversarial networks. In CVPR, pp.  1125–1134, 2017.
  26. Dcface: Synthetic face generation with dual condition diffusion model. In CVPR, pp.  12715–12725. IEEE, 2023.
  27. Identity-driven three-player generative adversarial network for synthetic-based face recognition. In CVPR, pp.  806–816, 2023.
  28. Mage: Masked generative encoder to unify representation learning and image synthesis. In CVPR, pp.  2142–2152, 2023.
  29. Photomaker: Customizing realistic human photos via stacked id embedding. In CVPR, pp.  8640–8650, 2024.
  30. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  31. Lcm-lora: A universal stable-diffusion acceleration module. arXiv preprint arXiv:2311.05556, 2023.
  32. Magface: A universal representation for face recognition and quality assessment. In CVPR, pp.  14225–14234, 2021.
  33. Agedb: The first manually collected, in-the-wild age database. In CVPRW, pp.  1997–2005, 2017.
  34. Arc2face: A foundation model of human faces. arXiv preprint arXiv:2403.11641, 2024.
  35. Scalable diffusion models with transformers. In ICCV, pp.  4195–4205, 2023.
  36. Synface: Face recognition with synthetic data. In ICCV, pp.  10860–10870. IEEE, 2021.
  37. Learning transferable visual models from natural language supervision. In ICML, pp.  8748–8763, 2021.
  38. Morph: A longitudinal image database of normal adult age-progression. In IEEE F&G, pp.  341–345, 2006.
  39. High-resolution image synthesis with latent diffusion models. In CVPR, pp.  10684–10695, 2022.
  40. Frontal to profile face verification in the wild. In WACV, pp.  1–9, 2016.
  41. High-fidelity guided image synthesis with latent diffusion models. In CVPR, pp.  5997–6006. IEEE, 2023.
  42. Doppelver: A benchmark for face verification. In International Symposium on Visual Computing, pp.  431–444. Springer, 2023.
  43. Visual autoregressive modeling: Scalable image generation via next-scale prediction. CVPR, 2024.
  44. Face0: Instantaneously conditioning a text-to-image model on a face. In SIGGRAPH, pp.  1–10, 2023.
  45. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024.
  46. What should be balanced in a” balanced” face recognition dataset. In BMVC, volume 1, pp.  2, 2023.
  47. What is a goldilocks face verification test set? arXiv preprint arXiv:2405.15965, 2024a.
  48. Identity overlap between face recognition train/test data: Causing optimistic bias in accuracy measurement. arXiv preprint arXiv:2405.09403, 2024b.
  49. Fastcomposer: Tuning-free multi-subject image generation with localized attention. arXiv preprint arXiv:2305.10431, 2023.
  50. Facestudio: Put your face everywhere in seconds. arXiv preprint arXiv:2312.02663, 2023.
  51. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023.
  52. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022.
  53. Adding conditional control to text-to-image diffusion models. In ICCV, pp.  3836–3847, 2023.
  54. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pp.  586–595, 2018.
  55. Cross-pose lfw: A database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5(7), 2018.
  56. Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197, 2017.
  57. Webface260m: A benchmark for million-scale deep face recognition. PAMI, 45(2):2627–2644, 2023.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

Sign Up for Free