Vec2Face+ Synthetic FR Dataset Generation

• The paper introduces Vec2Face+ as a generative framework that creates facial images from FR embedding vectors, emphasizing identity consistency alongside inter-class separability and intra-class variation.
• It leverages a feature masked autoencoder with AttrOP for attribute control and a LoRA-based pose control module to significantly reduce training time while preserving image and identity quality.
• Empirical results demonstrate that the synthetic VFace datasets achieve competitive verification accuracy compared to real datasets, though challenges remain in twin verification and demographic fairness.

Vec2Face+ is a feature-to-image generative framework for synthetic face recognition (FR) dataset construction that explicitly treats identity consistency as a primary dataset property alongside inter-class separability and intra-class variation. In its canonical dataset-generation form, introduced in "Vec2Face+ for Face Dataset Generation" (Wu et al., 23 Jul 2025), the method generates images directly from FR embedding vectors, controls identity and attributes in feature space, adds AttrOP for attribute variation, introduces LoRA-based pose control for profile synthesis, and removes a patch-based discriminator to reduce training time by approximately 20% without hurting image quality. The resulting VFace datasets are designed as privacy-motivated alternatives to web-scraped real-face corpora while remaining competitive on standard verification benchmarks.

1. Problem setting and conceptual basis

Vec2Face+ is motivated by the use of synthetic identities as FR training data under privacy and data-governance constraints. Prior synthetic FR datasets largely optimized two desiderata: large inter-class separability and large intra-class attribute variation. Vec2Face+ argues that these are necessary but insufficient, and identifies a third factor—intra-class identity consistency—as a neglected bottleneck. The central claim is that synthetic datasets often place subtly different identities inside a single class folder, thereby introducing label noise that degrades FR training (Wu et al., 23 Jul 2025).

The paper supports this claim through an analysis of five synthetic datasets—DigiFace, SFace, DCFace, IDiff-Face, and HSFace10K—against CASIA-WebFace. Across nine attributes, namely brightness, quality, age, yaw, expression, beard area, baldness, eyeglasses, and mustache, the synthetic datasets show intra-class variation that is on par with or larger than CASIA-WebFace, yet they still trail the real dataset in FR accuracy. The reported explanation is that synthetic datasets have lower intra-class identity consistency than CASIA-WebFace. The paper therefore positions identity consistency, not merely attribute diversity, as a key determinant of downstream recognition quality (Wu et al., 23 Jul 2025).

In Vec2Face+, identity semantics are anchored in FR embedding space. The input feature is $f_{im} \in \mathbb{R}^{512}$, extracted by a pretrained ArcFace-R100 model trained on Glint360K. Because FR feature space explicitly encodes identity similarity, the framework generates images from features rather than from noise plus image-domain conditions. This makes identity manipulation continuous in feature space and makes cosine similarity a direct operational tool for identity sampling, intra-class perturbation, leakage filtering, and consistency measurement (Wu et al., 23 Jul 2025).

The paper formalizes intra-class identity consistency as

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$

Higher $D_{consis}$ indicates more consistent identities. It likewise formalizes inter-class separability as

$D_{sep} = \frac{N_{sep}}{N_{total}},$

where $N_{sep}$ counts identities whose cosine similarity against all other identity prototypes is below a threshold, 0.4 by default. These definitions locate both dataset quality criteria squarely in FR embedding geometry (Wu et al., 23 Jul 2025).

2. Architecture and learning objective

The main Vec2Face+ generator consists of three stages: feature expansion, a feature masked autoencoder (fMAE), and an image decoder. The 512-dimensional FR feature is expanded by two linear layers into a 2-D feature map of shape $(49,768)$, matching a ViT-B patch embedding layout. The fMAE is similar to MAE, but its masking operates on rows of the expanded feature map rather than on image patches. The row masking ratio is sampled from a truncated normal distribution, $$x\% \sim N_{truncated}(\text{max}=1,\ \text{min}=0.5,\ \text{mean}=0.75)$$. Masked rows are filled with a condition vector after encoding. In the main model, that condition is a projection of the input FR feature. A four-layer deconvolutional decoder then maps the representation to a $112 \times 112 \times 3$ image (Wu et al., 23 Jul 2025).

Relative to Vec2Face, Vec2Face+ removes the patch-based discriminator and its GAN loss. The paper states that dropping this adversarial component reduces training time by approximately 20% without hurting image quality. This design change is one of the clearest architectural distinctions between the two dataset-generation variants (Wu et al., 23 Jul 2025).

Training uses image reconstruction, identity consistency, and perceptual supervision. Let $IM_{rec} = decoder(fMAE(expand(f_{im})))$, let $IM_{gt}$ be the training image, and let $$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$0, $$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$1. The losses are

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$2

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$3

and the LPIPS perceptual term

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$4

The total objective is

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$5

The training data for Vec2Face+ is WebFace4M, containing 4M images and 200K identities. Identity labels are not required for training; only images and FR features are used (Wu et al., 23 Jul 2025).

Pose control is implemented as a second model. The main generator is frozen, and a 4-layer CNN extracts features from an input landmark image containing five facial landmark points. These CNN features become the fMAE condition instead of the projected FR feature. Parameter-efficient adaptation is realized through LoRA applied to layers that control the response of the fMAE and decoder to landmark conditioning. The LoRA update is written as

$$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$6

where $$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$7, $$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$8, $$D_{consis} = \frac{1}{N K^2} \sum_{i=1}^N \sum_{j=1}^K \sum_{k=1}^K \frac{ f_{i,j} \cdot f_{i,k} }{ \|f_{i,j}\| \|f_{i,k}\| }.$$9 is the rank, and $D_{consis}$0 is a scaling factor. Training still uses $D_{consis}$1, but with landmark-conditioned generation rather than direct feature conditioning (Wu et al., 23 Jul 2025).

3. Identity sampling, attribute control, and pose synthesis

Vec2Face+ enforces inter-class separability by sampling identity prototype vectors in embedding space and filtering them by cosine similarity. Candidate vectors are drawn as $D_{consis}$2 and are accepted only if their maximum cosine similarity to all previously accepted identity vectors is at most $D_{consis}$3. The paper states that in 512-dimensional space, random Gaussian vectors are naturally sparse, making it easy to sample millions of identities that satisfy this constraint (Wu et al., 23 Jul 2025).

Within each identity, intra-class variation is introduced by perturbing the identity feature. The perturbed vector is

$D_{consis}$4

with $D_{consis}$5 and $D_{consis}$6 constrained to $D_{consis}$7. Identity preservation is enforced by requiring $D_{consis}$8 and by imposing image-quality thresholds. For identity images, the conditions are $D_{consis}$9 and $D_{sep} = \frac{N_{sep}}{N_{total}},$0; for intra-class images they are $D_{sep} = \frac{N_{sep}}{N_{total}},$1 and $D_{sep} = \frac{N_{sep}}{N_{total}},$2, with quality measured by MagFace (Wu et al., 23 Jul 2025).

General attribute variation is handled by AttrOP, a gradient-based attribute operation performed directly in feature space. Starting from an initial perturbed vector $D_{sep} = \frac{N_{sep}}{N_{total}},$3, the method iteratively updates $D_{sep} = \frac{N_{sep}}{N_{total}},$4 by synthesizing an image with the Vec2Face+ generator, evaluating the image with pretrained pose, quality, and FR models, and descending the attribute objective

$D_{sep} = \frac{N_{sep}}{N_{total}},$5

where

$D_{sep} = \frac{N_{sep}}{N_{total}},$6

$D_{sep} = \frac{N_{sep}}{N_{total}},$7

$D_{sep} = \frac{N_{sep}}{N_{total}},$8

Because $D_{sep} = \frac{N_{sep}}{N_{total}},$9 and $N_{sep}$0 are differentiable, gradient descent is applied as

$N_{sep}$1

The paper states that AttrOP increases not only pose variation but also other attribute variations indirectly. For efficiency and to limit identity drift at extreme poses, early stopping with at most 30 iterations is used (Wu et al., 23 Jul 2025).

Profile-pose synthesis is delegated to the LoRA-based pose control model. Compared with AttrOP, it is described as more efficient and more identity-preserving for extreme poses. The reported efficiency figure is especially large: generating 200K profile-pose images takes less than 30 minutes on an NVIDIA L40S with LoRA, versus more than 20 hours using AttrOP, corresponding to more than a 40× speedup. The paper further states that LoRA empirically preserves identity better than AttrOP for extreme poses such as yaw greater than $N_{sep}$2 (Wu et al., 23 Jul 2025).

4. Dataset construction and leakage control

Vec2Face+ constructs VFace datasets from three image bases. Base 1 uses random identity sampling plus AttrOP-generated identity images and perturbed samples; it yields approximately 15M images and is described as mostly frontal. Base 2 applies AttrOP with explicit yaw targets in $N_{sep}$3, using SixDRepNet for pose and the same quality thresholds as Base 1; it produces approximately 6M images, 20 per identity. The paper notes that strong pose changes can compromise identity for extreme angles. Base 3 uses the LoRA-based pose control model: for each identity, 30 candidate images are randomly chosen from Base 1, two profile landmark images are provided as conditions, and profile-pose images are generated in one shot. Base 3 is reported to preserve identity better at large yaw than AttrOP (Wu et al., 23 Jul 2025).

Each final identity contains 50 images. The assembly procedure starts from Base 1’s 50 images and randomly replaces 40 of them with a mixture of Base 2 and Base 3 outputs to enforce attribute diversity and pose coverage. VFace10K and VFace20K use the first 10K and 20K identities, respectively. For VFace100K and VFace300K, DBSCAN with cosine distance and default parameters is applied within each identity to remove outliers, producing datasets of 4M and 12M images, respectively, aligned to WebFace4M-scale evaluation (Wu et al., 23 Jul 2025).

Identity leakage is explicitly filtered. The paper extracts identity features from WebFace4M, the generator’s training set, and removes any generated image whose similarity to any real identity exceeds 0.4. This is the stated mechanism for preventing leakage from real training identities into synthetic outputs (Wu et al., 23 Jul 2025).

The resulting datasets are synthetic FR training corpora intended for conventional discriminative recognition training. For the FR evaluations reported in the paper, the backbone is SE-IResNet50 with ArcFace loss, input size $N_{sep}$4, and augmentations consisting of horizontal flip, random crop, low resolution, random erase, and photometric transforms. The optimizer is SGD, training runs for 40 epochs, and the learning rate is 0.1 with decays at epochs 18, 28, and 35 (Wu et al., 23 Jul 2025).

5. Empirical performance, ablations, and observed failure modes

Vec2Face+ reports state-of-the-art average accuracy among synthetic datasets at the 10K-identity scale on five standard real-world verification sets: LFW, CFP-FP, CPLFW, AgeDB-30, and CALFW. For VFace10K, the paper reports LFW 99.35, CFP-FP 93.56, CPLFW 88.03, AgeDB-30 94.33, CALFW 94.17, with an average of 93.89. It further reports that VFace100K reaches 94.88 average and VFace300K reaches 94.93 average, both higher than CASIA-WebFace at 94.79. The paper characterizes this as the first time a synthetic dataset beats CASIA-WebFace in average accuracy across these five test sets (Wu et al., 23 Jul 2025).

Dataset	Scale	Average on LFW / CFP-FP / CPLFW / AgeDB-30 / CALFW
VFace10K	10K identities	93.89
VFace100K	4M images	94.88
VFace300K	12M images	94.93
CASIA-WebFace	real-world baseline	94.79

The same paper reports competitive large-scale verification results on IJBB and IJBC at $N_{sep}$5. VFace10K obtains 82.92 and 85.75, while VFace300K improves to 85.56 and 88.28. WebFace4M remains higher at 95.28 and 96.84. On Hadrian and Eclipse, which emphasize frontal pose, neutral expression, and good quality, VFace10K reaches 70.65 and 65.63, while VFace300K reaches 72.37 and 69.38; CASIA remains higher on Hadrian at 77.82 and is slightly lower on Eclipse at 68.52 (Wu et al., 23 Jul 2025).

Ablation studies attribute the gains to both AttrOP and LoRA-based pose control. On the same five standard sets, Base 1 only yields an average of 91.04; adding AttrOP gives 93.66; adding LoRA pose control gives 93.59; combining them gives 93.89. The paper states that AttrOP improves age robustness, LoRA improves pose, and the combination yields the best overall accuracy (Wu et al., 23 Jul 2025).

Two limitations are emphasized. First, twin verification remains poor for synthetic FR datasets. Across 11 synthetic-trained models, only 1 out of 11 outperforms random guessing, defined as 50%, on identical-twin verification. VFace300K reaches 50.15, while CASIA reaches 54.17 and WebFace4M reaches 64.20. The paper attributes this to the threshold-based definition of synthetic identities in FR space, which creates large angular margins but does not reproduce the fine-grained relational structure of real populations, including familial and twin similarities (Wu et al., 23 Jul 2025).

Second, bias remains more severe than in real-data training. The paper reports that FR models trained on synthetic datasets exhibit larger gender and race disparities than those trained on real data on BFW and BA-test, evaluated as $N_{sep}$6 by demographic group. The qualitative pattern varies by test set, but the range between best and worst groups is larger for synthetic-trained models, and the paper concludes that balancing separability and attribute diversity alone is insufficient: data generation and selection must explicitly target fairness constraints (Wu et al., 23 Jul 2025).

6. Relation to Vec2Face, adjacent variants, and later usage

Vec2Face+ belongs to a broader Vec2Face family whose naming spans more than one problem setting. The 2024 "Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors" paper addresses large-scale synthetic FR dataset generation with an fMAE, image decoder, and patch-based discriminator, uses PCA-space vector sampling with cosine thresholding, introduces AttrOP, scales to 300K identities and 15M images, and reports average accuracy up to 93.52 on five real-world test sets (Wu et al., 2024). Vec2Face+ preserves the feature-to-image dataset-generation direction but shifts emphasis toward identity consistency, introduces LoRA-based pose control, and removes the patch-based discriminator to speed training (Wu et al., 23 Jul 2025).

Earlier papers with the Vec2Face name addressed a different task: reconstruction of faces from blackbox FR embeddings. "Vec2Face: Unveil Human Faces from their Blackbox Features in Face Recognition" and "Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognition" describe DiBiGAN and DAB-GAN, respectively, combining bijective metric learning, distillation, and feature-conditional generation for embedding inversion rather than synthetic FR dataset construction (Duong et al., 2020, Truong et al., 2022). This earlier line shares the feature-to-image premise but is methodologically and experimentally distinct from the 2024-2025 dataset-generation line.

Subsequent literature also clarifies a terminological point. "Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation" uses Vec2Face, not Vec2Face+, as a lightweight decoder to expand one Stable Diffusion XL reference image into 49 identity-consistent variants, yielding 50 images per synthetic identity; the paper explicitly states that it does not mention Vec2Face+ (Li et al., 14 Aug 2025). Likewise, the synthetic-dataset review "Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise" reports benchmark results and privacy observations for Vec2Face, but does not define or evaluate a Vec2Face+ variant; its explicit "Vec2Face+" discussion is presented as an inference rather than as a documented system (Borsukiewicz et al., 20 Oct 2025).

Work	Problem setting	Relation to Vec2Face+
"Vec2Face" (Duong et al., 2020)	Face reconstruction from blackbox FR features	Earlier inversion-oriented family member
"Vec2Face-v2" / DAB-GAN (Truong et al., 2022)	Attention-based blackbox feature inversion	Extended inversion model, not the 2025 dataset generator
"Vec2Face" (Wu et al., 2024)	Synthetic FR dataset generation at scale	Direct precursor to Vec2Face+
"Vec2Face+ for Face Dataset Generation" (Wu et al., 23 Jul 2025)	Synthetic FR dataset generation with identity-consistency emphasis	Canonical dataset-generation formulation of Vec2Face+
"Hybrid Generative Fusion..." (Li et al., 14 Aug 2025)	Challenge pipeline using SDXL + Vec2Face expansion	Uses Vec2Face only, not Vec2Face+

Within the dataset-generation literature, Vec2Face+ is therefore most precisely understood as the 2025 extension of Vec2Face that reformulates the design objective around identity consistency, adds efficient profile-pose control through LoRA, retains FR-feature-space identity control, and demonstrates synthetic datasets that exceed CASIA-WebFace in average accuracy on five standard verification benchmarks while still exposing unresolved issues in twin verification and demographic fairness (Wu et al., 23 Jul 2025).