---
title: 'Vec2Face: Vector-to-Face Synthesis in Recognition'
url: https://www.emergentmind.com/topics/vec2face-18535ed6-655b-452f-af84-efc16bc8fb38
type: topic
---

# Vec2Face: Vector-to-Face Synthesis in Recognition

Vec2Face denotes a family of vector-to-face synthesis methods in face recognition, but the literature uses the name for two distinct problem settings. The earlier line studies inversion of face-recognition embeddings: given a high-level feature vector extracted by a black-box matcher, it reconstructs a realistic face image that preserves identity. The later line studies synthetic dataset generation: given sampled or perturbed facial feature vectors, it generates images of non-existent identities with controlled inter-class separation and intra-class variation. The name therefore spans DiBiGAN in the original black-box inversion work, DAB-GAN in Vec2Face-v2, the feature-masked-autoencoder-based dataset generator introduced in 2024, and the subsequent Vec2Face+ extension for VFace dataset construction [2003.06958; 2209.04920; 2409.02979; 2507.17192].

## 1. Nomenclature and research scope

One common source of confusion is that Vec2Face is not a single static architecture. In the literature, the name is attached to multiple systems that share a vector-conditioned face-synthesis premise but optimize for different end goals. In the inversion setting, the vector is a facial representation produced by a black-box face-recognition engine, and the task is to recover a realistic face consistent with that representation. In the dataset-generation setting, the vector is sampled or perturbed to represent a synthetic identity, and the task is to generate many controllable images suitable for training face-recognition models [2003.06958; 2409.02979].

| Paper | Primary task | Characteristic components |
|---|---|---|
| Vec2Face / DiBiGAN [2003.06958] | Reconstruct faces from black-box FR features | Bijective metric, distillation, feature-conditional progressive GAN |
| Vec2Face-v2 / DAB-GAN [2209.04920] | Reconstruct faces from black-box FR features | Attention-based generator, bijective metric, global distillation |
| Vec2Face [2409.02979] | Generate synthetic FR training data from sampled vectors | fMAE, image decoder, loosely constrained vectors, AttrOP |
| Vec2Face+ [2507.17192] | Generate controlled synthetic datasets | Feature expansion, fMAE, decoder, AttrOP, LoRA-based pose control |

This split is substantive rather than terminological. The inversion papers frame Vec2Face as a privacy and security problem for deployed face-recognition systems, whereas the later papers frame it as a method for building large-scale synthetic datasets without relying on class labels or text prompts [2209.04920; 2409.02979].

## 2. DiBiGAN and the black-box feature reconstruction problem

The original Vec2Face paper formulates feature reconstruction under a black-box face-recognition engine composed of a feature extractor \(F:I\to F\), with images \(I\in\mathbb{R}^{W\times H\times C}\) and extracted features \(f=F(I)\in\mathbb{R}^M\), and optionally a classifier \(C:F\to Y\). The task is: given only the feature vector \(f=F(I)\) of some unknown face \(I\), synthesize an image \(\tilde I\) that looks realistic and shares the same identity as \(I\). The generator \(G_\theta:F\to I\) is trained through
\[
\tilde x = G(F(x);\theta), \qquad
\theta^* = \arg\min_\theta E_{x\sim p_I}\bigl[L^x_G(G(F(x);\theta),x)\bigr],
\]
with the explicit constraint that the matcher \(F(\cdot)\) is treated as a black box and cannot be back-propagated through [2003.06958].

To address that constraint, the paper introduces Bijective Generative Adversarial Networks in a Distillation framework, or DiBiGAN. Its first defining element is bijective metric learning. A one-to-one mapping \(H:I\leftrightarrow Z\), with \(Z\in\mathbb{R}^D\), is learned so that distances in image space can be expressed in the latent space \(Z\). The latent metric is defined as the \(2\)-Wasserstein distance between Gaussians,
\[
L^z_G(\tilde z,z)=\|\tilde\mu-\mu\|_2^2 + \mathrm{Tr}\!\left(\tilde\Sigma+\Sigma-2(\tilde\Sigma^{1/2}\Sigma\tilde\Sigma^{1/2})^{1/2}\right),
\]
and an identity-margin variant is imposed to separate different identities while pulling same-identity samples together. In practice, \(H\) is implemented as a Real-NVP / coupling-layer network trained with a supervised log-likelihood plus ID-aware Gaussian priors [2003.06958].

The second element is distillation. Because the original matcher is inaccessible, a white-box student network \(F^S_\phi(\cdot)\) is trained to mimic it by minimizing
\[
\phi^* = \arg\min_\phi E_{x\sim p_x}\bigl[\|1-\cos(F(x),F^S_\phi(x))\|^2\bigr].
\]
The generator is then supervised not only through the final embedding but also through intermediate feature maps,
\[
L^{distill}_G(\tilde x,x)
= \sum_{j=1}^n \lambda_j \frac{\|F^S_j(\tilde x)-F^S_j(x)\|_2}{W_jH_jC_j}
+ \lambda_a \|1-\cos(F^S(\tilde x),F^S(x))\|^2.
\]
This distillation term is intended to maximize the information exploited from the black-box engine and to preserve identity in the reconstructed image [2003.06958].

The third element is the generator design. DiBiGAN uses a progressive-GAN-style generator with five up-sampling convolutional blocks, a separate feature-conditional branch of 8 fully connected layers that ingest the 512-dimensional feature \(f=F(x)\), AdaIN-based conditional injection at each convolutional block, and a random latent \(v\) as the main input. The complete generator objective combines bijective metric, distillation, adversarial, and pixel \(L_1\) reconstruction losses,
\[
L_G=\lambda_b L^{biject}+\lambda_d L^{distill}+\lambda_{adv}L^{adv}+\lambda_r L^{recon},
\]
with exponentially scheduled weights across progressive resolutions [2003.06958].

## 3. Vec2Face-v2 and the transition to DAB-GAN

Vec2Face-v2 retains the black-box inversion formulation but replaces the CNN-centric generator with an attention-based architecture called DAB-GAN, short for Attention-based Bijective Generative Adversarial Networks in a Distillation framework. The paper preserves the central premises of the original work—bijective metric learning, student-network distillation, WGAN-GP adversarial training, and \(L_1\) reconstruction—but argues that a pure Transformer generator is better suited to capturing global facial structure [2209.04920].

In DAB-GAN, the generator contains five resolution blocks, each composed of grid-self-attention for long-range context, cross-attention that injects the identity feature \(f\) at every scale, and MLP up-projection followed by upsampling. The generator contains no convolutional layers. The discriminator is a multi-scale CNN with five convolutional blocks and minibatch-stddev. At the block level, with patch-sequence input \(x_\ell\) and identity feature \(f\), the model computes
\[
Q,K,V=W^Qx_\ell,\;W^Kx_\ell,\;W^Vx_\ell,
\]
\[
attention_{self}=\mathrm{softmax}(QK^T/\sqrt d)\,V,
\]
\[
cross\text{-}attn=\mathrm{softmax}((W^Q_cx_\ell)(W^K_cf)^T/\sqrt d)\,(W^V_cf),
\]
followed by residual addition, LayerNorm, and MLP refinement. This makes the conditioning signal explicitly available at every resolution rather than only through AdaIN-style modulation [2209.04920].

The empirical results reported for DAB-GAN are framed as consistent state-of-the-art performance on CelebA, LFW, CFP-FP, CPLFW, AgeDB, and CA-LFW. At \(112\times112\) in black-box reconstruction, DAB-GAN \(+\) distill \(+\) biject achieves MS-SSIM \(=0.301\), Inception Score \(=2.322\), LFW \(=99.21\%\), CFP-FP \(=92.17\%\), CPLFW \(=88.27\%\), AgeDB \(=94.18\%\), and CA-LFW \(=90.38\%\). In white-box reconstruction, the same configuration yields LFW \(=99.52\%\), CFP-FP \(=93.11\%\), CPLFW \(=89.36\%\), AgeDB \(=95.58\%\), and CA-LFW \(=92.73\%\). The paper also reports cross-matcher evaluation in black-box mode on ArcFace, FaceNet, SphereFace+, MagFace, SCF-ArcFace, and AdaFace, and states that DAB-GAN closes \(>95\%\) of the gap to real-face verification accuracy; on ArcFace, for example, real faces obtain \(99.78\%\) and DAB-GAN reconstructions obtain \(99.21\%\) [2209.04920].

These results are significant because they establish that the inversion attack generalizes beyond the training matcher and remains effective under in-the-wild pose, expression, and occlusion variation. The paper therefore treats Vec2Face-v2 not only as a reconstruction method but also as evidence that deployed face-recognition embeddings are vulnerable to inversion, motivating work on private or secure embeddings [2209.04920].

## 4. Vec2Face as large-scale synthetic identity generation

The 2024 paper reuses the name Vec2Face for a different objective: synthesizing face images of non-existent persons to create a dataset for training face-recognition models. The paper identifies three goals as essential: realistic face images, a large number of distinct identities for inter-class separation, and proper variation in appearance for each identity as intra-class variation. The method is described as a holistic model that uses only a sampled vector as input and can flexibly generate and control the identity of face images and their attributes [2409.02979].

Architecturally, the model is composed of a feature masked autoencoder and an image decoder. During training, a frozen pretrained FR network, ArcFace-R100, maps a real face image \(x\) to a 512-dimensional feature \(h=\mathrm{FR}(x)\). That feature is linearly expanded into a two-dimensional map, randomly row-masked, processed by the fMAE encoder and decoder, and then converted by a lightweight image decoder into a reconstructed image \(\widehat x\). The training losses are pixel-wise MSE reconstruction,
\[
\mathcal L_{rec}=\|\widehat x-x\|_2^2,
\]
identity consistency,
\[
\mathcal L_{id}=1-\mathrm{Cos}(\mathrm{FR}(\widehat x),\mathrm{FR}(x)),
\]
LPIPS perceptual loss,
\[
\mathcal L_{lpips}=\mathrm{LPIPS}(x,\widehat x),
\]
and an adversarial loss \(\mathcal L_{GAN}\) from a patch-based discriminator, enabled only after \(1\)k epochs. The total loss is
\[
\mathcal L_{total}=\mathcal L_{rec}+\mathcal L_{id}+\mathcal L_{lpips}+\mathcal L_{GAN}.
\]
At inference, the method skips the FR-and-mask steps and maps a random \(512\)-D vector directly through the learned stack to produce a face image [2409.02979].

Inter-class separation is enforced by sampling identity vectors in PCA space fitted on \(512\)-D FR vectors from MS1MV2. Candidates are drawn as
\[
u_i\sim\mathcal N(\mu,\Sigma),\qquad v_i=\mathrm{PCA}^{-1}(u_i),
\]
with the constraint
\[
\mathrm{Cos}(v_i,v_j)\le \tau,\qquad \tau=0.3.
\]
The paper states that in \(512\)-D most random draws already satisfy this constraint, and only \(\sim1.7\%\) are rejected when aiming for \(300\)k identities. Intra-class variation is generated by Gaussian perturbation,
\[
v_{i,k}=v_i+\delta_{i,k},\qquad \delta_{i,k}\sim\mathcal N(0,\sigma^2I),
\]
using a mixture in the largest-scale experiments such that \(40\%\) of the images use \(\sigma=0.3\), \(40\%\) use \(\sigma=0.5\), and \(20\%\) use \(\sigma=0.7\), while maintaining \(\mathrm{Cos}(v_i,v_{i,k})\ge0.5\) [2409.02979].

The paper also introduces AttrOP, a gradient-descent attribute-control procedure. Starting from an initial vector \(z\), with generated image \(x(z)=D(z)\), it optimizes
\[
\mathcal L_{attr}(z)
=
\underbrace{1-\mathrm{Cos}(\mathrm{FR}(x(z)),v_i)}_{\mathcal L_{id}}
+
\underbrace{(Q-M_{quality}(x(z)))}_{\mathcal L_{quality}}
+
\underbrace{\bigl|P-|M_{pose}(x(z))|\bigr|}_{\mathcal L_{pose}},
\]
and updates \(z\leftarrow z-\eta\nabla_z \mathcal L_{attr}(z)\) for \(5\)–\(10\) steps. This provides targeted control of pose and quality while preserving identity [2409.02979].

The scaling claims are central to the paper. Vec2Face is reported to synthesize as many as \(300\)K identities, whereas \(60\)K is stated as the largest number of identities created in previous works. With \(50\) perturbations per identity, this yields \(15\)M total images. The training cost is reported as \(\sim2\) weeks on \(10\times\) RTX6000 GPUs using \(1\)M real images from \(50\)k IDs, while inference on a single Titan-Xp runs at \(\approx467\) FPS versus Arc2Face’s \(1.5\) FPS, with model size \(0.68\) GB instead of \(3.4\) GB. On five standard in-the-wild benchmarks, the reported average accuracy rises from \(92.00\%\) for HSFace10K to \(93.52\%\) for HSFace300K; the scale-up sequence is \(92.00\%\), \(92.47\%\), \(92.97\%\), \(93.22\%\), and \(93.52\%\) for \(10\)k, \(20\)k, \(100\)k, \(200\)k, and \(300\)k identities, respectively [2409.02979].

## 5. Vec2Face+ and the VFace datasets

Vec2Face+ extends the dataset-generation line by emphasizing identity consistency under increased intra-class variation. It is described as a two-stage, feature-to-image generator trained on face-recognition embedding vectors rather than class labels. At inference, it performs identity sampling, perturbation for intra-class variation, image generation through a trained main model, and optional post-processing via AttrOP or LoRA-based pose control [2507.17192].

The training objective removes the GAN discriminator and treats the model purely as an image reconstructor:
\[
\mathcal L_{total}
=
\mathcal L_{rec}
+
\mathcal L_{id}
+
\lambda \mathcal L_{lpips},
\qquad \lambda=0.2.
\]
Here
\[
\mathcal L_{rec}=\|\mathrm{IM}_{rec}-\mathrm{IM}_{gt}\|_2^2,
\]
\[
\mathcal L_{id}=1-\frac{f_{rec}\cdot f_{gt}}{\|f_{rec}\|\|f_{gt}\|},
\]
with \(f_{(\cdot)}\) a \(512\)-D ArcFace embedding, and
\[
\mathcal L_{lpips}
=
\sum_{\ell}\left\|w_\ell\odot\bigl(\phi_\ell(\mathrm{IM}_{rec})-\phi_\ell(\mathrm{IM}_{gt})\bigr)\right\|_2^2,
\]
where \(\phi_\ell\) are VGG features and \(w_\ell\) are learned channel weights. The paper explicitly states that no adversarial, inter-class separability, or intra-class consistency losses are used; the latter two are enforced by sampling and dataset construction [2507.17192].

The main model uses feature expansion, fMAE, and a decoder. Its input is a \(512\)-D feature vector \(f_{im}\). Feature expansion applies a linear layer \(512\to1\times768\), then \(1\times768\to49\times768\), producing a \(49\)-token patch sequence of dimension \(768\). The feature-masked autoencoder is based on MAE (ViT-B) with row-wise masking, randomly dropping \(x\%\) of rows, with \(x\sim\mathcal N(0.75,\dots)\), and filling them via cross-attention. The image decoder contains four ConvTranspose2d layers that upsample from \(7\times7\) to \(112\times112\), with channel dimensions \(\{256\to128\to64\to3\}\), ReLU activations, and final Tanh or sigmoid [2507.17192].

Identity sampling in Vec2Face+ draws \(512\)-dimensional vectors \(v^{(i)}_{id}\) from a zero-mean unit-variance Gaussian and accepts a new sample only if \(\cos(v^{(i)}_{id},v^{(j)}_{id})\le\tau\), with \(\tau\approx0.3\). The paper states that in \(512\)-D space most random draws satisfy this “hollow-sphere” condition, so \(4\)M identities can be sampled easily. Intra-class variation is then generated by normalized perturbation,
\[
v_p=\frac{v_{id}+\epsilon}{\|v_{id}+\epsilon\|},\qquad
\epsilon\sim\mathcal N(0,\sigma^2I),\quad \sigma\in\{0.3,0.5,0.7\},
\]
subject to \(\cos(v_p,v_{id})\ge0.5\) [2507.17192].

The paper retains AttrOP and introduces LoRA-based pose control for profile faces. A \(4\)-layer CNN encodes a \(5\)-point landmark map into a \(49\times768\) feature map, while low-rank adapters with \(\Delta W=AB\), where \(A\in\mathbb{R}^{d\times r}\), \(B\in\mathbb{R}^{r\times d}\), and \(r\ll d\), are inserted into each transformer block’s attention weights. Only \((A,B)\) are learned on a small profile-pose dataset, and the original weights remain frozen. At inference, any pair of identity feature and landmark feature yields single-pass generation of profile faces with yaw up to \(80^\circ\), while the paper states that this preserves identity better than AttrOP and gives generation throughput \(\sim40\times\) faster than AttrOP [2507.17192].

Quantitatively, Vec2Face+ is used to construct VFace10K, VFace100K, and VFace300K. On the standard five-test average over LFW, CFP-FP, CPLFW, AgeDB-30, and CALFW, the reported average accuracies are \(93.89\%\) for VFace10K, \(94.88\%\) for VFace100K, and \(94.93\%\) for VFace300K, compared with \(94.79\%\) for CASIA-WebFace. The paper states that this is the first time a synthetic dataset beats the CASIA-WebFace in average accuracy. At the same time, it reports that only \(1\) out of \(11\) synthetic datasets outperforms random guessing, that is \(50\%\), in twin verification, and that models trained with synthetic identities are more biased than those trained with real identities [2507.17192].

## 6. Privacy, bias, and downstream uses

The inversion and dataset-generation branches of Vec2Face sit on opposite sides of the privacy question. The inversion papers show that face-recognition embeddings can be sufficiently informative to support realistic face recovery from black-box features, which the authors explicitly describe as a vulnerability of deployed face-recognition engines and a reason to study embedding-level defenses such as noise, encryption, or partial release [2209.04920]. By contrast, the later synthetic-data papers position vector-conditioned generation as a privacy-preserving alternative to training on real faces, while still acknowledging unresolved issues in leakage, bias, and difficult identity distinctions [2409.02979; 2507.17192; 2510.17372].

The 2025 survey and empirical evaluation of synthetic facial datasets provides a concentrated account of Vec2Face’s privacy and biometric properties. It describes Vec2Face as a public release with \(15\) million images of \(300\)k synthetic identities at \(112\times112\) px, and reports that the method discards any generated sample whose maximum cosine similarity to the nearest real image exceeds a threshold \(\tau\); only \(0.4\%\) of initial samples are reported as removed. In additional experiments against CASIA-WebFace, the bulk of synthetic–real similarities lie below \(0.4\), with a long tail of \(<1\%\) outliers above \(0.8\), which the paper states are manually filterable. For intra-class variability and separability, the survey reports genuine mean \(\pm\) std \(=0.458\pm0.129\), impostor mean \(\pm\) std \(=0.007\pm0.072\), Equal Error Rate \(=0.009\), FMR100 \(=0.008\), FMR1000 \(=0.044\), and FDR \(=9.372\). On RFW, the same source reports verification accuracies of \(84.15\%\) for African, \(85.35\%\) for Asian, \(90.28\%\) for Caucasian, \(87.50\%\) for Indian, and \(86.82\%\) overall, and notes that Vec2Face does not include explicit demographic balancing at generation time [2510.17372].

A related downstream use appears in a hybrid generative fusion system for competition data construction. There, Stable Diffusion XL produces one reference portrait per synthetic identity, and the cropped reference image is fed to Vec2Face, described as a lightweight decoder that rapidly generates \(49\) identity-consistent variants exhibiting controlled changes in pose, illumination, and facial expression while preserving core identity features. This two-stage identity generation yields \(50\) images per identity and was part of a first-place solution in the DataCV ICCV Challenge [2508.10672].

Taken together, these works suggest a dual legacy for Vec2Face. In one line, it is a sequence of inversion methods—DiBiGAN and DAB-GAN—that reveal how much identity information can be recovered from black-box face embeddings. In the other, it is a sequence of synthetic-data generators—Vec2Face and Vec2Face+—that use sampled or perturbed feature vectors to construct large-scale face-recognition training sets with controllable identity structure. The shared technical motif is vector-conditioned facial synthesis; the differing significance lies in whether that synthesis is used to invert a representation or to replace real training data with synthetic identities [2003.06958; 2209.04920; 2409.02979; 2507.17192].

Source: https://www.emergentmind.com/topics/vec2face-18535ed6-655b-452f-af84-efc16bc8fb38