FairFaceGen: Balanced Synthetic Face Dataset
- FairFaceGen is a balanced synthetic face dataset that uses demographically controlled seed identities and identity-preserving augmentations to evaluate FR performance and bias.
- It employs a two-stage generation pipeline with tools like Flux.1-dev, SD35, Arc2Face, and IP-Adapters to create controlled intra-class diversity.
- Empirical results indicate competitive benchmark accuracy and reduced racial bias, though real data still outperforms on challenging IJB-B/C protocols.
FairFaceGen is a balanced synthetic face dataset and generation framework introduced for studying whether face recognition (FR) systems trained on synthetic data can achieve both high accuracy and fairness. It is built by generating demographically controlled seed identities with Flux.1-dev and Stable Diffusion v3.5 (SD35), and then expanding those identities with identity-preserving augmentation methods including Arc2Face and four IP-Adapters. The work is organized around scale-controlled comparison: synthetic and real datasets are matched in identity count so that measured differences in FR performance and bias are more directly attributable to generation choices rather than dataset size (Korshunov et al., 28 Jul 2025).
1. Definition and research objective
FairFaceGen is described as a purpose-built balanced synthetic face dataset for evaluating the impact of synthetic data on both FR performance and demographic bias. Its explicit target is the question of whether synthetic data can simultaneously support strong recognition accuracy and bias mitigation. In that framing, FairFaceGen is not merely a collection of generated images; it is a controlled experimental substrate for isolating the effects of the seed generator, the identity augmentation generator, and the number of intra-class augmentations on downstream FR behavior (Korshunov et al., 28 Jul 2025).
The paper identifies three problems in prior synthetic-face studies. First, comparisons between synthetic and real datasets are often unfair because the datasets differ substantially in scale. Second, prior work tends to emphasize identity generation while underexploring the role of intra-class augmentation. Third, bias is often insufficiently evaluated even when synthetic data is nominally balanced. FairFaceGen addresses these points by controlling dataset scale, separating seed identity generation from augmentation generation, and evaluating fairness on Racial Faces in-the-Wild (RFW) (Korshunov et al., 28 Jul 2025).
A central methodological choice is that all datasets in the comparison are limited to approximately 10K identities. Synthetic variants contain about 10–11K identities, while the real baselines include CASIA-WebFace at about 10.5K identities and WebFace42M-10K as a 10K subset. This scale matching is presented as necessary for fair comparison (Korshunov et al., 28 Jul 2025).
2. Demographic design and synthetic generation pipeline
FairFaceGen constructs a balanced identity pool over 2 genders—male and female—4 races—Caucasian White, Black, East-Asian, and Latino/Hispanic—and 7 age groups ranging from children to seniors. The combinatorial design is explicit:
With 200 identities per combination, each seed generator yields
seed identities (Korshunov et al., 28 Jul 2025).
The generation process has two stages. In the first stage, seed identities are generated from text prompts using Flux.1-dev or SD35. The same prompts are used for both models so that output differences are attributable to the generator rather than prompt variation. In addition to the controlled demographic attributes, the prompts vary soft visual attributes such as hairstyle, hair color, glasses, earrings, lighting, background, photo style, image quality, and camera type. These soft attributes are sampled randomly and are therefore intended to be evenly distributed across groups (Korshunov et al., 28 Jul 2025).
In the second stage, each seed identity is expanded through identity-preserving augmentation. The augmentation count per identity is varied systematically across 8, 16, 24, and 32 images, allowing the study to separate the effect of identity balance from the effect of intra-class diversity (Korshunov et al., 28 Jul 2025).
| Stage | Components | Role |
|---|---|---|
| Seed identity generation | Flux.1-dev; SD35 | Create balanced synthetic identities |
| Identity-preserving augmentation | Arc2Face; four IP-Adapters | Create intra-class variation |
| Controlled comparison | Synthetic and real datasets at approximately 10K identities | Isolate generation effects |
This construction makes FairFaceGen a framework of dataset variants rather than a single fixed dataset. A plausible implication is that the paper treats demographic control, identity diversity, and augmentation fidelity as separable design axes rather than as a single monolithic notion of “synthetic data quality.”
3. Augmentation mechanisms and identity preservation
The augmentation stage is central to FairFaceGen’s design. Arc2Face conditions the generation process using an ArcFace embedding and is paired in the paper with Stable Diffusion v1.5 (SD15), using face embeddings from an ArcFace model pretrained on WebFace42M. The four IP-Adapter variants differ along two dimensions: the diffusion backbone (SD15 or SDXL) and the identity embedding source (InsightFace or CLIP) (Korshunov et al., 28 Jul 2025).
These methods are intended to generate multiple images of the same identity under variations in pose, expression, and illumination while preserving identity consistency. The paper emphasizes that this stage is not ancillary: it is one of the primary determinants of both accuracy and fairness. In effect, FairFaceGen separates the problem of generating who the identity is from the problem of generating how that identity varies across samples.
The paper supplements this design with an identity-similarity analysis using an independent EdgeFace model. For each identity, intra-class cosine distances among augmentations are computed; distributions close to zero indicate highly consistent augmentations. The reported result is that synthetic data, including FairFaceGen variants, has intra-class similarity distributions comparable to real WebFace data. Inter-class analysis is stricter: the minimum cosine distance from each identity to its most similar different identity is used to assess separation. On this measure, synthetic datasets generally have lower inter-class variability than real data. The paper notes that HyperFace is closest to real WebFace42M, while Flux-based identities are furthest from real WebFace42M, suggesting that Flux may be weaker as a seed generator for FR training (Korshunov et al., 28 Jul 2025).
This diagnostic result clarifies an important distinction. Identity-preserving augmentation can succeed in keeping samples within a class coherent while still producing an overall dataset whose classes are not sufficiently separated from one another. That distinction becomes consequential on more difficult FR benchmarks.
4. Experimental protocol and fairness metrics
All FR models in the study use the same backbone and loss: ResNet-50 trained with AdaFace loss. Face detection and landmark extraction use MTCNN; faces are aligned and resized to 112 × 112 RGB. Training proceeds for 30 epochs with batch size 256, an initial learning rate of 0.1, and learning-rate drops by a factor of 10 at epochs 12, 20, and 26. Standard AdaFace augmentation is applied with probability 0.2. At inference time, embeddings are compared using cosine similarity (Korshunov et al., 28 Jul 2025).
The evaluation is divided across three regimes. Standard FR performance is assessed on LFW, CA-LFW, CP-LFW, CFP-FP, and AgeDB-30. More difficult generalization is tested on IJB-B and IJB-C, using True Accept Rate (TAR) at
Bias is measured on RFW, which contains four racial groups—African, Asian, Caucasian, and Indian—with balanced genuine and impostor pairs (Korshunov et al., 28 Jul 2025).
The paper uses both direct group-wise performance and a compact fairness summary. The bias metric is
so lower standard deviation indicates lower disparity across demographic groups. A joint accuracy–fairness score is also defined as
This design makes the evaluation explicitly multi-objective: a synthetic dataset is not considered successful purely by maximizing average recognition accuracy if it also increases disparity (Korshunov et al., 28 Jul 2025).
The synthetic baselines are DCFace, Digi2Real, and HyperFace; the real baselines are CASIA-WebFace and WebFace42M-10K. This comparative set is intended to distinguish the effect of FairFaceGen’s balancing and augmentation strategy from generic effects of synthetic-data usage (Korshunov et al., 28 Jul 2025).
5. Empirical findings on accuracy, generalization, and bias
The paper’s most consequential finding is that standard benchmark accuracy does not guarantee strong real-world generalization. Models can score well on LFW-style benchmarks yet perform poorly on IJB-B/C. CASIA-WebFace is highlighted as a striking example: it achieves strong results on standard benchmarks but very poor IJB-B/C performance (Korshunov et al., 28 Jul 2025).
Against that background, FairFaceGen reveals a mixed but informative picture of synthetic data. Synthetic datasets are competitive on standard benchmarks, but they still lag behind real data on the challenging IJB-B/C protocols. The best synthetic baseline, DCFace-10K, remains below WebFace42M-10K on IJB-B/C. The paper therefore treats synthetic data as promising but not yet a replacement for real-data generalization on the hardest verification settings (Korshunov et al., 28 Jul 2025).
On fairness, the results are more favorable. SD35-based synthetic data shows the strongest behavior on RFW, and the best reported result is SD35-All-IPA with the lowest standard deviation,
which is lower than the real-data baselines. The paper presents this as evidence that demographically balanced synthetic generation can reduce racial disparity in FR (Korshunov et al., 28 Jul 2025).
The number and quality of intra-class augmentations are also decisive. Increasing images per identity from 8 to 16 generally improves recognition on both standard benchmarks and IJB-B/C. Beyond that, the effect becomes mixed: 24 and 32 images per identity do not guarantee better outcomes, and for both SD35 and Flux, increasing augmentation count eventually causes IJB-B/C performance to drop even if standard benchmark scores continue to improve. The paper interprets this as possibly arising from excessive intra-class variation, identity leakage between classes, or augmentations becoming less identity-faithful. It also reports that All-IPA mixtures can reduce IJB-B/C performance, indicating that greater synthetic diversity is not automatically beneficial (Korshunov et al., 28 Jul 2025).
The generator choice matters as well. Flux-based datasets show higher bias than SD35-based ones, and the paper suggests that Flux images may be more “professionally styled” and less representative of in-the-wild face variation. This suggests that visual plausibility alone is not a sufficient criterion for synthetic FR training data; distributional resemblance to operational imagery is also material (Korshunov et al., 28 Jul 2025).
6. Interpretation, limitations, and relation to adjacent research
FairFaceGen’s scope is narrower than generic face generation and broader than a simple dataset release. It is specifically about constructing balanced synthetic data for FR training under controlled comparisons. That places it in a distinct position relative to adjacent face-generation and benchmarking efforts. DeepFaceGen, for example, is a large-scale universal benchmark for face forgery detection that explicitly considers fairness across ethnicities and provides rich prompt labels, but its target task is detection rather than FR training (Bei et al., 2024). UniF2ace addresses fine-grained face understanding and generation in a unified multimodal model, emphasizing captioning, VQA, and text-to-face synthesis rather than fairness-aware FR data construction (Li et al., 11 Mar 2025). FaceScore focuses on face quality as a weak point of text-to-image diffusion models and proposes a face-specific metric and refinement method, again targeting generation quality rather than FR bias (Liao et al., 2024).
FairFaceGen therefore addresses a different research question: whether synthetic generation can be operationalized as a controlled intervention on demographic balance without sacrificing recognition performance. Its main limitation, made explicit by the results, is that balanced synthetic data still underperforms real data on IJB-B/C. Another limitation is that fairness gains depend strongly on augmentation design; demographic balance at the seed-identity stage does not by itself guarantee optimal downstream behavior. This suggests that fairness in synthetic FR datasets is mediated jointly by class balance, inter-class separation, and identity-preserving intra-class variability.
The paper’s practical guidance follows from these findings. If fairness matters, demographic attributes should be controlled during synthetic generation; synthetic FR datasets should not be judged only on easy benchmarks; dataset scale should be matched when comparing synthetic and real data; the seed generator should be chosen carefully, with SD35 generally yielding better fairness properties than Flux; and excessive intra-class augmentation should be avoided. The paper further suggests that combining synthetic and real data may offer the best route to improving both accuracy and fairness (Korshunov et al., 28 Jul 2025).
In that sense, FairFaceGen is best understood not as a claim that synthetic data has solved FR bias, but as a controlled empirical framework showing where synthetic balancing helps, where it fails, and which design variables most strongly govern the trade-off between recognition accuracy and demographic disparity.