---
title: FedFair-100 Benchmark for Federated Fairness
url: https://www.emergentmind.com/topics/fedfair-100-benchmark
type: topic
---

# FedFair-100 Benchmark for Federated Fairness

FedFair-100 is a synthetic tabular benchmark specifically designed to evaluate privacy-preserving and fairness-aware federated learning protocols under conditions of extreme heterogeneity and non-IID data partitioning. Developed for stress-testing fairness verification schemes, it emulates large-scale demographic variation and label–attribute relationships using statistical calibration to U.S. census distributions. As a primary testbed in CryptoFair-FL, FedFair-100 enables rigorous analysis of demographic parity and equalized odds violations—alongside utility—across privacy budgets, delivering reproducible results for comparison of federated and centralized approaches [2601.12447].

## 1. Construction and Data Generation

FedFair-100 comprises one million records ($M = 1\,000\,000$) generated from mixtures of tabular distributions. Parameters, including age, income, and education, are selected to match realistic U.S. census statistics. Each record is a triplet $(x, y, a)$, where $x$ denotes the feature vector in $\mathbb{R}^d$, $y$ the binary label, and $a$ the binary protected attribute. Across 100 simulated institutional clients, the prevalence of the protected attribute $a = 1$ varies uniformly from 5% to 45%, systematically inducing strong non-IID data heterogeneity. Conditional label–attribute probabilities $P[y=1|a]$ are institution-specific, generating diverse fairness challenges. Preprocessing normalizes features to zero mean and unit variance per institution; binary features are encoded as $\{0,1\}$, categorical features are one-hot encoded, and continuous features are clipped to $[-5, 5]$ after normalization.

## 2. Data Partitioning and Client Distribution

FedFair-100 simulates 100 federated learning clients ($n = 100$), each assigned approximately $m_i \approx M/n$ samples, perturbed by $\pm 20\%$ to reflect realistic institutional imbalance. The partitioning is highly non-IID: each client $i$ is characterized by its protected attribute rate $p_i = P[a=1] \in [0.05, 0.45]$ and conditional label–attribute probabilities. This protocol enforces client-level demographic and label correlations, mirroring regulatory and real-world deployment scenarios.

| Attribute               | Range/Type          | Handling                        |
|-------------------------|---------------------|----------------------------------|
| Client count ($n$)      | 100                 | Fixed simulation                 |
| Records per client      | $\approx 10\,000 \pm 20\%$ | Imbalanced, non-IID            |
| $P[a=1]$ per client     | $[0.05, 0.45]$      | Uniform sweep                    |
| Label–attribute $P[y=1|a]$ | Varied client-wise | Induces fairness variations      |

## 3. Feature, Label, and Protected Attribute Spaces

The feature space is $d \approx 30$ dimensions, spanning mixed continuous and categorical tabular variables. The label space is binary: $\mathcal{Y} = \{0, 1\}$. The protected attribute $A \in \{0, 1\}$ is explicitly represented for each record and reserved locally at the client level, never centralized—satisfying privacy constraints. The client-wise variation in $P[A=1]$ across institutions is central to the benchmark’s role in evaluating fairness enforcement.

## 4. Benchmark Tasks, Model Classes, and Evaluation Settings

FedFair-100 assesses binary classification under federated learning with explicit fairness constraints. Standard model classes are logistic regression or small neural networks $f_\theta : \mathbb{R}^d \to [0,1]$. Evaluation employs both fairness metrics—demographic parity (DP) and equalized odds (EO)—alongside utility measures: classification accuracy, F1-score, and AUROC. Data are split (per client) into 80/20 train/test partitions stratified by label and protected attribute. The protocol stipulates full client participation ($q = 1$) in each of $\sim$100 communication rounds (FedAvg) or $\sim$106 rounds (CryptoFair-FL) to convergence. Privacy budgets $\varepsilon$ are swept from 0.25 to 2.0 in increments of 0.25; each experimental condition is repeated in 5 independent runs, reporting mean and standard deviation.

## 5. Formal Fairness Metric Definitions

FedFair-100 operationalizes explicit definitions for fairness:

**Demographic Parity Violation**
\[
\Delta_{\mathrm{DP}}(\theta) = |P[\hat{Y}=1|A=0] - P[\hat{Y}=1|A=1]|
\]
where $\hat{Y} = \mathbf{1}\{f_\theta(X) > 0.5\}$.

**Equalized Odds Violation**
\[
\Delta_{\mathrm{EO}}(\theta) = \max_{y \in \{0,1\}} |P[\hat{Y}=1|A=0,Y=y] - P[\hat{Y}=1|A=1,Y=y]|
\]

In practice, fairness metrics are computed by aggregating client-wise counts,
\[
S_{a,\hat{y}} = \sum_{i=1}^n \sum_{j=1}^{m_i} \mathbf{1}\{a_j^{(i)}=a,\, \hat{y}_j^{(i)}=\hat{y}\},\quad a,\hat{y}\in \{0,1\}
\]
The decrypted noisy sums $\tilde{S}_{a, \hat{y}}$ yield empirical rates $\hat{P}_a$ for DP, where $\hat{\Delta}_{\mathrm{DP}} = |\hat{P}_0 - \hat{P}_1|$; analogous procedures apply for EO.

## 6. Experimental Protocol Details and Cryptographic Privacy

FedFair-100 adopts an 80/20 stratified train–test split by label and protected attribute. Communication proceeds over 100–106 rounds per run, with no client subsampling; all 100 clients participate per round. CryptoFair-FL utilizes threshold decryption in a $k$-of-$n$ scheme and introduces Laplace noise per client ($\Delta_s / \varepsilon$, $\Delta_s = 1$) for differential privacy. Privacy accounting leverages advanced composition via the moment accountant mechanism, consuming total privacy budget $\varepsilon$ over all rounds.

## 7. Baseline Protocols and Key Results

FedFair-100 provides comparative reference for several federated learning paradigms:

| Baseline                | $\Delta_{\mathrm{DP}}$ (at $\varepsilon = 1.0$) | AUROC      |
|-------------------------|---------------------------------------------------|------------|
| FedAvg                  | $\approx 0.182$                                  | $\approx 0.86$ |
| Local Fair              | $\approx 0.095$                                  | $\approx 0.84$ |
| SecAgg-NoFair           | Uncontrolled                                     | —          |
| CryptoFair-FL           | $0.033$                                          | $0.85$     |
| Centralized Fair        | $0.018$                                          | $0.86$     |

A tradeoff curve for $\Delta_\mathrm{DP}$ vs. $\varepsilon$ for CryptoFair-FL on FedFair-100 demonstrates monotonic improvement in fairness violation with higher privacy budgets:

- $\varepsilon=0.25\rightarrow\Delta_{\mathrm{DP}}\approx0.182$
- $\varepsilon=0.50\rightarrow\Delta_{\mathrm{DP}}\approx0.091$
- $\varepsilon=0.75\rightarrow\Delta_{\mathrm{DP}}\approx0.054$
- $\varepsilon=1.00\rightarrow\Delta_{\mathrm{DP}}\approx0.033$
- $\varepsilon=1.50\rightarrow\Delta_{\mathrm{DP}}\approx0.022$
- $\varepsilon=2.00\rightarrow\Delta_{\mathrm{DP}}\approx0.017$

Experimental results demonstrate that CryptoFair-FL achieves near-centralized fairness under privacy constraints, with only 2.3× computational overhead, and remains robust to attribute inference (adversarial probability < 0.05) [2601.12447]. The baselines characterize the landscape for federated fairness, privacy, and utility under heterogeneous data splits, establishing FedFair-100 as an authoritative benchmark for privacy–fairness tradeoff analysis in federated learning.

Source: https://www.emergentmind.com/topics/fedfair-100-benchmark