Fairness Gap (FG) in Algorithmic Fairness

Updated 30 June 2026

Fairness Gap (FG) is a quantitative measure that captures the disparity between the best- and worst-performing groups based on a chosen metric.
It is instantiated across diverse domains such as classification, federated learning, image restoration, and generative modeling using metrics like AUC, loss differences, and f-divergences.
FG informs fairness interventions by highlighting maximum group disparities, guiding methods in mitigating bias while balancing accuracy and fairness trade-offs.

The Fairness Gap (FG) inhabits a central role in contemporary algorithmic fairness research, providing a quantitative measure of the disparity in outcomes, predictions, or quality of service delivered by models to different subgroups defined by sensitive attributes. FG encompasses a family of metrics—often instantiated as the span between best- and worst-performing groups with respect to a chosen performance, distance, or divergence statistic. Its technical definitions and uses vary according to the task (classification, federated learning, generative modeling, image restoration, etc.) and the level of granularity (prediction, explanation, distributional fidelity), but consistently, FG operationalizes the notion of “maximum unfairness” within a defined group structure.

1. Core Definitions and Formalizations

The FG is most generically specified as the gap between the most and least advantaged groups according to a group-wise performance or divergence criterion. Given a set of subgroups $G = \{g_1, ..., g_K\}$ and a group-level metric $M_g$ induced by some model, the canonical definition is:

$\mathrm{FG} = \max_{g \in G} M_g - \min_{g \in G} M_g$

This form appears in varied contexts:

AUC Gap for Classification: $\mathrm{FG} = \max_{g \in G} \mathrm{AUC}_g - \min_{g \in G} \mathrm{AUC}_g$ , where $\mathrm{AUC}_g$ is the test AUC for group $g$ (Lee et al., 2023).
Loss Gap in Federated Learning: For per-client risk $L_k$ , $\mathrm{FG}_{\text{max--min}} = \max_{k} [L_k(\theta_{\text{global}}) - L_k^*] - \min_{k} [L_k(\theta_{\text{global}}) - L_k^*]$ (Erraji et al., 31 Mar 2026).
Perceptual Fairness in Image Restoration: $\mathrm{FG}_d = \max_{a} \mathrm{GPI}_d(a) - \min_{a} \mathrm{GPI}_d(a)$ , with $\mathrm{GPI}_d$ a group-wise statistical distance, e.g., Wasserstein or Total Variation, between the true and restored distributions (Ohayon et al., 2024).
Equalized Generative Treatment (EGT) in Generative Models: $M_g$ 0, leveraging $M_g$ 1-divergences as the quality metric (Verine et al., 9 Feb 2026).

FG unifies numerous fairness definitions (demographic parity, equalized odds, explanation fairness, etc.) as specific instantiations or related measures.

2. Instantiations Across Domains

FG’s instantiation adapts to the structure and desiderata of the relevant application area:

Classification (AUC Gap): Captures the worst-case discrepancy in ranking performance over any collection of (possibly overlapping) subgroups. FG is threshold-agnostic and model-agnostic, supporting intersectional and non-binary subgroups (Lee et al., 2023).
Federated Learning (Loss Gap): Measures the spread in excess risk (gap between global and local optimal model) across clients. FG operationalizes a “relative improvement” notion, targeting fairness in the benefit received as opposed to absolute parity (Erraji et al., 31 Mar 2026).
Image Restoration (Perceptual Fairness): FG is the span in statistical distances between restored and true group distributions (the Group Perceptual Index, GPI). This captures subtle distributional discrepancies even when hit-rate-based measures are parity (Ohayon et al., 2024).
Generative Modeling (EGT): Ensures parity of $M_g$ 2-divergence (e.g., precision/recall, Jensen–Shannon) between the generated and real data distributions for each group, thus enforcing equal generation fidelity irrespective of group size (Verine et al., 9 Feb 2026).
Matched Demographic Parity (Fairness Through Matching): Defines FG via the average predicted-output gap on matched instances, with the transport map controlling the granularity and mechanism of group alignment (Kim et al., 6 Jan 2025).
Distance Covariance (DC/CDC): Employs the difference in empirical (conditional) distance covariance between the model’s output and sensitive attributes as a fairness gap, capturing both linear and nonlinear dependencies (Huang et al., 2024).
Explanation Fairness: Ratio-based and value-based group gaps over explanation quality (not merely predictive outputs), enabling detection and mitigation of “procedure-oriented” as opposed to result-oriented biases (Zhao et al., 2022).
Surrogate-Fairness Gap: Quantifies the difference between the fairness of a surrogate function (e.g., smooth approximation to the indicator in parity constraints) and the true fairness criterion, driving the design of surrogates and rebalancing algorithms (Yao et al., 2023).

3. Methodologies for FG Calculation and Mitigation

The computation of FG follows a common workflow: define groups, compute the relevant group-wise metric, identify maxima and minima, and report the difference.

Stepwise Procedure (AUC Gap): (1) Define groups; (2) Partition the validation set; (3) Compute group-wise AUCs; (4) Evaluate FG as the absolute difference between extremal AUCs (Lee et al., 2023).
Algorithmic Mitigation: Techniques include in-processing regularizers (e.g., variance of loss gaps in federated learning (Erraji et al., 31 Mar 2026)), explicit constraint satisfaction (e.g., min-max optimization of worst-group $M_g$ 3-divergence in EGT (Verine et al., 9 Feb 2026)), and rebalancing or adaptation of functional surrogates (e.g., Balanced Surrogate for Demographic Parity (Yao et al., 2023)).
Distributional Matching: For Perceptual Fairness, group-conditioned critics or fairness regularizers on distributional gaps (such as FG of GPI) are incorporated as additional loss terms to minimize group-specific statistical distances (Ohayon et al., 2024).
Matching and Optimal Transport: The Fairness Through Matching paradigm introduces transport maps, either marginal (input) or joint (input-label), as aligners between groups, using average prediction differences on matched instances to define and minimize FG (Kim et al., 6 Jan 2025).
Statistical Independence Penalties: Empirical (conditional) distance covariance is employed as a differentiable penalty, plugged in directly to the learning objective to shrink the fairness gap (Huang et al., 2024).
Explanation-Fairness Objectives: Multi-term losses balance prediction accuracy, traditional representation-fairness, and explanation-quality gap constraints (Zhao et al., 2022).

4. Theoretical Properties and Interpretability

FG-based metrics possess several desirable and distinctive mathematical properties:

Threshold- and Model-Agnosticity: Metrics like AUC Gap and f-divergence-based FG do not depend on arbitrary choice of thresholds or model internals, making them universally comparable (Lee et al., 2023, Verine et al., 9 Feb 2026).
Worst-Case Focus: By construction, FG reflects the maximum attainable disparity across the set of groups, corresponding to robust fairness guarantees in scenarios with sensitive or intersectional subgroups (Lee et al., 2023, Ohayon et al., 2024).
Distributional Sensitivity: For distribution-based FGs (image restoration, generative models), the measure can detect subtle qualitative disparities not visible under coarser (hit rate, parity) statistics (Ohayon et al., 2024, Verine et al., 9 Feb 2026).
Simultaneous Guarantees: Some FG instantiations (e.g., MDP gap in matching) can provide upper bounds for classical parity gaps and their subset-specific analogs (Kim et al., 6 Jan 2025).
Trade-offs and Inevitable Lower Bounds: Enforcing low FG can couple model performance to the hardest-to-optimize group and, in some distributional settings, make zero FG unattainable under real-world corruption scenarios (Ohayon et al., 2024, Verine et al., 9 Feb 2026).
Stability and Bias Decomposition: Surrogate-based FGs highlight the role of large-margin points and the inherent discordance between bounded, smooth surrogates and indicator-based constraints (Yao et al., 2023).

5. Practical Thresholds and Interpretive Guidance

Interpretation of FG is context-dependent and may incorporate domain risk tolerances and regulatory frameworks.

AUC Gap Thresholds: Empirically, practitioners situate FG < 0.02 as negligible disparity, 0.02–0.05 as a small actionable gap, 0.05–0.10 as moderate, and FG ≥ 0.10 as a large fairness concern (Lee et al., 2023). These cutoffs align with observed iteration cycles where debiasing or reweighting reduce FG by 20–50%.
Distributional Gaps: For GPI-based FG, guidance hinges on the scale of typical groupwise perceptual distances (e.g., KID/Wasserstein), with reductions from 0.035 to 0.008 in face restoration cited as significant improvements (Ohayon et al., 2024).
Explanation-Fairness Gaps: Lower values of Δ_REF and Δ_VEF correlate with higher procedural parity; trade-off plots and Pareto frontiers jointly illustrate regimes where explanation-fair objectives help or hinder accuracy (Zhao et al., 2022).

6. Empirical Results and Case Analyses

FG-driven evaluations have been extensively validated in diverse experimental settings:

Education Outcomes (AUC Gap): In models predicting math achievement, cross-institutional transfer learning reduced FG from 0.15 to 0.06; across ten research teams, FG ranged from 0.04 to 0.18 (Lee et al., 2023).
Federated Learning: EAGLE reduced the variance in loss gaps by 30% or more compared to FedAvg, with only minimal global accuracy degradation (Erraji et al., 31 Mar 2026).
Face Restoration: Fairness regularization decreased FG (measured by KID) fourfold without visible loss in PSNR or perceptual quality (Ohayon et al., 2024).
Generative Modeling: Min-max fine-tuning with respect to worst-group $M_g$ 4-divergence consistently achieved the smallest reported FG, outperforming reweighting and conditional approaches in both image and language generation tasks (Verine et al., 9 Feb 2026).
Algorithmic Fairness by Matching: FTM achieved the lowest subset and classical group-fairness gaps across benchmarks, and demonstrates higher consistency metrics (individual fairness) than “reduction,” “regularization,” or adversarial baselines (Kim et al., 6 Jan 2025).
Distance Covariance Penalties: Empirical DC/CDC penalties cut fairness gaps on tabular and image datasets by orders of magnitude relative to no-fairness or maximal correlation alternatives (Huang et al., 2024).
Explanation Bias: CFA reduced Δ_REF (ratio-based explanation fairness) by a factor of 2–3 compared to an unconstrained baseline while improving statistical parity (Δ_SP) (Zhao et al., 2022).

7. Limitations and Ongoing Directions

FG-centric fairness metrics, while powerful, are not panaceas.

Lack of Error-Type Decomposition: FG often aggregates over error types (e.g., FPR vs. FNR); decomposition requires supplementary metrics (Lee et al., 2023).
No Causal Insight: FG does not specify why disparities arise (data imbalance, distribution drift, model class constraints) (Lee et al., 2023).
Brittleness in Surrogates: Surrogate-based FGs may be unstable or misleading if surrogate functions are poorly chosen; “Balanced Surrogate” approaches partially address this issue (Yao et al., 2023).
Incompatibility with Severe Group Overlap: In distributional settings, severe measurement corruption or domain shift may make perfect FG unattainable due to overlapping supports (Ohayon et al., 2024).
Inefficacy of Proportion-Based Controls: Empirical studies demonstrate that matching group sample proportions (MGO, EGO) does not ensure small FG in generative models (Verine et al., 9 Feb 2026).
Interpretability to Practitioners: Certain FG formulations (AUC, f-divergence, GPI-based) may be less immediately interpretable to stakeholders accustomed to threshold-based or outcome-oriented fairness (Lee et al., 2023).

Future developments concern robust FG metrics for complex, structured outputs; sharper generalization and estimation guarantees; alignment with causal and intersectional perspectives; and integration with explainability and controllable debiasing mechanisms.