BiasGym: Modular Bias Benchmarking

• BiasGym is a modular benchmarking platform that unifies bias detection, analysis, and mitigation in machine learning systems.
• It aggregates diverse methodologies such as inference-free detection, density ratio estimation, and causal de-correlation to enhance fairness evaluations.
• The platform standardizes evaluation metrics and supports integration with existing pipelines across vision, NLP, and statistical domains.

BiasGym is a modular benchmarking and diagnostic platform designed to provide comprehensive, reproducible evaluation and mitigation of bias in machine learning models. Its framework aggregates diverse methodologies for bias detection, analysis, and intervention, supporting both supervised and weakly supervised settings. BiasGym serves to streamline bias research workflows, facilitate empirical comparisons, and integrate state-of-the-art bias mitigation algorithms and evaluation protocols for both practitioners and researchers.

1. Conceptual Architecture of BiasGym

BiasGym is constructed as a benchmarking toolset that unifies bias detection, exploration, and mitigation pipelines through modular components, which can be flexibly configured and combined. Core modules include:

• Bias Detection: Enables automated identification of spurious correlations and underrepresented subgroups, employing both inference-based and inference-free strategies.
• Bias Analysis and Exploration: Supports fine-grained exploration of latent bias structures, including the partitioning of data into group-level subpopulations.
• Bias Mitigation: Includes a suite of algorithms for distribution balancing, causal intervention, reweighting, data augmentation, and fairness-oriented training.
• Metric Evaluation: Implements standardized metrics for both predictive performance and fairness, such as Equal Opportunity Difference, Average Odds Difference, Group Robustness, and FID scores for generative models.
• Integration Layer: Facilitates interaction with existing learning pipelines and external datasets, supporting both vision and NLP modalities.

This architectural modularity allows users to benchmark algorithms under different bias scenarios, compare mitigation techniques, and analyze trade-offs in robustness and fairness.

2. Bias Detection Methodologies

BiasGym incorporates both input-level and model-level diagnosis, synthesizing approaches from recent literature:

• Inference-Free Detection (Serna et al., 2021): Bias can be detected directly from model weights without inference. By training a bias detector on large databases of models exhibiting varying degrees and types of bias, BiasGym can audit models for latent bias by analyzing learned parameters alone (e.g., through 1×1 convolutional, pooling, and dense layers).
• Weakly Supervised Density Ratio Estimation (Choi et al., 2019): Uses a small reference dataset to train a binary classifier that distinguishes between biased and reference samples. The output is converted to importance weights via the density ratio formula:

$$w(x) = \gamma \cdot \frac{c^*(Y=1|x)}{1 - c^*(Y=1|x)}$$

where $c^*$ is the Bayes-optimal classifier probability. This enables automatic detection and quantification of bias for subsequent mitigation.

• Bias Exploration via Overfitting (BEO) (Zhao et al., 11 May 2025): Exploits overfitting tendencies of ERM-trained models by training on small, biased subsets, then capturing highly confident predictions to produce pseudo bias labels and partition data into fine-grained latent subgroups.

3. Bias Mitigation Strategies

BiasGym benchmarks and integrates advanced bias mitigation algorithms:

• Importance Weighting with Density Ratios (Choi et al., 2019): Trains generative models using reweighted objectives that "undo" dataset bias, efficiently leveraging both large biased datasets and small unbiased references.
• Fine-Grained Class-Conditional Distribution Balancing (FG-CCDB) (Zhao et al., 11 May 2025): Matches marginal and class-conditional distributions at the group or subgroup level rather than assuming unimodal Gaussian approximations. Reweights samples within each latent subgroup:

$w_{i,j} = \frac{p(s=i)}{p(s=i|y=j)}$

Samples in subgroup $(i,j)$ get shared weights, achieving stronger mitigation of spurious correlations.

• Causal de-correlation (Xiao et al., 2023): Removes causal dependence of sensitive features on the label via Average Causal Effect (ACE) estimation and targeted mutation operations, optimized via multi-objective frameworks considering both performance (e.g., F1 score) and fairness metrics (e.g., EOD, AOD).
• Data Augmentation (BiaSwap) (Kim et al., 2021): Unsupervised augmentation-by-swapping, using a biased classifier trained with generalized cross-entropy to identify bias-guiding/contrary samples, then generating bias-swapped images via patch-level CAM-guided style transfer.
• Balanced Training Objectives (Han et al., 2021): Rebalances loss across demographic groups and target classes, using explicit reweighting terms to minimize true positive rate disparities, combined with gated architectures and Bayesian input perturbation for improved fairness.

4. Evaluation Protocols and Metrics

BiasGym standardizes evaluation across multiple axes:

• Performance Metrics: Accuracy, Precision, Recall, F1 Score
• Fairness Metrics: Equal Opportunity Difference, Average Odds Difference, Statistical Parity Difference, group-wise classification rates
• Distribution Similarity: FID scores for generative sample quality, WEAT (Word Embedding Association Test)
• Robustness: Worst-group accuracy in binary and multi-class tasks
• Variance Analysis: Particularly for aggregate queries (e.g., AVG visit duration; see (Zeighami et al., 17 Feb 2024))

BiasGym facilitates comparative studies across benchmark datasets, such as Adult Income, Compas, and domain-specific data (location, images, text), accumulating results to identify best-performing methods for mixed fairness-performance scenarios.

5. Practical Applications and Case Studies

BiasGym enables targeted bias audits and interventions in diverse domains:

• Vision Models: Gender and ethnicity bias detection in face classifiers (Serna et al., 2021), fairness-oriented generative models via reference sampling (Choi et al., 2019), augmenting data via style swapping (Kim et al., 2021).
• Natural Language Processing: Balanced training for author-level demographic fairness (Han et al., 2021), gender debiasing in word embeddings using hyperbolic geometry metrics (Kumar et al., 2021).
• Population Statistics: Neural estimation of population-level queries from biased mobile location data, improving accuracy and equity in policy-relevant scenarios (Zeighami et al., 17 Feb 2024).
• Causal Auditing: Mutation-guided fairness optimization for sensitive decision-making in domains such as finance, healthcare, and justice (Xiao et al., 2023).

For each task, BiasGym enables systematic exploration of group-level disparities and the effectiveness of debiasing procedures under controlled settings.

6. Integration and Extensibility

BiasGym is designed to be extensible:

• Algorithm-agnostic Compatibility: Through pre-processing, in-processing, and post-processing modules, BiasGym can benchmark and plug in any bias mitigation algorithm, including adversarial debiasing, reweighting, or ensemble methods.
• Annotation-free Operation: Algorithms such as BEO (Zhao et al., 11 May 2025) allow BiasGym to operate without explicit bias annotations by generating pseudo-labels through model overfitting signals.
• Database Generation and Auditing: Supports creation of large “biased model” databases for model-level audit (as in IFBiD (Serna et al., 2021)), facilitating transfer learning approaches to bias detection and cross-domain evaluations.

7. Future Directions and Implications

BiasGym provides a foundation for continual research and benchmarking advances:

• Adoption of causality-based interventions in broader algorithmic contexts to achieve more robust fairness.
• Expansion of auxiliary feature sets (especially for population-level estimation models (Zeighami et al., 17 Feb 2024)).
• Automated, annotation-free bias detection scaling to larger models and multi-modal data.
• Harmonized evaluation protocols, facilitating reproducible research and standardization of bias mitigation reporting.
• Integration with interpretability modules to elucidate learned bias representations and aid policy compliance.

BiasGym synthesizes methodological advances from recent literature, offering a comprehensive environment for detecting, analyzing, and mitigating bias in machine learning systems across diverse data modalities and application domains.