FairPrep: Data-Centric Fairness Framework

• FairPrep is a data-centric fairness program that treats the entire pre-processing lifecycle—aspects like cleaning, encoding, scaling, and imputation—as key to achieving fair outcomes.
• It offers multiple frameworks ranging from developer-centric empirical studies to causal analyses and benchmarking systems, ensuring fairness is integrated before model training.
• The approaches emphasize rigorous data isolation, systematic componentization, and stage-wise audits, balancing predictive utility with fairness metrics.

Searching arXiv for the cited FairPrep-related papers to ground the article. FairPrep is a label that appears in the arXiv fairness literature in several distinct but closely related senses, all centered on the claim that fairness is not only a property of a trained predictor but also of the upstream data lifecycle. In one usage, FairPrep is a developer-centered framework for rigorous empirical studies of fairness-enhancing interventions; in later usages, it denotes a causal analysis of preprocessing stages, a canonical transformation for simultaneous inter- and within-group fairness, and a modular benchmarking environment for fairness-aware pre-processing on tabular data (Schelter et al., 2019, Biswas et al., 2021, Lazri et al., 2023, Oldfield et al., 21 Aug 2025). Across these usages, the unifying theme is that data cleaning, encoding, scaling, resampling, representation repair, and evaluation protocol can materially change both model utility and formal fairness metrics before any in-processing or post-processing method is applied.

1. Terminological scope and core idea

The literature does not attach FairPrep to a single standardized algorithm. Rather, the name is used for several data-centric frameworks that elevate pre-model operations to first-class objects of fairness analysis. The 2019 framework "FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions" formalizes experimental best practices for fairness evaluation (Schelter et al., 2019). "Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline" studies the fairness contribution of individual transformers inside an ordered pipeline (Biswas et al., 2021). "A Canonical Data Transformation for Achieving Inter- and Within-group Fairness" uses a group-specific mapping into a canonical domain so that group score distributions coincide while within-group score relationships are preserved (Lazri et al., 2023). "Revisiting Pre-processing Group Fairness: A Modular Benchmarking Framework" presents an extensible benchmarking system built on AIF360 (Oldfield et al., 21 Aug 2025).

Despite these differences, the shared premise is stable. FairPrep-style work treats the dataset, preprocessing stack, and evaluation protocol as fairness-bearing objects. That premise departs from purely model-centric mitigation and makes fairness analyzable at the level of data isolation, missing-value handling, feature transformation, class balancing, distribution repair, and threshold selection. A plausible implication is that FairPrep is best understood as a data-centric research program rather than as one monolithic method.

2. Developer-centered framework for empirical fairness studies

In the 2019 usage, FairPrep is a design and evaluation framework intended to help data scientists follow best practices in software engineering and machine learning (Schelter et al., 2019). Its motivation is explicitly upstream: bias can be introduced or amplified by data integration, cleaning, feature engineering, resampling, and evaluation structure, while critical opportunities for human oversight and bias control are missed if attention begins only at model training.

The framework is organized around three design principles. The first is Data Isolation (Inversion of Control): training, validation, and test sets are strictly separated; all data transformations are fit only on training data; and user code never sees the test set until final evaluation. The second is Componentization and Extensibility, with abstract interfaces for MissingValueHandler, FeatureTransformer, Preprocessor, Learner, and Postprocessor. The third is an explicit lifecycle with three phases: a Train+Validation Loop, Best Model Selection, and Held-Out Test Evaluation.

Methodologically, FairPrep incorporates upstream operations such as complete-case analysis, simple imputation, and model-based imputation via Datawig; scaling through StandardScaler or MinMaxScaler; and hyperparameter tuning through grid search with $k$-fold cross-validation. By construction, transformers are fit on the training split and then applied to validation and test data. The default model-selection objective is cross-validated accuracy,

$$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$

although alternate multi-objective selection functions can optimize fairness metrics as well (Schelter et al., 2019).

FairPrep evaluates both performance and fairness. The reported fairness metrics include disparate impact,

$$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$

and equalized-odds-style quantities such as false-positive-rate difference and false-negative-rate difference. Empirically, the framework is used on German Credit, Adult Income, Ricci, and ProPublica COMPAS with combinations of baseline learners, pre-processing interventions, and post-processing interventions. The reported findings are that hyperparameter tuning reduces variability, feature scaling matters especially for SGD-based logistic regression, and missing-value imputation can retain incomplete records without degrading fairness; the authors also identify shortcomings in prior studies, including hyperparameter selection on the test set and feature transformations applied before train/test splits (Schelter et al., 2019).

3. Compositional fairness of preprocessing stages

The 2021 causal formulation extends FairPrep from an experimental scaffold to a theory of how preprocessing stages contribute to the global fairness of a pipeline (Biswas et al., 2021). A pipeline is modeled as

$$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$

where $S_{1},\dots,S_{m-1}$ are preprocessing transformers and $S_m$ is the classifier. For a given stage $S_k$, fairness impact is measured by comparing the full pipeline to a counterfactual pipeline in which $S_k$ is removed or replaced by a canonical alternative.

The paper defines stage-local fairness metrics $\mathsf{SF}_M(S_k)$ for $$M\in\{\mathrm{SPD},\mathrm{EOD},\mathrm{AOD},\mathrm{ERD}\}$$. The central result is a composition theorem: the change in global fairness from the raw-data pipeline to the $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$0-stage pipeline is the sum of the local contributions of the first $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$1 transformers. For statistical parity difference,

$$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$2

and analogous additivity holds for EOD, AOD, and ERD (Biswas et al., 2021). This gives a formal account of how local preprocessing decisions compose into end-to-end fairness outcomes.

The empirical study spans 37 pipelines over five datasets and surveys 69 stages across 11 categories, including missing-value processing, encoders, scalers, nonlinear transforms, dimensionality reduction, feature selection, sampling, and custom feature engineering. Several recurring patterns are reported. Row-dropping induces moderate to high bias because it disproportionately removes minority-group instances, whereas imputation is much fairer. LabelEncoder often incurs bias due to arbitrary ordinal structure, while OneHotEncoder is neutral in all reported runs. StandardScaler, RobustScaler, and MinMaxScaler are typically fair, but aggressive feature pruning, SMOTE, undersampling, and custom feature engineering can introduce substantial bias (Biswas et al., 2021).

This strand of FairPrep is important because it shifts fairness diagnosis from aggregate model outcomes to transformer-local causes. In practical terms, it supports a workflow in which each stage is audited, replaced, or neutralized downstream so that the final pipeline meets a fairness budget.

4. Canonical transformation for inter- and within-group fairness

A different FairPrep formulation addresses a tension between inter-group and within-group fairness (Lazri et al., 2023). Standard group-fairness procedures may improve parity across protected groups while distorting relative treatment among individuals inside the same group. The proposed remedy is a pre-processing map $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$3 for each group $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$4, coupled with a fixed baseline scoring model $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$5, such that all groups are mapped into an inter-group-fair canonical domain and within-group score relationships are approximately preserved.

Inter-group fairness is framed as threshold-invariant demographic parity (TIDP), quantified by

$$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$6

Within-group fairness is defined through the preservation of pairwise signed score distances within each demographic group, and measured by $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$7, the fraction of within-group pairs whose relative order or spacing changes by more than $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$8 when moving from the original score mapping to the repaired one (Lazri et al., 2023).

The algorithm has three phases. First, a baseline predictor is trained. Second, each group’s empirical score CDF is histogram-matched to the population CDF, yielding a monotonic score-domain alignment. Third, feature correspondences are extended to new data through a k-d tree over each group’s training features: for a query point, $$\theta^* \;=\;\arg\max_{\theta\in\Theta}~\frac1k\sum_{j=1}^k \mathrm{Acc}_{\mathrm{fold}\;j}(\theta),$$9 nearest neighbors are found, inverse-distance weights are computed, and the corresponding population representatives are interpolated to produce $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$0. The reported complexity is model-dependent for baseline training, $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$1 for histogram matching, $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$2 for correspondences if scores are sorted, $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$3 for building each k-d tree, and $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$4 per test query (Lazri et al., 2023).

The theoretical rationale is direct. Histogram matching is monotonic, so within-group score order is preserved in the score domain; because each group’s score CDF is aligned to the population CDF, the transformed score distributions coincide across groups, yielding TIDP up to sampling error. Experiments on COMPAS and Law School compare this pre-processing framework with two regularization-based methods. Reported results include lower $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$5 and lower $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$6 at smaller accuracy and AUC cost than the regularization baselines, with an example on COMPAS + logistic regression showing baseline AUC $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$7, pre-processing AUC $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$8, and regularization AUC drops greater than $$\mathrm{DI} \;=\; \frac{\Pr(\hat Y=1\mid A=\text{unpriv})} {\Pr(\hat Y=1\mid A=\text{priv})},$$9 (Lazri et al., 2023).

5. Modular benchmarking framework built on AIF360

In the 2025 usage, FairPrep becomes an extensible benchmarking system for fairness-aware pre-processing on tabular data (Oldfield et al., 21 Aug 2025). The framework has two stages—Pre-processing and Benchmarking—connected through a YAML-based batch interface. Its architectural goal is not to propose a new mitigation criterion but to standardize the evaluation of existing pre-processing methods, datasets, models, thresholds, and random seeds.

Datasets are implemented as subclasses of AIF360’s StandardDataset, with five built-in datasets: Adult Census, Bank Marketing, COMPAS, German Credit, and MEPS Panel 21. Preprocessing interventions wrap four AIF360 algorithms: Reweighing (RW), Learned Fair Representations (LFR), Disparate Impact Remover (DIR), and Optimised Pre-processing (OPP). Predictive models are any scikit-learn–style estimators, with built-ins including LogisticRegression, RandomForestClassifier, and SVM. The runner executes all combinations specified in YAML, performing fit/transform, model training, threshold selection on validation data, test evaluation, and generation of CSV, JSON, and plot-based reports (Oldfield et al., 21 Aug 2025).

The framework distinguishes data-level and model-level metrics. At the preprocessing stage it computes quantities such as base rate, statistical parity difference, disparate impact,

$$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$0

for reweighing, and $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$1-NN consistency. At the benchmarking stage it reports accuracy, balanced accuracy, AUC, demographic parity difference, equal opportunity difference, equalized odds difference, and the Theil index (Oldfield et al., 21 Aug 2025).

The Adult Census excerpt illustrates the intended use. RW leaves the base rate unchanged but sets disparate impact to $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$2 and SPD to $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$3; LFR shifts the base rate to $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$4 and raises consistency to $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$5; DIR has negligible effect on DI and SPD; and OPP improves DI and SPD with moderate label distortion. In the downstream threshold sweep on Adult + logistic regression, the paper reports that after RW, fairness metrics remain near $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$6 across thresholds while balanced accuracy remains stable. This later FairPrep therefore functions as an experimental control plane for comparing pre-processing methods under reproducible conditions (Oldfield et al., 21 Aug 2025).

6. Related methods, boundary cases, and recurrent limitations

FairPrep sits inside a broader ecosystem of pre-processing methods that formalize different fairness targets. FLAP learns counterfactually fair decisions from biased training data by applying either orthogonalization,

$$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$7

or a marginal-distribution mapping $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$8, and proves that under stated structural assumptions counterfactual fairness is equivalent to $$\mathcal P \;=\;\bigl(S_{1},S_{2},\dots,S_{m-1},S_{m}\bigr),$$9 (Chen et al., 2022). FairBalance derives equalized-odds pre-processing from weighted empirical loss minimization and assigns weights

$S_{1},\dots,S_{m-1}$0

so that each demographic group’s weighted positive-to-negative ratio is $S_{1},\dots,S_{m-1}$1, yielding zero average odds difference on the training data (Yu et al., 2021). FairRR formulates pre-processing as a Randomized Response design over group-specific label-flip probabilities, solves a four-variable linear program, and targets demographic parity, equal opportunity, or predictive equality at a prescribed disparity level $S_{1},\dots,S_{m-1}$2 (Zeng et al., 2024). LatentPre augments causal fairness policies with a discrete latent variable $S_{1},\dots,S_{m-1}$3 and estimates it by an expectation-maximization procedure so that fair repair remains possible when the observed attribute space is ambiguous or incomplete (Zheng et al., 27 Mar 2026). Task-tailored pre-processing formulates the pre-processing map as a bilevel optimization that trades off task loss against the Hirschfeld–Gebelein–Rényi correlation between predictions and sensitive attributes, and derives downstream fairness-improvement and utility-preservation bounds for arbitrary supervised learners (Sohn et al., 17 Jan 2026).

A recurring caution in this literature is that fairness gains from preprocessing can be illusory if predictive signal is destroyed. In credit-risk modeling, Truncated SVD was reported to keep accuracy at $S_{1},\dots,S_{m-1}$4 while lowering AUC from $S_{1},\dots,S_{m-1}$5 to $S_{1},\dots,S_{m-1}$6, with recall collapsing to $S_{1},\dots,S_{m-1}$7; the resulting $S_{1},\dots,S_{m-1}$8 arose because both groups had TPR $S_{1},\dots,S_{m-1}$9, which the paper explicitly characterizes as trivial “fairness” (Wu, 2024). This boundary case is significant for FairPrep-style work because it shows that pre-processing must be assessed jointly with utility and error profiles rather than through parity metrics alone.

Taken together, these strands position FairPrep as a data-centric fairness perspective with several concrete realizations: rigorous lifecycle-aware experimentation, stage-wise causal diagnosis, canonical transformation for simultaneous inter- and within-group fairness, and standardized benchmarking of pre-processing methods. The literature therefore treats fairness not merely as a constraint on a learned hypothesis, but as a property of the full upstream pipeline that generates the hypothesis.