FairStacks: Fair Model Stacking

• FairStacks is a convex fair model stacking meta-learner that uses linear combinations of pre-trained classifiers to expand the fairness–accuracy Pareto frontier.
• It employs a constrained convex optimization framework integrating score-bias constraints, regularization, and a bias-budget parameter to enforce group fairness.
• Empirical evaluations demonstrate that FairStacks consistently improves both fairness and accuracy across diverse datasets and benchmark tasks.

FairStacks is a convex “fair model stacking” meta-learner designed to expand and optimize the empirical Pareto frontier of the fairness–accuracy tradeoff in machine learning, enabling practitioners to maximize accuracy for a prescribed level of group fairness. By leveraging linear combinations of pre-trained base classifiers and enforcing a strict score-bias constraint, FairStacks consistently improves both the best-achievable fairness and accuracy on benchmark tasks, providing a model-agnostic and post-processing approach to algorithmic fairness (Little et al., 2022).

1. Formalization of the Fairness–Accuracy Pareto Frontier

Let $H$ denote any finite collection of pre-trained predictors $h \in H$, where each $h$ maps features $x$ to a probability score $\hat h(x) \in [0,1]$ or a hard label $\{0,1\}$. Each model is evaluated by:

• an accuracy measure $\mathrm{Accuracy}(h) \in [0,1]$, increasing in “better” fits (e.g., classification accuracy, $1-\mathrm{MSE}$, etc.)
• a group-fairness measure $\mathrm{Fairness}(h)\in[0,1]$, increasing in “more fair” behavior (e.g., $$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$)

Pareto Optimal Model: A model $h \in H$0 is Pareto-optimal if there is no $h \in H$1 with $h \in H$2 and $h \in H$3, with at least one inequality strict.

TAF Curve: The Tradeoff-between-Fairness-and-Accuracy curve for $h \in H$4 is $h \in H$5, where

$h \in H$6

$h \in H$7 reports the best achievable accuracy by any model in $h \in H$8 meeting at least fairness level $h \in H$9. Practically, this is computed by sorting $h$0 in decreasing fairness and tracking the running maximum of accuracy.

FAUC: For a nonnegative weight function $h$1, the weighted Fairness-Area-Under-the-Curve is

$h$2

Unweighted ($h$3), $h$4 is the ordinary area under the TAF curve. This enables direct, model-agnostic comparison of model families on fairness and accuracy, even when their tradeoff curves cross.

2. Score Bias and Group Fairness Constraint

FairStacks ensembles are constructed as linear combinations of base models $h$5. For two contrast groups $h$6 (e.g., $h$7 vs $h$8 for a binary attribute):

• Score Bias: For weight vector $h$9 and score matrix $x$0,

$x$1

• For each base model $x$2, $x$3. By linearity,

$x$4

The ensemble bias constraint directly encodes any group fairness constraint definable as a difference in expectation between groups.

3. FairStacks Convex Optimization Framework

FairStacks solves the constrained optimization:

$x$5

where:

• $x$6 are training features and labels,
• $x$7 is the vector of base-model scores,
• $x$8 is a convex surrogate loss (e.g., squared error, binomial deviance),
• $x$9 is a ridge penalty for regularization,
• $\hat h(x) \in [0,1]$0 is a bias-budget parameter (smaller $\hat h(x) \in [0,1]$1 enforces stricter fairness).

Alternatively, a penalized form may be solved:

$\hat h(x) \in [0,1]$2

where $\hat h(x) \in [0,1]$3 is varied across a grid to trace the fairness–accuracy frontier.

4. Algorithmic Solution and Implementation

The FairStacks algorithm proceeds as follows:

1. Inputs: Pre-trained base models $\hat h(x) \in [0,1]$4; training data $\hat h(x) \in [0,1]$5; protected groups $\hat h(x) \in [0,1]$6.
2. Precompute:
   • $\hat h(x) \in [0,1]$7 for all $\hat h(x) \in [0,1]$8.
   • $\hat h(x) \in [0,1]$9.
3. Grid search: Select grid $\{0,1\}$0 for bias penalty; grid for $\{0,1\}$1 (via cross-validation).
4. For each $\{0,1\}$2: Solve convex program for weights $\{0,1\}$3 (via Newton or QP solvers):

$\{0,1\}$4

• Compute fairness $\{0,1\}$5; accuracy $\{0,1\}$6 or direct $\{0,1\}$7.
• Accumulate $\{0,1\}$8 pairs to form taf curve and compute area under it for fauc.

1. Output: $\{0,1\}$9, $\mathrm{Accuracy}(h) \in [0,1]$0, ensemble weights $\mathrm{Accuracy}(h) \in [0,1]$1.

5. Theoretical Guarantees

FairStacks provides several formal properties:

• Frontier Expansion (Proposition 1): For original models $\mathrm{Accuracy}(h) \in [0,1]$2 and FairStacks solutions $\mathrm{Accuracy}(h) \in [0,1]$3,

$\mathrm{Accuracy}(h) \in [0,1]$4

Since all $\mathrm{Accuracy}(h) \in [0,1]$5 can be realized as stacking solutions with $\mathrm{Accuracy}(h) \in [0,1]$6, and the stacking set is a superset of $\mathrm{Accuracy}(h) \in [0,1]$7, stacking always expands the Pareto frontier.

• Monotonicity in Fairness (Theorem 2): Under mild assumptions on the decision function $\mathrm{Accuracy}(h) \in [0,1]$8 (e.g., thresholding), the decision-level fairness of weights $\mathrm{Accuracy}(h) \in [0,1]$9 improves (bias decreases) as $1-\mathrm{MSE}$0, so the constraint yields (approximately) monotonic increases in group fairness.

6. Empirical Evaluation

Performance of FairStacks was evaluated on five benchmark datasets:

Dataset	Size / Features	Protected Attributes
Adult Income	$1-\mathrm{MSE}$1k, $1-\mathrm{MSE}$2	gender, race
Bank Marketing	$1-\mathrm{MSE}$3k, $1-\mathrm{MSE}$4	age $1-\mathrm{MSE}$5?
COMPAS Recidivism	$1-\mathrm{MSE}$6k, $1-\mathrm{MSE}$7	race, gender
Default of Credit Card	$1-\mathrm{MSE}$8k, $1-\mathrm{MSE}$9	gender
Communities & Crime	$\mathrm{Fairness}(h)\in[0,1]$0k, $\mathrm{Fairness}(h)\in[0,1]$1	race

Base learners included standard random-forest trees, “minipatch” trees (1,000 sub-sample trees), Scikit-learn/XGBoost classifiers, and three in-processing fairness methods: Adversarial Debiasing [Zhang et al. 2018], Reduction-Based Fair Classification [Agarwal et al. 2018], and Fair Adversarial GBT [Grari et al. 2019]. Data were split $\mathrm{Fairness}(h)\in[0,1]$2 train / $\mathrm{Fairness}(h)\in[0,1]$3 stacking-CV / $\mathrm{Fairness}(h)\in[0,1]$4 test; 5-fold CV for $\mathrm{Fairness}(h)\in[0,1]$5 was performed within the stacking split to optimize the $\mathrm{Fairness}(h)\in[0,1]$6-step-weighted fauc. Brier score was used for $\mathrm{Fairness}(h)\in[0,1]$7 to emphasize calibrated probabilities.

Selected results (FAUC, Demographic Parity, mean $\mathrm{Fairness}(h)\in[0,1]$8 SE, 10 random splits):

Method	Adult (gender)	COMPAS (race)	Default (gender)	C&C (race)
Random Forest	$\mathrm{Fairness}(h)\in[0,1]$9	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$0	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$1	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$2
Zhang et al. (2018)	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$3	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$4	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$5	—
Grari et al. (2019)	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$6	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$7	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$8	—
Agarwal et al. (2018)	$$\mathrm{Fairness}=1-|\text{Demographic Parity gap}|$$9	$h \in H$00	$h \in H$01	—
FS on minipatch trees	$h \in H$02	$h \in H$03	$h \in H$04	$h \in H$05
FS on RF trees	$h \in H$06	$h \in H$07	$h \in H$08	$h \in H$09
FS on all classifiers	$h \in H$10	$h \in H$11	$h \in H$12	$h \in H$13
FS Kitchen-Sink (all)	$h \in H$14	$h \in H$15	$h \in H$16	$h \in H$17

In all settings, FairStacks ensembles dominate competing methods—their taf curves are strictly above all baselines for every fairness level, yielding the highest area (FAUC). The “kitchen-sink” ensemble, pooling all base learners, consistently achieves the best fairness–accuracy balance.

7. Relevance and Context

FairStacks is a simple, entirely model-agnostic, post-processing meta-learner. It can accept any collection of pre-fitted predictors and, by solving a single convex program per fairness budget $h \in H$18, efficiently traces and expands the empirical Pareto frontier between fairness and accuracy. Theoretical guarantees and empirical results demonstrate pointwise expansion of the frontier and consistently higher FAUC on diverse real-world datasets, providing practitioners and researchers with an effective plug-and-play approach to optimize the fairness–accuracy tradeoff (Little et al., 2022).