Generalizability of feasibility under relaxed fairness constraints

Ascertain the extent to which the observed feasibility of achieving approximate parity across False Positive Rate, False Negative Rate, and Positive Predictive Value with an epsilon margin-of-error of approximately 0.05 and prevalence differences up to 10–20% generalizes across datasets, application domains, and deployment contexts.

Background

The authors’ analysis and experiments suggest that allowing small margins-of-error (e.g., ε ≤ 0.05) often yields large fairness regions even with moderate prevalence differences between groups. This provides optimistic guidance for practitioners about the possibility of multi-metric fairness.

However, they explicitly state uncertainty about how broadly these particular parameter choices will generalize, indicating the need for systematic investigation across varied contexts to establish robust guidelines.

References

"Further exploration is needed to understand what exactly constitutes small and moderate, but in our analysis we observed that cases with a $5\%$ margin-of-error and prevalence differences up to $10\%$ (and in some cases up to $20\%$) afforded feasible solutions. We are unsure how well these particular settings will generalize, but the larger implication is hopeful."

— The Possibility of Fairness: Revisiting the Impossibility Theorem in Practice  (2302.06347 - Bell et al., 2023) in Section 5, Discussion