Insufficiently Justified Disparate Impact: A New Criterion for Subgroup Fairness
Abstract: In this paper, we develop a new criterion, "insufficiently justified disparate impact" (IJDI), for assessing whether recommendations (binarized predictions) made by an algorithmic decision support tool are fair. Our novel, utility-based IJDI criterion evaluates false positive and false negative error rate imbalances, identifying statistically significant disparities between groups which are present even when adjusting for group-level differences in base rates. We describe a novel IJDI-Scan approach which can efficiently identify the intersectional subpopulations, defined across multiple observed attributes of the data, with the most significant IJDI. To evaluate IJDI-Scan's performance, we conduct experiments on both simulated and real-world data, including recidivism risk assessment and credit scoring. Further, we implement and evaluate approaches to mitigating IJDI for the detected subpopulations in these domains.
- Machine bias. In Ethics of Data and Analytics. Auerbach Publications, 254–264.
- Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems. 4349–4357.
- J. Buolamwini and T. Gebru. 2018. Gender shades: intersectional accuracy disparities in commercial gender classification. In Proc. 1st Conf. on Fairness, Accountability and Transparency, PMLR 81. 77–91.
- Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5, 2 (2017), 153–163.
- Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018).
- Algorithmic Decision Making and the Cost of Fairness. In Proc. 23rd Conf. on Knowledge Discovery and Data Mining.
- An analysis of the New York City Police Department’s “stop-and-frisk” policy in the context of claims of racial bias. J. Amer. Statist. Assoc. 102 (2007), 813–823.
- Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems. 3323–3331.
- Multicalibration: Calibration for the (Computationally-Identifiable) Masses. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 1939–1948.
- Lily Hu and Yiling Chen. 2020. Fair Classification and Social Welfare. In Proc. Conference on Fairness, Accountability, and Transparency. 535–545.
- Omitted and included variable bias in tests for disparate impact. Working paper.
- Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning. PMLR, 2564–2572.
- Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. ACM, 247–254.
- A survey on bias and fairness in machine learning. Comput. Surveys 54, 6 (2021), 1–35.
- On Fairness and Calibration. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2017/file/b8b9c74ac526fffbeb2d39ab038d1cd7-Paper.pdf
- Fairness Constraints: A Flexible Approach for Fair Classification. Journal of Machine Learning Research 20, 75 (2019), 1–42.
- Zhe Zhang and Daniel B. Neill. 2016. Identifying significant predictive bias in classifiers. Presented at Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML). arXiv preprint arXiv:1611.08292 (2016).
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.