Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis (2206.00667v3)

Published 1 Jun 2022 in cs.LG and cs.AI

Abstract: Fairness in machine learning has attained significant focus due to the widespread application in high-stake decision-making tasks. Unregulated machine learning classifiers can exhibit bias towards certain demographic groups in data, thus the quantification and mitigation of classifier bias is a central concern in fairness in machine learning. In this paper, we aim to quantify the influence of different features in a dataset on the bias of a classifier. To do this, we introduce the Fairness Influence Function (FIF). This function breaks down bias into its components among individual features and the intersection of multiple features. The key idea is to represent existing group fairness metrics as the difference of the scaled conditional variances in the classifier's prediction and apply a decomposition of variance according to global sensitivity analysis. To estimate FIFs, we instantiate an algorithm FairXplainer that applies variance decomposition of classifier's prediction following local regression. Experiments demonstrate that FairXplainer captures FIFs of individual feature and intersectional features, provides a better approximation of bias based on FIFs, demonstrates higher correlation of FIFs with fairness interventions, and detects changes in bias due to fairness affirmative/punitive actions in the classifier. The code is available at https://github.com/ReAILe/bias-explainer.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Hiring by algorithm: predicting and preventing disparate impact. Available at SSRN, 2016. URL: http://sorelle.friedler.net/papers/SSRN-id2746078.pdf.
  2. Machine bias risk assessments in criminal sentencing. ProPublica, May, 23, 2016.
  3. Explainability for fair machine learning. arXiv preprint arXiv:2010.07389, 2020.
  4. Ai fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias, Oct 2018. URL https://arxiv.org/abs/1810.01943.
  5. Fairness seen as global sensitivity analysis. arXiv preprint arXiv:2103.04613, 2021.
  6. Richard Berk. Accuracy and fairness for juvenile justice risk assessments. Journal of Empirical Legal Studies, 16(1):175–194, 2019.
  7. A survey of bias in machine learning through the prism of statistical parity. The American Statistician, pages 1–11, 2021.
  8. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018.
  9. Optimized pre-processing for discrimination prevention. In Advances in Neural Information Processing Systems, pages 3992–4001, 2017.
  10. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE symposium on security and privacy (SP), pages 598–617. IEEE, 2016.
  11. Carl de Boor. Subroutine package for calculating with b-splines. Technical report, Los Alamos National Lab.(LANL), Los Alamos, NM (United States), 1971.
  12. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.
  13. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
  14. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226, 2012.
  15. The jackknife estimate of variance. The Annals of Statistics, pages 586–596, 1981.
  16. EU Commission. Better regulation toolbox, 2021.
  17. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 259–268, 2015.
  18. Fairness metrics: A comparative analysis. In 2020 IEEE International Conference on Big Data (Big Data), pages 3662–3666. IEEE, 2020.
  19. Justicia: A stochastic SAT approach to formally verify fairness. In Proceedings of AAAI, 2 2021.
  20. Algorithmic fairness verification with graphical models. In Proceedings of AAAI, 2 2022.
  21. Which explanation should i choose? a function approximation perspective to characterizing post hoc explanations. arXiv preprint arXiv:2206.01254, 2022.
  22. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315–3323, 2016.
  23. SALib: An open-source python library for sensitivity analysis. The Journal of Open Source Software, 2(9), jan 2017. doi: 10.21105/joss.00097. URL https://doi.org/10.21105/joss.00097.
  24. Wassily Hoeffding. A class of statistics with asymptotically normal distribution. Breakthroughs in Statistics: Foundations and Basic Theory, pages 308–334, 1992.
  25. Human imperceptible attacks and applications to improve fairness. arXiv preprint arXiv:2111.15603, 2021.
  26. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1–33, 2012.
  27. Decision theory for discrimination-aware classification. In 2012 IEEE 12th International Conference on Data Mining, pages 924–929. IEEE, 2012.
  28. Correlates of perceived fairness and accuracy of performance evaluation. Journal of Applied psychology, 63(6):751, 1978.
  29. Global sensitivity analysis for systems with independent and/or correlated inputs. The journal of physical chemistry A, 114(19):6022–6032, 2010.
  30. Achieving fairness at no utility cost via data reweighing with influence. In International Conference on Machine Learning, pages 12917–12930. PMLR, 2022.
  31. Catherine Loader. Smoothing: local regression techniques. In Handbook of computational statistics, pages 571–596. Springer, 2012.
  32. Clive Loader. Local regression and likelihood. Springer Science & Business Media, 2006.
  33. Scott M Lundberg. Explaining quantitative measures of fairness. In Fair & Responsible AI Workshop@ CHI2020, 2020.
  34. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774, 2017.
  35. From local explanations to global understanding with explainable ai for trees. Nature Machine Intelligence, 2(1):2522–5839, 2020.
  36. Using machine learning in admissions: Reducing human and algorithmic bias in the selection process. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, pages 1323–1323, 2021.
  37. Ann C McGinley. Ricci v. DeStefano: A masculinities theory analysis. Harv. JL & Gender, 33:581, 2010.
  38. Exacerbating algorithmic bias through fairness attacks. arXiv preprint arXiv:2012.08723, 2020.
  39. Explaining algorithmic fairness through fairness-aware causal path decomposition. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1287–1297, 2021.
  40. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  41. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144. ACM, 2016.
  42. Alvin E Roth. The Shapley value: essays in honor of Lloyd S. Shapley. Cambridge University Press, 1988.
  43. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019.
  44. Global sensitivity analysis: the primer. John Wiley & Sons, 2008.
  45. Five ways to ensure that models serve society: a manifesto, 2020.
  46. Larry Schumaker. Spline functions: basic theory. Cambridge University Press, 2007.
  47. Axiomatic characterization of data-driven influence measures for classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 718–725, 2019.
  48. Il’ya M Sobol’. On sensitivity estimation for nonlinear mathematical models. Matematicheskoe modelirovanie, 2(1):112–118, 1990.
  49. Il’ya M Sobol’. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Mathematics and computers in simulation, 55(1-3):271–280, 2001.
  50. IM Sobol. Multidimensional quadrature formulas and haar functions, 1969.
  51. Poisoning attacks on algorithmic fairness. arXiv preprint arXiv:2004.07401, 2020.
  52. Nikolaj Tollenaar and PGM Van der Heijden. Which method predicts recidivism best?: a comparison of statistical, machine learning and data mining predictive models. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176(2):565–584, 2013.
  53. Computational complexity of spline interpolation. International journal of systems science, 18(5):945–954, 1987.
  54. U.S. Environmental Protection Agency. Guidance on the development, evaluation, and application of environmental models, 2009. https://web.archive.org/web/20110426180258/http://www.epa.gov/CREM/library/cred_guidance_0309.pdf.
  55. Fairness definitions explained. In 2018 IEEE/ACM International Workshop on Software Fairness (FairWare), pages 1–7. IEEE, 2018.
  56. Towards intersectionality in machine learning: Including more identities, handling underrepresentation, and performing evaluation. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 336–349, 2022a.
  57. Understanding instance-level impact of fairness constraints. In International Conference on Machine Learning, pages 23114–23130. PMLR, 2022b.
  58. Learning fair representations. In International Conference on Machine Learning, pages 325–333, 2013.
  59. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340, 2018.
  60. Fairness in decision-making—the causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  61. Faht: an adaptive fairness-aware decision tree classifier. arXiv preprint arXiv:1907.07237, 2019.
  62. Indre Zliobaite. On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723, 2015.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - ReAILe/bias-explainer (4 stars)