Papers
Topics
Authors
Recent
Search
2000 character limit reached

Fair Enough? A map of the current limitations of the requirements to have fair algorithms

Published 21 Nov 2023 in cs.AI, cs.CY, cs.HC, and cs.LG | (2311.12435v4)

Abstract: In recent years, the increase in the usage and efficiency of Artificial Intelligence and, more in general, of Automated Decision-Making systems has brought with it an increasing and welcome awareness of the risks associated with such systems. One of such risks is that of perpetuating or even amplifying bias and unjust disparities present in the data from which many of these systems learn to adjust and optimise their decisions. This awareness has on the one hand encouraged several scientific communities to come up with more and more appropriate ways and methods to assess, quantify, and possibly mitigate such biases and disparities. On the other hand, it has prompted more and more layers of society, including policy makers, to call for fair algorithms. We believe that while many excellent and multidisciplinary research is currently being conducted, what is still fundamentally missing is the awareness that having fair algorithms is per se a nearly meaningless requirement that needs to be complemented with many additional social choices to become actionable. Namely, there is a hiatus between what the society is demanding from Automated Decision-Making systems, and what this demand actually means in real-world scenarios. In this work, we outline the key features of such a hiatus and pinpoint a set of crucial open points that we as a society must address in order to give a concrete meaning to the increasing demand of fairness in Automated Decision-Making systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. , 1947. Italian Constitution. URL: https://www.senato.it/istituzione/la-costituzione.
  2. Fairwashing: the risk of rationalization, in: Chaudhuri, K., Salakhutdinov, R. (Eds.), Proceedings of the 36th International Conference on Machine Learning, PMLR. pp. 161–170. URL: https://proceedings.mlr.press/v97/aivodji19a.html.
  3. The role of accuracy in algorithmic process fairness across multiple domains, in: Proceedings of the 22nd ACM Conference on Economics and Computation, Association for Computing Machinery, New York, NY, USA. p. 29–49. doi:10.1145/3465456.3467620.
  4. Discrimination, in: Zalta, E.N. (Ed.), The Stanford Encyclopedia of Philosophy. Winter 2020 ed.. Metaphysics Research Lab, Stanford University.
  5. Substantive equality. The Cambridge Law Journal 59, 562–585. doi:10.1017/S0008197300000246.
  6. Fairness and Machine Learning: Limitations and Opportunities. fairmlbook.org. http://www.fairmlbook.org.
  7. Big data’s disparate impact. California law review , 671–732doi:10.15779/Z38BG31.
  8. Bias on demand: A modelling framework that generates synthetic data with bias, in: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 1002–1013. doi:10.1145/3593013.3594058.
  9. The possibility of fairness: Revisiting the impossibility theorem in practice, in: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 400–422. doi:10.1145/3593013.3594007.
  10. AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. URL: https://arxiv.org/abs/1810.01943.
  11. Are emily and greg more employable than lakisha and jamal? a field experiment on labor market discrimination. American Economic Review 94, 991–1013. doi:10.1257/0002828042002561.
  12. Equality of Opportunity, in: Encyclopedia of Political Theory. SAGE Publications, Inc. doi:10.4135/9781412958660.
  13. Sex bias in graduate admissions: Data from berkeley. Science 187, 398–404. doi:10.1126/science.187.4175.398.
  14. Gender shades: Intersectional accuracy disparities in commercial gender classification, in: Friedler, S.A., Wilson, C. (Eds.), Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR. pp. 77–91. URL: https://proceedings.mlr.press/v81/buolamwini18a.html.
  15. Inherent limitations of ai fairness. arXiv preprint arXiv:2212.06495 URL: https://arxiv.org/abs/2212.06495.
  16. A clarification of the nuances in the fairness metrics landscape. Scientific Reports 12, 4209. doi:10.1038/s41598-022-07939-1.
  17. Path-specific counterfactual fairness. Proceedings of the AAAI Conference on Artificial Intelligence 33, 7801–7808. doi:10.1609/aaai.v33i01.33017801.
  18. The new faces of fairness. Journal of educational Measurement 38, 369–382. doi:1745-3984.2001.tb01132.x.
  19. Emergent unfairness in algorithmic fairness-accuracy trade-off research, in: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, Association for Computing Machinery, New York, NY, USA. p. 46–54. doi:10.1145/3461702.3462519.
  20. A pro-innovation approach to AI regulation. https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach.
  21. Fairness through awareness, in: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, Association for Computing Machinery, New York, NY, USA. p. 214–226. doi:10.1145/2090236.2090255.
  22. Equality of Opportunity, in: Zalta, E.N., Nodelman, U. (Eds.), The Stanford Encyclopedia of Philosophy. Fall 2023 ed.. Metaphysics Research Lab, Stanford University.
  23. Equal Employment Opportunity Commission, 2015. Uniform Guidelines on Employment Selection Procedures. URL: https://www.govinfo.gov/content/pkg/CFR-2011-title29-vol4/xml/CFR-2011-title29-vol4-part1607.xml. 29 Code of Federal Regulation §1607.4(D).
  24. Selection, ignorability and challenges with causal fairness, in: Schölkopf, B., Uhler, C., Zhang, K. (Eds.), Proceedings of the First Conference on Causal Learning and Reasoning, PMLR. pp. 275–289. URL: https://proceedings.mlr.press/v177/fawkes22a.html.
  25. Causation, Statistics, and Sociology. European Sociological Review 17, 1–20. doi:10.1093/esr/17.1.1.
  26. gov.uk, 2023. Prime Minister launches new AI Safety Institute. gov.uk: Press Release URL: https://www.gov.uk/government/news/prime-minister-launches-new-ai-safety-institute.
  27. Causal effects of perceived immutable characteristics. The Review of Economics and Statistics 93, 775–785. URL: http://www.jstor.org/stable/23016076.
  28. Dimensions of diversity in human perceptions of algorithmic fairness, in: Equity and Access in Algorithms, Mechanisms, and Optimization, Association for Computing Machinery, New York, NY, USA. doi:10.1145/3551624.3555306.
  29. Promises and pitfalls of algorithm use by state authorities. Philosophy & Technology 35, 33. doi:10.1007/s13347-022-00528-0.
  30. An empirical study on the perceived fairness of realistic, imperfect machine learning models, in: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 392–402. doi:10.1145/3351095.3372831.
  31. Formation stories and causality in sociology. Sociological Theory 32, 259–282. doi:10.1177/0735275114558632.
  32. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society 22, 900–915. doi:10.1080/1369118X.2019.1573912.
  33. Statistics and causal inference. Journal of the American Statistical Association 81, 945–960. URL: http://www.jstor.org/stable/2289064.
  34. What’s sex got to do with fair machine learning? arXiv preprint arXiv:2006.01770 doi:10.48550/arXiv.2006.01770.
  35. 50 years of test (un)fairness: Lessons for machine learning, in: Proceedings of the Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 49–58. doi:10.1145/3287560.3287600.
  36. Amazon doesn’t consider the race of its customers. should it? Bloomberg URL: https://www.bloomberg.com/graphics/2016-amazon-same-day.
  37. Jefatura del Estado, 2022. Comprehensive Law for Equal Treatment and Non-Discrimination. URL: https://www.boe.es/eli/es/l/2022/07/12/15/con.
  38. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness, in: Dy, J., Krause, A. (Eds.), Proceedings of the 35th International Conference on Machine Learning, PMLR. pp. 2564–2572. URL: https://proceedings.mlr.press/v80/kearns18a.html.
  39. Race-aware algorithms: Fairness, nondiscrimination and affirmative action. California Law Review 110, 1539.
  40. Inherent trade-offs in the fair determination of risk scores. arXiv:1609.05807.
  41. Are “intersectionally fair” ai algorithms really fair to women of color? a philosophical analysis, in: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 485–494. doi:10.1145/3531146.3533114.
  42. Counterfactual fairness. Advances in neural information processing systems 30. URL: https://proceedings.neurips.cc/paper_files/paper/2017/hash/1271a7029c9df08643b631b02cf9e116-Abstract.html.
  43. The landscape and gaps in open source fairness toolkits, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Association for Computing Machinery, New York, NY, USA. doi:10.1145/3411764.3445261.
  44. Does mitigating ml’s impact disparity require treatment disparity? Advances in neural information processing systems 31. URL: https://proceedings.neurips.cc/paper_files/paper/2018/hash/8e0384779e58ce2af40eb365b318cc32-Abstract.html.
  45. Troubling trends in machine learning scholarship. arXiv preprint arXiv:1807.03341 .
  46. Subverting machines, fluctuating identities: Re-learning human categorization, in: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 1005–1015. doi:10.1145/3531146.3533161.
  47. Machine learning fairness notions: Bridging the gap with real-world applications. Information Processing & Management 58, 102642. doi:https://doi.org/10.1016/j.ipm.2021.102642.
  48. On the applicability of machine learning fairness notions. SIGKDD Explor. Newsl. 23, 14–23. doi:10.1145/3468507.3468511.
  49. The construction of human kinds. Oxford University Press.
  50. The AI Index 2023 Annual Report. Technical Report. Institute for Human-Centered AI, Stanford University, Stanford, CA. URL: https://aiindex.stanford.edu/report/.
  51. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application 8, 141–163. doi:10.1146/annurev-statistics-042720-125902.
  52. Introduction to causal inference. Course Lecture Notes (draft) .
  53. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453. doi:10.1126/science.aax2342.
  54. The book of why: the new science of cause and effect. Basic books.
  55. Elements of Causal Inference. The MIT Press, Cambridge.
  56. Causal fairness analysis. Foundations and Trends in Machine Learning XX, XX. doi:10.1561/XXXXXXXXXX. to appear - technical report available at https://causalai.net/r90.pdf.
  57. A multidisciplinary survey on discrimination analysis. The Knowledge Engineering Review 29, 582–638. doi:10.1017/S0269888913000039.
  58. Multi-dimensional discrimination in law and machine learning - a comparative overview, in: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 89–100. doi:10.1145/3593013.3593979.
  59. Can We Trust Fair-AI? Proceedings of the AAAI Conference on Artificial Intelligence 37, 15421–15430. doi:10.1609/aaai.v37i13.26798.
  60. Missed Opportunities in Fair AI. pp. 961–964. doi:10.1137/1.9781611977653.ch110.
  61. Fairness and abstraction in sociotechnical systems, in: Proceedings of the Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA. p. 59–68. doi:10.1145/3287560.3287598.
  62. Scoping fairness objectives and identifying fairness metrics for recommender systems: The practitioners’ perspective, in: Proceedings of the ACM Web Conference 2023, Association for Computing Machinery, New York, NY, USA. p. 3648–3659. doi:10.1145/3543507.3583204.
  63. Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature. Big Data & Society 9, 20539517221115189. doi:10.1177/20539517221115189.
  64. Supreme Court of the United States, 2015. Tex. Dep’t of Hous. & Cmty. Affairs v. Inclusive Cmtys. Project, Inc. URL: https://casetext.com/case/texas-dept-of-housing-and-community-affairs-v-inclusive-communities-project-inc?. 576 U.S. 519, 135 S. Ct. 2507, 192 L. Ed. 2d 514, 83 U.S.L.W. 4555.
  65. The Council of The European Union, 2000a. Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin. URL: https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:32000L0043:en:HTML.
  66. The Council of The European Union, 2000b. Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32000L0078.
  67. The Council of The European Union, 2004. Council Directive 2004/113/EC of 13 December 2004 implementing the principle of equal treatment between men and women in the access to and supply of goods and services. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32004L0113.
  68. The European Commission, 2021a. Proposal for a Directive of the European Parliament and of the Council on consumer credits. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0347.
  69. The European Commission, 2021b. Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC0206.
  70. Non-discrimination. URL: https://commission.europa.eu/aid-development-cooperation-fundamental-rights/your-rights-eu/know-your-rights/equality/non-discrimination_en. accessed: 2023-08-22.
  71. The European Parliament and the Council of The European Union, 2006. Directive 2006/54/EC of the European Parliament and of the Council of 5 July 2006 on the implementation of the principle of equal opportunities and equal treatment of men and women in matters of employment and occupation (recast). URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32006L0054.
  72. The European Parliament and the Council of The European Union, 2008. Directive 2008/48/EC of the European Parliament and of the Council of 23 April 2008 on credit agreements for consumers and repealing Council Directive 87/102/EEC. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32008L0048.
  73. Charter of Fundamental Rights of the European Union. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A12012P%2FTXT.
  74. Cathy Carson v. Bethmehem Steel Corp. URL: https://casetext.com/case/carson-v-bethlehem-steel-corporation?. 82 F.3d 157 (7th Cir. 1996).
  75. U.S. Government Publishing Office, 1964. Civil Rights Act of 1964. URL: https://www.govinfo.gov/app/details/COMPS-342. Public Law 88–352; 78 Stat. 241, as Amended Through P.L. 114–95, Enacted December 10, 2015.
  76. U.S. Government Publishing Office, 1974. Equal Credit Opportunity Act of 1974. URL: https://www.govinfo.gov/content/pkg/USCODE-2011-title15/html/USCODE-2011-title15-chap41-subchapIV.htm. United States Code, Title 15. Chapter 41, Subchapter IV.
  77. U.S. Supreme Court, 1977. Castaneda, Sheriff v. Partida. URL: https://casetext.com/case/castaneda-v-partida?. 430 U.S. 482, 97 S. Ct. 1272.
  78. U.S. Supreme Court, 2009. Frank Ricci et al. v. John DeStefano et al. URL: https://casetext.com/case/ricci-v-destefano-13?. 557 U.S. 557, 129 S. Ct. 2658, 174 L. Ed. 2d 490, 21 Fla. L. Weekly Supp. 1049.
  79. Why fairness cannot be automated: Bridging the gap between eu non-discrimination law and ai. Computer Law & Security Review 41, 105567. doi:https://doi.org/10.1016/j.clsr.2021.105567.
  80. Fairlearn: Assessing and improving fairness of ai systems. arXiv:2303.16626.
  81. The what-if tool: Interactive probing of machine learning models. IEEE Transactions on Visualization and Computer Graphics 26, 56–65. doi:10.1109/TVCG.2019.2934619.
  82. White House Office of Science and Technology Policy , 2022. Blueprint for an AI Bill of Rights: Making Automated Systems Work For The American People. https://www.whitehouse.gov/ostp/ai-bill-of-rights/.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up