Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Disciplining Deliberation: A Sociotechnical Perspective on Machine Learning Trade-offs (2403.04226v2)

Published 7 Mar 2024 in cs.CY

Abstract: This paper examines two prominent formal trade-offs in AI -- between predictive accuracy and fairness, and between predictive accuracy and interpretability. These trade-offs have become a central focus in normative and regulatory discussions as policymakers seek to understand the value tensions that can arise in the social adoption of AI tools. The prevailing interpretation views these formal trade-offs as directly corresponding to tensions between underlying social values, implying unavoidable conflicts between those social objectives. In this paper, I challenge that prevalent interpretation by introducing a sociotechnical approach to examining the value implications of trade-offs. Specifically, I identify three key considerations -- validity and instrumental relevance, compositionality, and dynamics -- for contextualizing and characterizing these implications. These considerations reveal that the relationship between model trade-offs and corresponding values depends on critical choices and assumptions. Crucially, judicious sacrifices in one model property for another can, in fact, promote both sets of corresponding values. The proposed sociotechnical perspective thus shows that we can and should aspire to higher epistemic and ethical possibilities than the prevalent interpretation suggests, while offering practical guidance for achieving those outcomes. Finally, I draw out the broader implications of this perspective for AI design and governance, highlighting the need to broaden normative engagement across the AI lifecycle, develop legal and auditing tools sensitive to sociotechnical considerations, and rethink the vital role and appropriate structure of interdisciplinary collaboration in fostering a responsible AI workforce.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (102)
  1. Elizabeth Anderson. 2006. The epistemology of democracy. Episteme 3, 1-2 (2006), 8–22.
  2. Algorithms on regulatory lockdown in medicine. Science 366, 6470 (2019), 1202–1204.
  3. Beware explanations from AI in health care. Science 373, 6552 (2021), 284–286.
  4. Dan Bang and Chris D Frith. 2017. Making better decisions in groups. Royal Society open science 4, 8 (2017), 170193.
  5. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2–11.
  6. Updates in human-AI teams: Understanding and addressing the performance/compatibility tradeoff. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 2429–2437.
  7. Does the whole exceed its parts? the effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.
  8. Fairness and Machine Learning. fairmlbook.org. http://www.fairmlbook.org.
  9. Julia B Bear and Anita Williams Woolley. 2011. The role of gender in team collaboration and performance. Interdisciplinary science reviews 36, 2 (2011), 146–153.
  10. The price of interpretability. arXiv preprint arXiv:1907.03419 (2019).
  11. Model Multiplicity: Opportunities, Concerns, and Solutions. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT ’22). Association for Computing Machinery, New York, NY, USA, 850–863.
  12. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 1721–1730.
  13. Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153–163.
  14. Alexandra Chouldechova and Aaron Roth. 2020. A snapshot of the frontiers of fairness in machine learning. Commun. ACM 63, 5 (2020), 82–89.
  15. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. 797–806.
  16. A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 690–704.
  17. Characterizing fairness over the set of good models under selective labels. In International Conference on Machine Learning. PMLR, 2144–2155.
  18. Kathleen A Creel. 2020. Transparency in complex computational systems. Philosophy of Science 87, 4 (2020), 568–589.
  19. Alexander D’Amour. 2021. Revisiting Rashomon: A Comment on” The Two Cultures”. Observational Studies 7, 1 (2021), 59–63.
  20. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 (2020).
  21. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 525–534.
  22. Dealing with disagreements: Looking beyond the majority vote in subjective annotations. Transactions of the Association for Computational Linguistics 10 (2022), 92–110.
  23. Learning under selective labels in the presence of expert consistency. arXiv preprint arXiv:1807.00905 (2018).
  24. Algorithmic fairness in business analytics: Directions for research and practice. Production and Operations Management 31, 10 (2022), 3749–3770.
  25. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
  26. Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General 144, 1 (2015), 114.
  27. Hard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical Commitments. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 242–242.
  28. Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness. arXiv preprint arXiv:2202.08821 (2022).
  29. Cynthia Dwork and Christina Ilvento. 2018. Fairness under composition. arXiv preprint arXiv:1806.06122 (2018).
  30. Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability. arXiv preprint arXiv:2010.13764 (2020).
  31. What science can do for democracy: a complexity science approach. Humanities and Social Sciences Communications 7, 1 (2020), 1–4.
  32. Sina Fazelpour and David Danks. 2021. Algorithmic bias: Senses, sources, solutions. Philosophy Compass 16, 8 (2021), e12760.
  33. Sina Fazelpour and Maria De-Arteaga. 2021. Diversity in sociotechnical machine learning systems. arXiv preprint arXiv:2107.09163 (2021).
  34. Sina Fazelpour and Zachary C Lipton. 2020. Algorithmic fairness from a non-ideal perspective. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 57–63.
  35. Algorithmic Fairness and the Situated Dynamics of Justice. Canadian Journal of Philosophy (2021), 1–17.
  36. Will Fleisher. 2022. Understanding, idealization, and explainable AI. Episteme 19, 4 (2022), 534–560.
  37. Ben Green and Yiling Chen. 2019. Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. In Proceedings of the conference on fairness, accountability, and transparency. 90–99.
  38. Stephen Grimm. 2012. The value of understanding. Philosophy Compass 7, 2 (2012), 103–117.
  39. XAI—Explainable artificial intelligence. Science Robotics 4, 37 (2019), eaay7120.
  40. A Survey on Automated Fact-Checking. Transactions of the Association for Computational Linguistics 10 (2022), 178–206.
  41. Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016).
  42. Deborah Hellman. 2020. Measuring algorithmic fairness. Virginia Law Review 106, 4 (2020), 811–866.
  43. Jonathan Herington. 2020. Measuring fairness in an unfair World. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 286–292.
  44. Lu Hong and Scott E Page. 2004. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences 101, 46 (2004), 16385–16389.
  45. Abigail Z Jacobs and Hanna Wallach. 2021. Measurement and fairness. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 375–385.
  46. The global landscape of AI ethics guidelines. Nature Machine Intelligence 1, 9 (2019), 389–399.
  47. Gabbrielle M Johnson. 2021. Algorithmic bias: on the implicit biases of social technology. Synthese 198, 10 (2021), 9941–9961.
  48. Fairness-aware learning through regularization approach. In 2011 IEEE 11th International Conference on Data Mining Workshops. IEEE, 643–650.
  49. Michael Kearns and Aaron Roth. 2019. The ethical algorithm: The science of socially aware algorithm design. Oxford University Press.
  50. Jon Kleinberg. 2018. Inherent trade-offs in algorithmic fairness. In Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems. 40–40.
  51. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2018), 237–293.
  52. Discrimination in the Age of Algorithms. Journal of Legal Analysis 10 (2018), 113–174.
  53. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digital Medicine 4, 1 (2021), 1–6.
  54. Maya Krishnan. 2020. Against interpretability: a critical examination of the interpretability problem in machine learning. Philosophy & Technology 33, 3 (2020), 487–502.
  55. Hélène Landemore. 2020. Open democracy: Reinventing popular rule for the twenty-first century. Princeton University Press.
  56. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–35.
  57. Betweenness centrality and the interdisciplinarity of cognitive science. Cognitive Science: A Multidisciplinary Journal (2008).
  58. Zachary C Lipton. 2018. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31–57.
  59. Delayed impact of fair machine learning. In International Conference on Machine Learning. PMLR, 3150–3158.
  60. Michele Loi and Markus Christen. 2021. Choosing how to discriminate: navigating ethical trade-offs in fair algorithmic design for the insurance sector. Philosophy & Technology 34 (2021), 967–992.
  61. Tania Lombrozo. 2011. The instrumental value of explanations. Philosophy Compass 6, 8 (2011), 539–551.
  62. Alex John London. 2019. Artificial intelligence and black-box medical decisions: accuracy versus explainability. Hastings Center Report 49, 1 (2019), 15–21.
  63. Good explanation for algorithmic transparency. Available at SSRN 3503603 (2019).
  64. Predict responsibly: improving fairness and accuracy by learning to defer. Advances in Neural Information Processing Systems 31 (2018).
  65. Participatory problem formulation for fairer machine learning through community based system dynamics. arXiv preprint arXiv:2005.07572 (2020).
  66. Predictive multiplicity in classification. In International Conference on Machine Learning. PMLR, 6765–6774.
  67. The independence thesis: When individual and social epistemology diverge. Philosophy of Science 78, 4 (2011), 653–677.
  68. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38.
  69. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.
  70. Christoph Molnar. 2022. Interpretable Machine Learning (2 ed.). christophm.github.io/interpretable-ml-book/
  71. Hussein Mozannar and David Sontag. 2020. Consistent estimators for learning to defer to an expert. In International Conference on Machine Learning. PMLR, 7076–7087.
  72. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences 116, 44 (2019), 22071–22080.
  73. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447–453.
  74. Cailin O’Connor and James Owen Weatherall. 2019. The misinformation age. Yale University Press.
  75. Enhancing communication & collaboration in interdisciplinary research. sage publications.
  76. Scott Page. 2019. The diversity bonus. Princeton University Press.
  77. Daniel Paravisini and Antoinette Schoar. 2013. The incentive effect of scores: Randomized evidence from credit committees. Technical Report. National Bureau of Economic Research.
  78. Katherine W Phillips. 2017. Commentary. What is the real value of diversity in organizations? Questioning our assumptions. In The diversity bonus. Princeton University Press, 223–246.
  79. Tony A Plate. 1999. Accuracy versus interpretability in flexible modeling: Implementing a tradeoff using gaussian process models. Behaviormetrika 26, 1 (1999), 29–50.
  80. Corinne Post and Kris Byron. 2015. Women on boards and firm financial performance: A meta-analysis. Academy of management Journal 58, 5 (2015), 1546–1571.
  81. Direct uncertainty prediction for medical second opinions. In International Conference on Machine Learning. PMLR, 5281–5290.
  82. Iyad Rahwan. 2018. Society-in-the-loop: programming the algorithmic social contract. Ethics and information technology 20, 1 (2018), 5–14.
  83. The fallacy of AI functionality. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 959–972.
  84. A unifying framework for combining complementary strengths of humans and ML toward better predictive decision-making. arXiv preprint arXiv:2204.10806 (2022).
  85. Empirical observation of negligible fairness–accuracy trade-offs in machine learning for public policy. Nature Machine Intelligence 3, 10 (2021), 896–904.
  86. Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (2019), 206–215.
  87. A human-centered review of algorithms used within the us child welfare system. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15.
  88. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency. 59–68.
  89. A study in Rashomon curves and volumes: A new perspective on generalization and model simplicity in machine learning. arXiv preprint arXiv:1908.01755 (2019).
  90. An empirical analysis of backward compatibility in machine learning systems. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3272–3280.
  91. Multiple diversity concepts and their ethical-epistemic implications. European journal for philosophy of science 8 (2018), 761–780.
  92. Catherine Stinson and Sofie Vlaad. 2024. A feeling for the algorithm: Diversity, expertise, and artificial intelligence. Big Data & Society 11, 1 (2024), 20539517231224247.
  93. Harini Suresh and John V Guttag. 2019. A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002 2, 8 (2019).
  94. Elham Tabassi. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). (2023).
  95. Kate Vredenburgh. 2021. The right to explanation. Journal of Political Philosophy (2021).
  96. Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 436–444.
  97. Zeerak Waseem. 2016. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on NLP and computational social science. 138–142.
  98. Warren W Willingham and Nancy S Cole. 2013. Gender and fair assessment. Routledge.
  99. H Peyton Young. 1994. Equity. Princeton University Press.
  100. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics. PMLR, 962–970.
  101. How transparency modulates trust in artificial intelligence. Patterns (2022), 100455.
  102. Who Leads and Who Follows in Strategic Classification? Advances in Neural Information Processing Systems 34 (2021).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now