Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI (2312.08064v3)

Published 13 Dec 2023 in cs.AI

Abstract: Fairness is a growing concern for high-risk decision-making using AI but ensuring it through purely technical means is challenging: there is no universally accepted fairness measure, fairness is context-dependent, and there might be conflicting perspectives on what is considered fair. Thus, involving stakeholders, often without a background in AI or fairness, is a promising avenue. Research to directly involve stakeholders is in its infancy, and many questions remain on how to support stakeholders to feedback on fairness, and how this feedback can be integrated into AI models. Our work follows an approach where stakeholders can give feedback on specific decision instances and their outcomes with respect to their fairness, and then to retrain an AI model. In order to investigate this approach, we conducted two studies of a complex AI model for credit rating used in loan applications. In study 1, we collected feedback from 58 lay users on loan application decisions, and conducted offline experiments to investigate the effects on accuracy and fairness metrics. In study 2, we deepened this investigation by showing 66 participants the results of their feedback with respect to fairness, and then conducted further offline analyses. Our work contributes two datasets and associated code frameworks to bootstrap further research, highlights the opportunities and challenges of employing lay user feedback for improving AI fairness, and discusses practical implications for developing AI applications that more closely reflect stakeholder views about fairness.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. FairSight: Visual Analytics for Fairness in Decision Making. IEEE Transactions on Visualization and Computer Graphics, 1–1.
  2. Power to the people: The role of humans in interactive machine learning. Ai Magazine, 35(4): 105–120.
  3. Fair active learning. Expert Systems with Applications, 199: 116981.
  4. Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23. New York, NY, USA: Association for Computing Machinery. ISBN 9781450394215.
  5. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5): 4:1–4:15. Conference Name: IBM Journal of Research and Development.
  6. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1): 3–44.
  7. FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning. In 2019 IEEE Conference on Visual Analytics Science and Technology (VAST), 46–56.
  8. Building classifiers with independency constraints. In 2009 IEEE international conference on data mining workshops, 13–18. IEEE.
  9. Fairway: A Way to Build Fair ML Software. In ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, 654–665. New York, NY, USA: Association for Computing Machinery. ISBN 9781450370431.
  10. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, 785–794. New York, NY, USA: Association for Computing Machinery. ISBN 9781450342322.
  11. Soliciting stakeholders’ fairness notions in child maltreatment predictive systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–17.
  12. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, 797–806.
  13. A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems, 8(2): 1–37.
  14. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, 214–226.
  15. Interactive machine learning. In Proceedings of the 8th international conference on Intelligent user interfaces, 39–45.
  16. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 259–268.
  17. A Comparative Study of Fairness-Enhancing Interventions in Machine Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, 329–338. New York, NY, USA: Association for Computing Machinery. ISBN 9781450361255.
  18. D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias. IEEE Transactions on Visualization and Computer Graphics, 29(1): 473–482.
  19. Human-centred machine learning. In Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems, 3558–3565.
  20. You Are the Only Possible Oracle: Effective Test Selection for End Users of Interactive Machine Learning Systems. IEEE Transactions on Software Engineering, 40(3): 307–323.
  21. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29.
  22. Algorithm AS 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics), 28(1): 100–108.
  23. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, 1–16. New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-5970-2.
  24. Home Credit. 2018. Home Credit Default Risk. https://www.kaggle.com/competitions/home-credit-default-risk/data. Accessed: 2023-07-17.
  25. Bia mitigation for machine learning classifiers: A comprehensive survey. arXiv preprint arXiv:2207.07068.
  26. On combining biclustering mining and AdaBoost for breast tumor classification. IEEE Transactions on Knowledge and Data Engineering, 32(4): 728–738.
  27. Fae: A fairness-aware ensemble framework. In 2019 IEEE International Conference on Big Data (Big Data), 1375–1380. IEEE.
  28. Classifying without discriminating. In 2009 2nd international conference on computer, control and communication, 1–6. IEEE.
  29. Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 33(1): 1–33.
  30. Exploiting reject option in classification for social discrimination control. Information Sciences, 425: 18–33.
  31. Fairness-Aware Classifier with Prejudice Remover Regularizer. In Flach, P. A.; De Bie, T.; and Cristianini, N., eds., Machine Learning and Knowledge Discovery in Databases, 35–50. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-642-33486-3.
  32. Interactive optimization for steering machine classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1343–1352.
  33. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International conference on machine learning, 2564–2572. PMLR.
  34. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the conference on fairness, accountability, and transparency, 100–109.
  35. Forming and reacting to overall fairness: A cross-cultural comparison. Organizational Behavior and Human Decision Processes, 104(1): 83–95.
  36. Kirkpatrick, K. 2016. Battling Algorithmic Bias: How Do We Ensure Algorithms Treat Us Fairly? Commun. ACM, 59(10): 16–17.
  37. One-vs.-One Mitigation of Intersectional Bias: A General Method for Extending Fairness-Aware Binary Classification. In de Paz Santana, J. F.; de la Iglesia, D. H.; and López Rivero, A. J., eds., New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence, 43–54. Cham: Springer International Publishing. ISBN 978-3-030-87687-6.
  38. Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In Proceedings of the 2018 world wide web conference, 853–862.
  39. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces, IUI ’15, 126–137. New York, NY, USA: Association for Computing Machinery. ISBN 9781450333061.
  40. Counterfactual fairness. Advances in neural information processing systems, 30.
  41. Human-Centered Approaches to Fair and Responsible AI. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, CHI EA ’20, 1–8. New York, NY, USA: Association for Computing Machinery. ISBN 978-1-4503-6819-3.
  42. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems, 1–13.
  43. FIND: Human-in-the-Loop Debugging Deep Text Classifiers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 332–348. Online: Association for Computational Linguistics.
  44. A review on human–ai interaction in machine learning and insights for medical applications. International journal of environmental research and public health, 18(4): 2121.
  45. A Survey on Bias and Fairness in Machine Learning. ACM Comput. Surv., 54(6).
  46. Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review.
  47. Towards Responsible AI: A Design Space Exploration of Human-Centered Artificial Intelligence User Interfaces to Investigate Fairness. International Journal of Human–Computer Interaction, 1–27.
  48. Toward Involving End-Users in Interactive Human-in-the-Loop AI Fairness. ACM Trans. Interact. Intell. Syst., 12(3).
  49. Stakeholder-in-the-Loop Fair Decisions: A Framework to Design Decision Support Systems in Public and Private Organizations. In Nah, F.; and Siau, K., eds., HCI in Business, Government and Organizations, 34–46. Cham: Springer Nature Switzerland. ISBN 978-3-031-35969-9.
  50. Active fairness in algorithmic decision making. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 77–83.
  51. Pearl, J. 1995. Causal diagrams for empirical research. Biometrika, 82(4): 669–688.
  52. Interactive machine teaching: a human-centered approach to building machine-learned models. Human–Computer Interaction, 35(5-6): 413–451.
  53. Rice, W. E. 1996. Race, Gender, ”Redlining,” and the Discriminatory Access to Loans, Credit, and Insurance: An Historical and Empirical Analysis of Consumers Who Sued Lenders and Insurers in Federal and State Courts, 1950-1995. San Diego L. Rev., 33(583).
  54. Crownn: Human-in-the-loop Network with Crowd-generated Inputs. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7555–7559. IEEE.
  55. Settles, B. 2009. Active learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences. Available online: http://digital.library.wisc.edu/1793/60660.
  56. Shneiderman, B. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction, 36(6): 495–504.
  57. A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual &Group Unfairness via Inequality Indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, 2239–2248. New York, NY, USA: Association for Computing Machinery. ISBN 9781450355520.
  58. Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, 2459–2468. New York, NY, USA: Association for Computing Machinery. ISBN 9781450362016.
  59. Interacting meaningfully with machine learning systems: Three experiments. International journal of human-computer studies, 67(8): 639–662.
  60. Surowiecki, J. 2005. The wisdom of crowds. Anchor.
  61. Fairness definitions explained. In Proceedings of the international workshop on software fairness, 1–7.
  62. Repairing without retraining: Avoiding disparate impact with counterfactual distributions. In International Conference on Machine Learning, 6618–6627. PMLR.
  63. Fairlearn: Assessing and Improving Fairness of AI Systems. arXiv:2303.16626.
  64. The what-if tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics, 26(1): 56–65.
  65. Wright, S. 1934. The method of path coefficients. The annals of mathematical statistics, 5(3): 161–215.
  66. A survey of human-in-the-loop for machine learning. Future Generation Computer Systems.
  67. XGBoost Developers. 2022. XGBoost. https://xgboost.readthedocs.io/en/stable/. Accessed: 2023-07-26.
  68. Learning fair representations. In International conference on machine learning, 325–333. PMLR.
  69. A Causal Framework for Discovering and Removing Direct and Indirect Discrimination. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI’17, 3929–3935. AAAI Press. ISBN 9780999241103.
  70. Longitudinal Fairness with Censorship. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11): 12235–12243.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Evdoxia Taka (3 papers)
  2. Yuri Nakao (10 papers)
  3. Ryosuke Sonoda (4 papers)
  4. Takuya Yokota (3 papers)
  5. Lin Luo (27 papers)
  6. Simone Stumpf (16 papers)
Citations (1)
View on Semantic Scholar