Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fair Machine Guidance to Enhance Fair Decision Making in Biased People (2404.05228v1)

Published 8 Apr 2024 in cs.HC

Abstract: Teaching unbiased decision-making is crucial for addressing biased decision-making in daily life. Although both raising awareness of personal biases and providing guidance on unbiased decision-making are essential, the latter topics remains under-researched. In this study, we developed and evaluated an AI system aimed at educating individuals on making unbiased decisions using fairness-aware machine learning. In a between-subjects experimental design, 99 participants who were prone to bias performed personal assessment tasks. They were divided into two groups: a) those who received AI guidance for fair decision-making before the task and b) those who received no such guidance but were informed of their biases. The results suggest that although several participants doubted the fairness of the AI system, fair machine guidance prompted them to reassess their views regarding fairness, reflect on their biases, and modify their decision-making criteria. Our findings provide insights into the design of AI systems for guiding fair decision-making in humans.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (88)
  1. A Reductions Approach to Fair Classification. In Proceedings of the 2018 International Conference on Machine Learning. https://doi.org/10.48550/arXiv.1803.02453
  2. Machine Bias. (2022), 254–264.
  3. Becoming Homeless: A Human Experience. In Proceedings of the 2018 ACM SIGGRAPH Virtual, Augmented, and Mixed Reality. https://doi.org/10.1145/3226552.3226576
  4. CrowDEA: Multi-view Idea Prioritization with Crowds. In Proceedings of the 2020 AAAI Conference on Human Computation and Crowdsourcing. https://doi.org/10.48550/arXiv.2008.02354
  5. “If I Had All the Time in the World”: Ophthalmologists’ Perceptions of Anchoring Bias Mitigation in Clinical AI Support. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581513
  6. Barry Becker and Ronny Kohavi. 1996. Adult. UCI Machine Learning Repository. https://doi.org/10.24432/C5XW20
  7. Marianne Bertrand and Sendhil Mullainathan. 2004. Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination. American Economic Review 94, 4 (2004), 991–1013. https://doi.org/10.1257/0002828042002561
  8. Jack W. Brehm. 1966. A Theory of Psychological Reactance. 135 (1966).
  9. A Systematic Review of Algorithm Aversion in Augmented Decision Making. Journal of Behavioral Decision Making 33, 2 (2020), 220–239. https://doi.org/10.1002/bdm.2155
  10. Toon Calders and Sicco Verwer. 2010. Three Naive Bayes Approaches for Discrimination-free Classification. Data Mining and Knowledge Discovery 21, 2 (2010), 277–292. https://doi.org/10.1007/s10618-010-0190-x
  11. The Mixed Effects of Online Diversity Training. National Academy of Sciences 116, 16 (2019), 7778–7783. https://doi.org/10.1073/pnas.1816076116
  12. News Literacy Education in a Polarized Political Climate: How Games Can Teach Youth to Spot Misinformation. Harvard Kennedy School Misinformation Review 1 (2020). https://doi.org/10.37016/mr-2020-020
  13. Near-optimal Machine Teaching via Explanatory Teaching Sets. In Proceedings of the 2018 International Conference on Artificial Intelligence and Statistics.
  14. How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3501831
  15. Soliciting Stakeholders’ Fairness Notions in Child Maltreatment Predictive Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445308
  16. Are Two Heads Better than One in AI-assisted Decision Making? Comparing the Behavior and Performance of Groups and Individuals in Human-AI Collaborative Recidivism Risk Assessment. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581015
  17. A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376638
  18. Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err. Journal of Experimental Psychology: General 144, 1 (2015), 114. https://doi.org/10.2139/ssrn.2466040
  19. Method for Appropriating the Brief Implicit Association Test to Elicit Biases in Users. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517570
  20. Fairness through Awareness. In Proceedings of the 2012 Innovations in Theoretical Computer Science Conference. https://doi.org/10.1145/2090236.2090255
  21. Diversity of Inference Strategies can Enhance the ‘Wisdom-of-Crowds’ effect. Palgrave Communications 4, 1 (2018), 1–9. https://doi.org/10.1057/s41599-018-0161-1
  22. Krzysztof Z. Gajos and Lena Mamykina. 2022. Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning. In Proceedings of the 2022 International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3490099.3511138
  23. Adam D. Galinsky and Gordon B. Moskowitz. 2007. Further Ironies of Suppression: Stereotype and Counterstereotype Accessibility. Journal of Experimental Social Psychology 43, 5 (2007), 833–841. https://doi.org/10.1016/j.jesp.2006.09.001
  24. Meric Altug Gemalmaz and Ming Yin. 2022. Understanding Decision Subjects’ Fairness Perceptions and Retention in Repeated Interactions with AI-based Decision Systems. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3514094.3534201
  25. Alexandra Goedderz and Adam Hahn. 2022. Biases Left Unattended: People are Surprised at Racial Bias Feedback until They Pay Attention to Their Biased Reactions. Journal of Experimental Social Psychology 102 (2022), 104374. https://doi.org/10.1016/j.jesp.2022.104374
  26. Claudia Goldin and Cecilia Rouse. 2000. Orchestrating Impartiality: The Impact of “Blind” Auditions on Female Musicians. American Economic Review 90, 4 (2000), 715–741. https://doi.org/10.1257/aer.90.4.715
  27. Measuring Individual Differences in Implicit Cognition: The Implicit Association Test. Journal of Personality and Social Psychology 74, 6 (1998), 1464. https://doi.org/10.1037/0022-3514.74.6.1464
  28. Marie-Pascale Grimon and Christopher Mills. 2022. The Impact of Algorithmic Tools on Child Protection: Evidence from a Randomized Controlled Trial. Job Market Paper (2022).
  29. Kent D. Harber. 1998. Feedback to Minorities: Evidence of a Positive Bias. Journal of Personality and Social Psychology 74, 3 (1998), 622. https://doi.org/10.1037/0022-3514.74.3.622
  30. Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems 29 (2016). https://doi.org/10.48550/arXiv.1610.02413
  31. Christopher Harris. 2020. Mitigating Cognitive Biases in Machine Learning Algorithms for Decision Making. In Proceedings of the 2020 Companion Proceedings of the Web Conference. https://doi.org/10.1145/3366424.3383562
  32. Peter Hase and Mohit Bansal. 2020. Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?. In Proceedings of the 2020 Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.491
  33. Knowing about Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581025
  34. Edward R. Hirt and Keith D. Markman. 1995. Multiple Explanation: A Consider-an-alternative Strategy for Debiasing Judgments. Journal of Personality and Social Psychology 69, 6 (1995), 1069. https://doi.org/10.1037/0022-3514.69.6.1069
  35. Hans Hofmann. 1994. Statlog (German Credit Data). UCI Machine Learning Repository. https://doi.org/10.24432/C5NC77
  36. Teaching Multiple Concepts to a Forgetful Learner. Advances in Neural Information Processing Systems 32 (2019). https://doi.org/10.5555/3454287.3454651
  37. Generalized Demographic Parity for Group Fairness. In Proceedings of the 2022 International Conference on Learning Representations.
  38. “Because AI is 100% Right and Safe”: User Attitudes and Sources of AI Authority in India. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517533
  39. Jennifer Y. Kim and Loriann Roberson. 2022. I’m Biased and so are You. What Should Organizations Do? A Review of Organizational Implicit-bias Training Programs. Consulting Psychology Journal 74, 1 (2022), 19. https://doi.org/10.1037/cpb0000211
  40. Sneaking in through the Back Door: How Category-based Stereotype Suppression Leads to Rebound in Feature-based Effects. Journal of Experimental Social Psychology 44, 3 (2008), 833–839.
  41. Regina König and Angela Heine. 2023. Learning to Detect Sexism: An Evaluation of the Effects of a Brief Video-based Intervention Using ROC Analysis. Frontiers in Psychology 13 (2023), 1005633. https://doi.org/10.1016/j.jesp.2007.07.011
  42. To Self-persuade or be Persuaded: Examining Interventions for Users’ Privacy Setting Selection. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3502009
  43. Boosting Medical Diagnostics by Pooling Independent Judgments. Proceedings of the National Academy of Sciences of the United States of America 113, 31 (2016), 8777–8782. https://doi.org/10.1073/pnas.1601827113
  44. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-subject Studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3593013.3594087
  45. Richard P. Larrick. 2004. Debiasing. Blackwell Handbook of Judgment and Decision Making (2004), 316–338. https://doi.org/10.1002/9780470752937.ch16
  46. Race, Gender and Beauty: The Effect of Information Provision on Online Hiring Biases. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376874
  47. Yiqiao Liao and Parinaz Naghizadeh. 2023. Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria. In Proceedings of the 2023 AAAI Conference on Artificial Intelligence. https://doi.org/10.1609/aaai.v37i7.26054
  48. Sarah Lichtenstein and Baruch Fischhoff. 1980. Training for Calibration. Organizational Behavior and Human Performance 26, 2 (1980), 149–171. https://doi.org/10.1016/0030-5073(80)90052-5
  49. Human Perceptions on Moral Responsibility of AI: A Case Study in AI-assisted Bail Decision-making. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445260
  50. Iterative Machine Teaching. In Proceedings of the 2017 International Conference on Machine Learning. https://doi.org/10.48550/arXiv.1705.10470
  51. Towards Black-box Iterative Machine Teaching. In Proceedings of the 2018 International Conference on Machine Learning. https://doi.org/10.48550/arXiv.1710.07742
  52. Iterative Teaching by Label Synthesis. Advances in Neural Information Processing Systems 34 (2021), 21681–21695. https://doi.org/10.48550/arXiv.2110.14432
  53. Boosting People’s Ability to Detect Microtargeted Advertising. Scientific Reports 11, 1 (2021), 15541. https://doi.org/10.1038/s41598-021-94796-z
  54. The Chicago Face Database: A Free Stimulus Set of Faces and Norming Data. Behavior Research Methods 47, 4 (2015), 1122–1135. https://doi.org/10.3758/s13428-014-0532-5
  55. Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-assisted Decision-making. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581058
  56. Zilin Ma and Krzysztof Z. Gajos. 2022. Not Just a Preference: Reducing Biased Decision-making on Dating Websites. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517587
  57. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6 (2021), 1–35. https://doi.org/10.1145/3457607
  58. Consequences of Stereotype Suppression: Stereotypes on AND not on the Rebound. Journal of Experimental Social Psychology 34, 4 (1998), 355–377. https://doi.org/10.1006/jesp.1998.1355
  59. Toward Involving End-users in Interactive Human-in-the-loop AI Fairness. ACM Transactions on Interactive Intelligent Systems 12, 3 (2022), 1–30. https://doi.org/10.1145/3514258
  60. Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. In Proceedings of the 2018 North American Chapter of the Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1097
  61. Individuals Fail to Reap the Collective Benefits of Diversity Because of Over-reliance on Personal Information. Journal of the Royal Society Interface 15, 142 (2018), 20180155. https://doi.org/10.1098/rsif.2018.0155
  62. Tomoko Oe and Takashi Oka. 2003. Overcoming the Ironic Rebound: Effective and Ineffective Strategies for Stereotype Suppression. Progress in Asian Social Psychology: Conceptual and Empirical Contributions. 331 (2003), 233–246. https://doi.org/10.1177/2372732220983840
  63. The Relative Influence of Advice from Human Experts and Statistical Methods on Forecast Adjustments. Journal of Behavioral Decision Making 22, 4 (2009), 390–409. https://doi.org/10.1002/bdm.637
  64. Understanding the Impact of Explanations on Advice-taking: A User Study for AI-based Clinical Decision Support Systems. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3502104
  65. Designing Fair AI in Human Resource Management: Understanding Tensions Surrounding Algorithmic Evaluation and Envisioning Stakeholder-centered Solutions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517672
  66. Putting Yourself in the Skin of a Black Avatar Reduces Implicit Racial Bias. Consciousness and Cognition 22, 3 (2013), 779–787. https://doi.org/10.1016/j.concog.2013.04.016
  67. What You See is What You Get? The Impact of Representation Criteria on Human Bias in Hiring. In Proceedings of the 2019 AAAI Conference on Human Computation and Crowdsourcing. https://doi.org/10.48550/arXiv.1909.03567
  68. Attitudes and Attitude Change. Annual Review of Psychology 48, 1 (1997), 609–647. https://doi.org/10.1146/annurev-psych-122216-011911
  69. Manipulating and Measuring Model Interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445315
  70. Iterative Teaching by Data Hallucination. In Proceedings of the 2023 International Conference on Artificial Intelligence and Statistics. https://doi.org/10.48550/arXiv.2210.17467
  71. Amy Rechkemmer and Ming Yin. 2022. When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3501967
  72. Guidelines for Adolescent Well Care: Is There Consensus? Current Opinion in Pediatrics 18, 4 (2006), 365–370. https://doi.org/10.1097/01.mop.0000236383.41531.8e
  73. Jon Roozenbeek and Sander Van der Linden. 2019. Fake News Game Confers Psychological Resistance Against Online Misinformation. Palgrave Communications 5, 1 (2019), 1–10. https://doi.org/10.2478/plc-2013-0009
  74. Cheap Talk and Credibility: The Consequences of Confidence and Accuracy on Advisor Credibility and Persuasiveness. Organizational Behavior and Human Decision Processes 121, 2 (2013), 246–255. https://doi.org/10.1016/j.obhdp.2013.02.001
  75. Measuring Non-expert Comprehension of Machine Learning Fairness Metrics. In Proceedings of the 2020 International Conference on Machine Learning. https://doi.org/10.48550/arXiv.2001.00089
  76. A Framework of High-stakes Algorithmic Decision-making for the Public Sector Developed through a Case Study of Child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. https://doi.org/10.1145/3476089
  77. Algorithmic Appreciation or Aversion? The Moderating Effects of Uncertainty on Algorithmic Decision Making. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544549.3585908
  78. Effects of Distance Between Initial Estimates and Advice on Advice Utilization. Judgment and Decision Making 10, 2 (2015), 144–171. https://doi.org/10.1017/S1930297500003922
  79. Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-based Treatment Recommendations in Health Care. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581075
  80. Shyam Sundar and Jinyoung Kim. 2019. Machine Heuristic: When We Trust Computers More than Humans with Our Personal Information. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300768
  81. Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517732
  82. Teaching an Active Learner with Contrastive Examples. Advances in Neural Information Processing Systems 34 (2021), 17968–17980. https://doi.org/10.48550/arXiv.2110.14888
  83. Factors Influencing Perceived Fairness in Algorithmic Decision-making: Algorithm Outcomes, Development Procedures, and Individual Differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376813
  84. Xinru Wang and Ming Yin. 2021. Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-assisted Decision-making. In Proceedings of the 2021 International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3397481.3450650
  85. Timothy D. Wilson and Nancy Brekke. 1994. Mental Contamination and Mental Correction: Unwanted Influences on Judgments and Evaluations. Psychological Bulletin 116, 1 (1994), 117. https://doi.org/10.1037/0033-2909.116.1.117
  86. Iterative Classroom Teaching. In Proceedings of the 2019 AAAI Conference on Artificial Intelligence. https://doi.org/10.1609/aaai.v33i01.33015684
  87. Mitigating Unwanted Biases with Adversarial Learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3278721.3278779
  88. Unlearn What You Have Learned: Adaptive Crowd Teaching with Exponentially Decayed Memory Learners. In Proceedings of the 2018 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/3219819.3219952
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Mingzhe Yang (9 papers)
  2. Hiromi Arai (9 papers)
  3. Naomi Yamashita (10 papers)
  4. Yukino Baba (13 papers)
Citations (3)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets