Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making (2312.11507v1)

Published 11 Dec 2023 in cs.HC and cs.LG

Abstract: The unprecedented performance of machine learning models in recent years, particularly Deep Learning and transformer models, has resulted in their application in various domains such as finance, healthcare, and education. However, the models are error-prone and cannot be used autonomously, especially in decision-making scenarios where, technically or ethically, the cost of error is high. Moreover, because of the black-box nature of these models, it is frequently difficult for the end user to comprehend the models' outcomes and underlying processes to trust and use the model outcome to make a decision. Explainable Artificial Intelligence (XAI) aids end-user understanding of the model by utilizing approaches, including visualization techniques, to explain and interpret the inner workings of the model and how it arrives at a result. Although numerous research studies have been conducted recently focusing on the performance of models and the XAI approaches, less work has been done on the impact of explanations on human-AI team performance. This paper surveyed the recent empirical studies on XAI's impact on human-AI decision-making, identified the challenges, and proposed future research directions.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (113)
  1. Susmita Ray. A quick review of machine learning algorithms. In 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon), pages 35–39. IEEE, 2019.
  2. Artificial intelligence for decision making in the era of big data–evolution, challenges and research agenda. International journal of information management, 48:63–71, 2019.
  3. State-of-the-art in artificial neural network applications: A survey. Heliyon, 4(11):e00938, 2018.
  4. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016.
  5. Neural network based models for software effort estimation: a review. Artificial Intelligence Review, 42(2):295–307, 2014.
  6. Gan-based tabular data generator for constructing synopsis in approximate query processing: Challenges and solutions. arXiv preprint arXiv:2212.09015, 2022.
  7. A cti v is: Visual exploration of industry-scale deep neural network models. IEEE transactions on visualization and computer graphics, 24(1):88–97, 2017.
  8. On the explainability of natural language processing deep models. ACM Comput. Surv., 55(5), dec 2022.
  9. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296, 2017.
  10. A survey of the state of explainable ai for natural language processing. arXiv preprint arXiv:2010.00711, 2020.
  11. Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57, 2018.
  12. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems, pages 1–12, 2019.
  13. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608, 2017.
  14. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  15. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053, 2020.
  16. Understanding uncertainty: How lay decision-makers perceive and interpret uncertainty in human-ai decision making. In Proceedings of the 28th International Conference on Intelligent User Interfaces, pages 379–396, 2023.
  17. Current challenges and future opportunities for xai in machine learning-based clinical decision support systems: a systematic review. Applied Sciences, 11(11):5088, 2021.
  18. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2017.
  19. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 295–305, 2020.
  20. Towards a science of human-ai decision making: An overview of design space in empirical human-subject studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1369–1385, 2023.
  21. Darpa’s explainable artificial intelligence (xai) program. AI magazine, 40(2):44–58, 2019.
  22. Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3):247–278, 2021.
  23. Interpretability of machine learning models and representations: an introduction. In 24th european symposium on artificial neural networks, computational intelligence and machine learning, pages 77–82. CIACO, 2016.
  24. Feature-based explanations don’t help people detect misclassifications of online toxicity. In Proceedings of the international AAAI conference on web and social media, volume 14, pages 95–106, 2020.
  25. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021.
  26. A study of real-time information on user behaviors during search and rescue (sar) training of firefighters. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pages 387–394, 2022.
  27. Sibyl: Understanding and addressing the usability challenges of machine learning in high-stakes decision making. IEEE Transactions on Visualization and Computer Graphics, 28(1):1161–1171, 2021.
  28. Harry Surden. Artificial intelligence and law: An overview. Georgia State University Law Review, 35:19–22, 2019.
  29. Harry Surden. The ethics of artificial intelligence in law: Basic questions. Forthcoming chapter in Oxford Handbook of Ethics of AI, pages 19–29, 2020.
  30. Fast facts: Preventing child abuse & neglect. https://www.cdc.gov/violenceprevention/childabuseandneglect/fastfact.html, Apr 2022. Accessed: November 04, 2023.
  31. Cara Kelly. Child Maltreatment 2021. U.S. Department of Health & Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau, 2023.
  32. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny county methodology and implementation. Center for Social data Analytics, 2017.
  33. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 543–550, 2004.
  34. Sarah T Roberts. Digital detritus:’error’and the logic of opacity in social media content moderation. First Monday, 2018.
  35. Tarleton Gillespie. Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press, 2018.
  36. David Kaye. Speech police: The global struggle to govern the Internet. Columbia Global Reports, New York, NY, 2019.
  37. What do we mean when we talk about transparency? toward meaningful transparency in commercial content moderation. International Journal of Communication, 13:18, 2019.
  38. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1):2053951719897945, 2020.
  39. Global internet forum to counter terrorism. Global internet forum to counter terrorism: About. https://perma.cc/44V5-554U?type=standard, Apr 2019. Accessed: November 05, 2023.
  40. Machine learning techniques for hate speech classification of twitter data: State-of-the-art, future challenges and research directions. Computer Science Review, 38:100311, 2020.
  41. Fuzzy multi-task learning for hate speech type identification. In The world wide web conference, pages 3006–3012, 2019.
  42. Nikhil Chakravartula. Hateminer at semeval-2019 task 5: hate speech detection against immigrants and women in twitter using a multinomial naive bayes classifier. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 404–408, 2019.
  43. A preventive measure on hate speech detection on online social network using naïve bayes. -, 2020.
  44. Using naïve bayes algorithm in detection of hate tweets. International Journal of Scientific and Research Publications, 8(3):99–107, 2018.
  45. Mineriaunam at semeval-2019 task 5: Detecting hate speech in twitter using multiple features in a combinatorial framework. In Proceedings of the 13th international workshop on semantic evaluation, pages 447–452, 2019.
  46. Ua at semeval-2019 task 5: setting a strong linear baseline for hate speech detection. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 508–513, 2019.
  47. Vais hate speech detection system: A deep learning based approach for system combination. arXiv preprint arXiv:1910.05608, 2019.
  48. Mandola: A big-data processing and visualization platform for monitoring and detecting online hate speech. ACM Trans. Internet Technol., 20(2), mar 2020.
  49. German hatespeech classification with naive bayes and logistic regression-hshl at germeval 2019-task 2. In KONVENS, 2019.
  50. Ynu nlp at semeval-2019 task 5: Attention and capsule ensemble for identifying hate speech. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 529–534, 2019.
  51. Anna Liu. Neural network models for hate speech classification in tweets. PhD thesis, Harvard University, 2018.
  52. Prediction uncertainty estimation for hate speech classification. In Statistical Language and Speech Processing: 7th International Conference, SLSP 2019, Ljubljana, Slovenia, October 14–16, 2019, Proceedings 7, pages 286–298. Springer, 2019.
  53. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information fusion, 58:82–115, 2020.
  54. A guide to deep learning in healthcare. Nature medicine, 25(1):24–29, 2019.
  55. Deep learning for healthcare: review, opportunities and challenges. Briefings in bioinformatics, 19(6):1236–1246, 2018.
  56. Deep learning for finance: deep portfolios. Applied Stochastic Models in Business and Industry, 33(1):3–12, 2017.
  57. Deep learning in finance and banking: A literature review and classification. Frontiers of Business Research in China, 14(1):1–24, 2020.
  58. Context-aware legal citation recommendation using deep learning. In Proceedings of the eighteenth international conference on artificial intelligence and law, pages 79–88, 2021.
  59. Empirical study of deep learning for text classification in legal document review. In 2018 IEEE International Conference on Big Data (Big Data), pages 3317–3320. IEEE, 2018.
  60. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020.
  61. Explainable ai for designers: A human-centered perspective on mixed-initiative co-creation. In 2018 IEEE conference on computational intelligence and games (CIG), pages 1–8. IEEE, 2018.
  62. Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE access, 6:52138–52160, 2018.
  63. Or Biran and Courtenay Cotton. Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI), volume 8, pages 8–13, 2017.
  64. Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38, 2019.
  65. Explaining explanations: An overview of interpretability of machine learning. arXiv e-prints, pages arXiv–1806, 2018.
  66. Melissa Hamilton. The sexist algorithm. Behavioral sciences & the law, 37(2):145–157, 2019.
  67. Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
  68. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198–4205, Online, July 2020. Association for Computational Linguistics.
  69. Evaluating saliency methods for neural language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5034–5052, Online, June 2021. Association for Computational Linguistics.
  70. A fine-grained interpretability evaluation benchmark for neural nlp. arXiv preprint arXiv:2205.11097, 2022.
  71. Explainable ai: A brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II 8, pages 563–574. Springer, 2019.
  72. A multidisciplinary survey and framework for design and evaluation of explainable ai systems. ACM Trans. Interact. Intell. Syst., 11(3–4), sep 2021.
  73. Gulsum Alicioglu and Bo Sun. A survey of visual analytics for explainable artificial intelligence methods. Computers & Graphics, 102:502–520, 2022.
  74. Effects of explainable artificial intelligence on trust and human behavior in a high-risk decision task. Computers in Human Behavior, 139:107539, 2023.
  75. Appropriate reliance on ai advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI ’23, page 410–422, New York, NY, USA, 2023. Association for Computing Machinery.
  76. Explanations can reduce overreliance on ai systems during decision-making. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1):1–38, 2023.
  77. Human-ai collaboration via conditional delegation: A case study of content moderation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2022.
  78. Does explainable artificial intelligence improve human decision-making? Proceedings of the AAAI Conference on Artificial Intelligence, 35(8):6618–6626, May 2021.
  79. Deep learning in image classification using residual network (resnet) variants for detection of colorectal cancer. Procedia Computer Science, 179:423–431, 2021.
  80. ’it’s reducing a human being to a percentage’ perceptions of justice in algorithmic decisions. In Proceedings of the 2018 Chi conference on human factors in computing systems, pages 1–14, 2018.
  81. A meta-analysis of the utility of explainable artificial intelligence in human-ai decision-making. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 617–626, 2022.
  82. Evidence accumulation in decision making: Unifying the “take the best” and the “rational” models. Psychonomic bulletin & review, 11(2):343–352, 2004.
  83. Herbert A Simon. From substantive to procedural rationality. In 25 years of economic theory: Retrospect and prospect, pages 65–86. Springer, 1976.
  84. Fast and frugal heuristics: The adaptive toolbox. In Simple heuristics that make us smart, pages 3–34. Oxford University Press, 1999.
  85. Using cognitive decision models to prioritize e-mails. In Proceedings of the Twenty-fourth Annual Conference of the Cognitive Science Society, pages 578–583. Routledge, 2019.
  86. A comparison of sequential sampling models for two-choice reaction time. Psychological review, 111(2):333, 2004.
  87. Sequential sampling models of human text classification. Cognitive Science, 27(2):159–193, 2003.
  88. Selecting methods for the analysis of reliance on automation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52(4):287–291, 2008.
  89. Trust in automation: Designing for appropriate reliance. Human factors, 46(1):50–80, 2004.
  90. The media equation: How people treat computers, television, and new media like real people. Cambridge, UK, 10(10), 1996.
  91. Human reliance on machine learning models when performance feedback is limited: Heuristics and risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021.
  92. Interpretable machine learning: definitions, methods, and applications. arXiv preprint arXiv:1901.04592, 2019.
  93. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  94. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  95. Finding deceptive opinion spam by any stretch of the imagination. arXiv preprint arXiv:1107.4557, 2011.
  96. Negative deceptive opinion spam. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 497–501, 2013.
  97. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, WWW ’17, page 1391–1399, Republic and Canton of Geneva, CHE, 2017. International World Wide Web Conferences Steering Committee.
  98. A benchmark dataset for learning to intervene in online hate speech. arXiv preprint arXiv:1909.04251, 2019.
  99. Rationalizing neural predictions. arXiv preprint arXiv:1606.04155, 2016.
  100. Learning attitudes and attributes from multi-aspect reviews. In 2012 IEEE 12th International Conference on Data Mining, pages 1020–1025. IEEE, 2012.
  101. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pages 507–517, 2016.
  102. Reclor: A reading comprehension dataset requiring logical reasoning. arXiv preprint arXiv:2002.04326, 2020.
  103. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
  104. Paz Sastre Domínguez and Ángel Juan Gordo López. Data activism versus algorithmic control. new governance models, old asymmetries. IC Revista Cientifica de Informacion y Comunicacion, 16:183–208, 2019.
  105. Uci machine learning repository. -, 2017.
  106. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  107. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  108. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency, pages 29–38, 2019.
  109. " why is’ chicago’deceptive?" towards building model-driven tutorials for humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2020.
  110. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1–52, 2021.
  111. To trust or to think: cognitive forcing functions can reduce overreliance on ai in ai-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1):1–21, 2021.
  112. A bayesian cognition approach for belief updating of correlation judgement through uncertainty visualizations. IEEE Transactions on Visualization and Computer Graphics, 27(2):978–988, 2021.
  113. When do data visualizations persuade? the impact of prior attitudes on learning about correlations from scatterplot visualizations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA, 2023. Association for Computing Machinery.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (1)
  1. Milad Rogha (3 papers)