Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations (2307.00364v2)

Published 1 Jul 2023 in cs.LG, cs.AI, cs.CY, and cs.HC

Abstract: Explainable Artificial Intelligence (XAI) plays a crucial role in enabling human understanding and trust in deep learning systems. As models get larger, more ubiquitous, and pervasive in aspects of daily life, explainability is necessary to minimize adverse effects of model mistakes. Unfortunately, current approaches in human-centric XAI (e.g. predictive tasks in healthcare, education, or personalized ads) tend to rely on a single post-hoc explainer, whereas recent work has identified systematic disagreement between post-hoc explainers when applied to the same instances of underlying black-box models. In this paper, we therefore present a call for action to address the limitations of current state-of-the-art explainers. We propose a shift from post-hoc explainability to designing interpretable neural network architectures. We identify five needs of human-centric XAI (real-time, accurate, actionable, human-interpretable, and consistent) and propose two schemes for interpretable-by-design neural network workflows (adaptive routing with InterpretCC and temporal diagnostics with I2MD). We postulate that the future of human-centric XAI is neither in explaining black-boxes nor in reverting to traditional, interpretable models, but in neural networks that are intrinsically interpretable.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI).  In IEEE access, Vol. 6, pp. 52138–52160. IEEE.
  2. OpenXAI: Towards a transparent evaluation of model explanations.  In Advances in Neural Information Processing Systems, Vol. 35, pp. 15784–15799.
  3. Ripple: Concept-based interpretation for raw time series models in education.  In The 37th AAAI Conference on Artificial Intelligence (EAAI).
  4. Estimating or propagating gradients through stochastic neurons for conditional computation.  In arXiv preprint arXiv:1308.3432.
  5. Disagreement amongst counterfactual explanations: How transparency can be deceptive..
  6. AI in education needs interpretable machine learning: Lessons from open learner modelling.  In International Conference on Machine Learning.
  7. Fairness via explanation quality: Evaluating disparities in the quality of post hoc explanations.  In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pp. 203–214.
  8. Explanations based on the missing: Towards contrastive explanations with pertinent negatives.  In Neural Information Processing Systems.
  9. Explainable artificial intelligence: A survey.  In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), pp. 0210–0215. IEEE.
  10. Predictive medicine using interpretable recurrent neural networks.  In International Conference on Pattern Recognition.
  11. Counterfactual visual explanations.  In PMLR.
  12. Shap and lime: An evaluation of discriminative power in credit risk.  In Frontiers in Artificial Intelligence.
  13. Explainable artificial intelligence (XAI) from a user perspective: A synthesis of prior literature and problematizing avenues for future research.  In Technological Forecasting and Social Change, Vol. 186, p. 122120. Elsevier.
  14. Explainable artificial intelligence (XAI): how the visualization of AI predictions affects user cognitive load and confidence.  In Information Systems and Neuroscience: NeuroIS Retreat 2021, pp. 237–246. Springer.
  15. Towards realistic individual recourse and actionable explanations in black-box decision making systems.  In arXiv preprint arXiv:1907.09615.
  16. Building interpretable predictive models for pediatric hospital readmission using tree-lasso logistic regression.  In Artificial intelligence in medicine, Vol. 72, pp. 12–21. Elsevier.
  17. Highly accurate protein structure prediction with alphafold.  In Nature, Vol. 596, pp. 583–589. Nature Publishing Group UK London.
  18. Designing for confidence: The impact of visualizing artificial intelligence decisions.  In Frontiers in Neuroscience, Vol. 16. Frontiers Media SA.
  19. Machine learning model interpretability for precision medicine.  In arXiv preprint arXiv:1610.09045.
  20. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV).  In ICML.
  21. Alibi explain: algorithms for explaining machine learning models.  In JMLR.
  22. The disagreement problem in explainable machine learning: A practitioner’s perspective.  In arXiv preprint arXiv:2202.01602.
  23. Effects of explainable artificial intelligence on trust and human behavior in a high-risk decision task.  In Computers in Human Behavior, Vol. 139, p. 107539. Elsevier.
  24. Algorithmic hiring in practice: Recruiter and hr professional’s perspectives on AI use in hiring.  In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 166–176.
  25. Towards interpretable deep learning models for knowledge tracing.  In Artificial Intelligence in Education.
  26. On interpretability of deep learning based skin lesion classifiers using concept activation vectors.  In 2020 international joint conference on neural networks (IJCNN), pp. 1–10. IEEE.
  27. A unified approach to interpreting model predictions.  In Neural Information Processing Systems.
  28. Explaining data-driven document classifications.  In Management Information Systems Quarterly.
  29. But are you sure? an uncertainty-aware perspective on explainable AI.  In International Conference on Artificial Intelligence and Statistics, pp. 7375–7391. PMLR.
  30. Explaining machine learning classifiers through diverse counterfactual explanations.  In Conference on Fairness, Accountability, and Transparency, pp. 607–617.
  31. NASEM (2021). Human-AI teaming: State-of-the-art and research needs.  In National Academy of Sciences, Engineering, and Medicine.
  32. Learning model-agnostic counterfactual explanations for tabular data.  In The Web Conference.
  33. Why should I trust you?: Explaining the predictions of any classifier.  In KDD.
  34. Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  In Nature Machine Intelligence, Vol. 1, pp. 206–215. Nature Publishing Group UK London.
  35. Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE international conference on computer vision, pp. 618–626.
  36. Fooling lime and shap: Adversarial attacks on post hoc explanation methods.  In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pp. 180–186.
  37. Trusting the explainers: Teacher validation of explainable artificial intelligence for course design.  In LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 345–356.
  38. Evaluating the explainers: Black-box explainable machine learning for student success prediction in MOOCs.  In Educational Data Mining.
  39. Interpreting language models through knowledge graph extraction.  In NeurIPS Explainable AI Workshop.
  40. Improving students’ performance by interpretable explanations using ensemble tree-based approaches.  In IEEE International Symposium on Applied Computational Intelligence and Informatics.
  41. Machine learning for human learners: opportunities, issues, tensions and threats.  In Education Tech Research and Development.
  42. Real-time prediction of taxi demand using recurrent neural networks.  In IEEE Transactions on Intelligent Transportation Systems, Vol. 19, pp. 2572–2581. IEEE.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Vinitra Swamy (15 papers)
  2. Jibril Frej (13 papers)
  3. Tanja Käser (45 papers)
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets