Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective (2401.04374v2)

Published 9 Jan 2024 in cs.AI and cs.LG

Abstract: Given the complexity and lack of transparency in deep neural networks (DNNs), extensive efforts have been made to make these systems more interpretable or explain their behaviors in accessible terms. Unlike most reviews, which focus on algorithmic and model-centric perspectives, this work takes a "data-centric" view, examining how data collection, processing, and analysis contribute to explainable AI (XAI). We categorize existing work into three categories subject to their purposes: interpretations of deep models, referring to feature attributions and reasoning processes that correlate data points with model outputs; influences of training data, examining the impact of training data nuances, such as data valuation and sample anomalies, on decision-making processes; and insights of domain knowledge, discovering latent patterns and fostering new knowledge from data and models to advance social values and scientific discovery. Specifically, we distill XAI methodologies into data mining operations on training and testing data across modalities, such as images, text, and tabular data, as well as on training logs, checkpoints, models and other DNN behavior descriptors. In this way, our study offers a comprehensive, data-centric examination of XAI from a lens of data mining methods and applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (275)
  1. F. Rosenblatt, “The perceptron: a probabilistic model for information storage and organization in the brain.” Psychological review, vol. 65, no. 6, p. 386, 1958.
  2. F. Hayes-Roth, “Rule-based systems,” Communications of the ACM, vol. 28, no. 9, pp. 921–932, 1985.
  3. J. L. Kolodner, “An introduction to case-based reasoning,” Artificial Intelligence Review, vol. 6, no. 1, pp. 3–34, 1992.
  4. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, 2015.
  5. D. Frye, P. D. Zelazo, and T. Palfai, “Theory of mind and rule-based reasoning,” Cognitive Development, vol. 10, no. 4, pp. 483–527, 1995.
  6. S. Dutta, “Strategies for implementing knowledge-based systems,” IEEE Transactions on Engineering Management, vol. 44, no. 1, pp. 79–90, 1997.
  7. F. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable machine learning,” arXiv preprint arXiv:1702.08608, 2017.
  8. D. V. Carvalho, E. M. Pereira, and J. S. Cardoso, “Machine learning interpretability: A survey on methods and metrics,” Electronics, 2019.
  9. T. Jing, H. Xia, R. Tian, H. Ding, X. Luo, J. Domeyer, R. Sherony, and Z. Ding, “Inaction: Interpretable action decision making for autonomous driving,” in European Conference on Computer Vision.   Springer, 2022, pp. 370–387.
  10. S. Wang, X. Du, G. Liu, H. Xing, Z. Jiao, J. Yan, Y. Liu, H. Lv, and Y. Xia, “An interpretable data-driven medical knowledge discovery pipeline based on artificial intelligence,” IEEE Journal of Biomedical and Health Informatics, 2023.
  11. M. T. Ribeiro, S. Singh, and C. Guestrin, “" why should i trust you?" explaining the predictions of any classifier,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
  12. E. Toreini, M. Aitken, K. Coopamootoo, K. Elliott, C. G. Zelaya, and A. Van Moorsel, “The relationship between trust in ai and trustworthy machine learning technologies,” in Proceedings of the 2020 conference on fairness, accountability, and transparency, 2020, pp. 272–283.
  13. S. Afzal, A. Chaudhary, N. Gupta, H. Patel, C. Spina, and D. Wang, “Data-debugging through interactive visual explanations,” in Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2021 Workshops, WSPA, MLMEIN, SDPRA, DARAI, and AI4EPT, Delhi, India, May 11, 2021 Proceedings 25.   Springer, 2021, pp. 133–142.
  14. Q. Jia, X. Li, L. Yu, J. Bian, P. Zhao, S. Li, H. Xiong, and D. Dou, “Learning from training dynamics: Identifying mislabeled data beyond manually designed features,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  15. R. Marcinkevičs and J. E. Vogt, “Interpretable and explainable machine learning: A methods-centric overview with concrete examples,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, p. e1493, 2023.
  16. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE International Conference on Computer Vision, 2017.
  17. X. Zhang, X. Zhu, and S. Wright, “Training set debugging using trusted items,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018.
  18. P. W. Koh and P. Liang, “Understanding black-box predictions via influence functions,” in Proceedings of the International Conference on Machine Learning.   PMLR, 2017.
  19. V. Feldman and C. Zhang, “What neural networks memorize and why: Discovering the long tail via influence estimation,” in Proceedings of the Advances in Neural Information Processing Systems, 2020.
  20. A. Datta, S. Sen, and Y. Zick, “Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems,” in 2016 IEEE symposium on security and privacy (S&P).   IEEE, 2016, pp. 598–617.
  21. C.-K. Yeh, A. Taly, M. Sundararajan, F. Liu, and P. Ravikumar, “First is better than last for language data influence,” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35.   Curran Associates, Inc., 2022, pp. 32 285–32 298. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/d07022783ff6f7bf7a288c207b7dcbd1-Paper-Conference.pdf
  22. S. Ghosh, P. Tino, and K. Bunte, “Visualisation and knowledge discovery from interpretable models,” in 2020 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2020, pp. 1–8.
  23. C. Zednik and H. Boelsen, “Scientific exploration and explainable artificial intelligence,” Minds and Machines, vol. 32, no. 1, pp. 219–239, 2022.
  24. X. Li, H. Xiong, X. Li, X. Wu, X. Zhang, J. Liu, J. Bian, and D. Dou, “Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond,” Knowledge and Information Systems, vol. 64, no. 12, pp. 3197–3234, 2022.
  25. M. Nauta, J. H. Hegeman, J. Geerdink, J. Schlötterer, M. van Keulen, and C. Seifert, “Interpreting and correcting medical image classification with pip-net,” arXiv preprint arXiv:2307.10404, 2023.
  26. M.-S. Chen, J. Han, and P. S. Yu, “Data mining: an overview from a database perspective,” IEEE Transactions on Knowledge and data Engineering, vol. 8, no. 6, pp. 866–883, 1996.
  27. X. Wu, X. Zhu, G.-Q. Wu, and W. Ding, “Data mining with big data,” IEEE transactions on knowledge and data engineering, vol. 26, no. 1, pp. 97–107, 2013.
  28. E. Tjoa and C. Guan, “A survey on explainable artificial intelligence (xai): Toward medical xai,” IEEE Transactions on Neural Networks and Learning Systems, 2020.
  29. Y. Feng and Y. Tu, “Phases of learning dynamics in artificial neural networks in the absence or presence of mislabeled data,” Machine Learning: Science and Technology, vol. 2, no. 4, p. 043001, 2021.
  30. Z. Qi, S. Khorram, and F. Li, “Visualizing deep networks by optimizing with integrated gradients.” in CVPR Workshops, vol. 2, 2019, pp. 1–4.
  31. Z. J. Wang, R. Turko, O. Shaikh, H. Park, N. Das, F. Hohman, M. Kahng, and D. H. P. Chau, “Cnn explainer: learning convolutional neural networks with interactive visualization,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, pp. 1396–1406, 2020.
  32. D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and P. Group*, “Preferred reporting items for systematic reviews and meta-analyses: the prisma statement,” Annals of Internal Medicine, vol. 151, no. 4, pp. 264–269, 2009.
  33. X. Li, H. Xiong, X. Li, X. Zhang, J. Liu, H. Jiang, Z. Chen, and D. Dou, “G-lime: Statistical learning for local interpretations of deep neural networks using global priors,” Artificial Intelligence, vol. 314, p. 103823, 2023.
  34. L. Merrick, “Randomized ablation feature importance,” arXiv preprint arXiv:1910.00174, 2019.
  35. D. Ramaswamy and G. Harish, “Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization,” in Proceedings of the Winter Conference on Applications of Computer Vision, 2020.
  36. S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Proceedings of the Advances in Neural Information Processing Systems, 2017.
  37. M. Sundararajan and A. Najmi, “The many shapley values for model explanation,” in International conference on machine learning.   PMLR, 2020, pp. 9269–9278.
  38. K. Aas, M. Jullum, and A. Løland, “Explaining individual predictions when features are dependent: More accurate approximations to shapley values,” Artificial Intelligence, vol. 298, p. 103502, 2021.
  39. H. Zhang, Y. Xie, L. Zheng, D. Zhang, and Q. Zhang, “Interpreting multivariate shapley interactions in dnns,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, 2021, pp. 10 877–10 886.
  40. M. Sundararajan, K. Dhamdhere, and A. Agarwal, “The shapley taylor interaction index,” in International conference on machine learning.   PMLR, 2020, pp. 9259–9268.
  41. D. D. Lundstrom, T. Huang, and M. Razaviyayn, “A rigorous study of integrated gradients method and extensions to internal neuron attributions,” in International Conference on Machine Learning.   PMLR, 2022, pp. 14 485–14 508.
  42. D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” arXiv preprint arXiv:1706.03825, 2017.
  43. A. Shrikumar, P. Greenside, and A. Kundaje, “Learning important features through propagating activation differences,” in International conference on machine learning.   PMLR, 2017, pp. 3145–3153.
  44. Z. T. Fernando, J. Singh, and A. Anand, “A study on the interpretability of neural retrieval models using deepshap,” in Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 2019, pp. 1005–1008.
  45. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016.
  46. S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2020.
  47. E. Voita, D. Talbot, F. Moiseev, R. Sennrich, and I. Titov, “Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
  48. L. Xu, X. Yan, W. Ding, and Z. Liu, “Attribution rollout: a new way to interpret visual transformer,” Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 1, pp. 163–173, 2023.
  49. J. Chen, X. Li, L. Yu, D. Dou, and H. Xiong, “Beyond intuition: Rethinking token attributions inside transformers,” Transactions on Machine Learning Research, 2022.
  50. N. Frosst and G. Hinton, “Distilling a neural network into a soft decision tree,” arXiv preprint arXiv:1711.09784, 2017.
  51. F. Di Castro and E. Bertini, “Surrogate decision tree visualization interpreting and visualizing black-box classification models with surrogate decision tree,” in CEUR Workshop Proceedings, vol. 2327.   CEUR-WS, 2019.
  52. P. Kontschieder, M. Fiterau, A. Criminisi, and S. R. Bulo, “Deep neural decision forests,” in Proceedings of the IEEE International Conference on Computer Vision, 2015.
  53. G. Montavon, S. Lapuschkin, A. Binder, W. Samek, and K.-R. Müller, “Explaining nonlinear classification decisions with deep taylor decomposition,” Pattern Recognition, 2017.
  54. D. Bau, B. Zhou, A. Khosla, A. Oliva, and A. Torralba, “Network dissection: Quantifying interpretability of deep visual representations,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
  55. B. Zhou, D. Bau, A. Oliva, and A. Torralba, “Interpreting deep visual representations via network dissection,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 9, pp. 2131–2145, 2018.
  56. J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, “Understanding neural networks through deep visualization,” arXiv preprint arXiv:1506.06579, 2015.
  57. V. Couteaux, O. Nempont, G. Pizaine, and I. Bloch, “Towards interpretability of segmentation networks by analyzing deepdreams,” in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support: Second International Workshop, iMIMIC 2019, and 9th International Workshop, ML-CDS 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Proceedings 9.   Springer, 2019, pp. 56–63.
  58. M. B. Muhammad and M. Yeasin, “Eigen-cam: Class activation map using principal components,” in 2020 international joint conference on neural networks (IJCNN).   IEEE, 2020, pp. 1–7.
  59. L.-D. Quach, K. N. Quoc, A. N. Quynh, N. N. Thai, and T. G. Nguyen, “Using gradient-weighted class activation mapping to explain deep learning models on agricultural dataset,” IEEE Access, 2023.
  60. M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13.   Springer, 2014, pp. 818–833.
  61. O. Boz, “Extracting decision trees from trained neural networks,” in Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 2002, pp. 456–461.
  62. Q. Zhang, Y. Yang, H. Ma, and Y. N. Wu, “Interpreting cnns via decision trees,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
  63. C.-H. Chang, E. Creager, A. Goldenberg, and D. Duvenaud, “Explaining image classifiers by counterfactual generation,” arXiv preprint arXiv:1807.08024, 2018.
  64. R. K. Mothilal, A. Sharma, and C. Tan, “Explaining machine learning classifiers through diverse counterfactual explanations,” in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 2020.
  65. Y. Goyal, Z. Wu, J. Ernst, D. Batra, D. Parikh, and S. Lee, “Counterfactual visual explanations,” in Proceedings of the International Conference on Machine Learning.   PMLR, 2019.
  66. T. Laugel, M.-J. Lesot, C. Marsala, X. Renard, and M. Detyniecki, “Unjustified classification regions and counterfactual explanations in machine learning,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2019.
  67. A. Ilyas, S. Santurkar, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” in Proceedings of the Advances in Neural Information Processing Systems, 2019.
  68. C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su, “This looks like that: deep learning for interpretable image recognition,” Advances in neural information processing systems, vol. 32, 2019.
  69. J. Donnelly, A. J. Barnett, and C. Chen, “Deformable protopnet: An interpretable image classifier using deformable prototypes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 265–10 275.
  70. M. Nauta, J. Schlötterer, M. van Keulen, and C. Seifert, “Pip-net: Patch-based intuitive prototypes for interpretable image classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 2744–2753.
  71. G. Singh, “Think positive: An interpretable neural network for image recognition,” Neural Networks, vol. 151, pp. 178–189, 2022.
  72. M. Nauta and C. Seifert, “The co-12 recipe for evaluating interpretable part-prototype image classifiers,” in World Conference on Explainable Artificial Intelligence.   Springer, 2023, pp. 397–420.
  73. J. Wang, P. Jiajie, Z. Liu, and H. Zhao, “Hqprotopnet: An evidence-based model for interpretable image recognition,” in 2023 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2023, pp. 1–8.
  74. J. Bien and R. Tibshirani, “Prototype selection for interpretable classification,” The Annals of Applied Statistics, 2011.
  75. B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas et al., “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav),” in International conference on machine learning.   PMLR, 2018, pp. 2668–2677.
  76. R. Zhang, P. Madumal, T. Miller, K. A. Ehinger, and B. I. Rubinstein, “Invertible concept-based explanations for cnn models with non-negative concept activation vectors,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 13, 2021, pp. 11 682–11 690.
  77. M. Moayeri, K. Rezaei, M. Sanjabi, and S. Feizi, “Text2concept: Concept activation vectors directly from text,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3743–3748.
  78. J. Crabbé and M. van der Schaar, “Concept activation regions: A generalized framework for concept-based explanations,” Advances in Neural Information Processing Systems, vol. 35, pp. 2590–2607, 2022.
  79. G. Pruthi, F. Liu, S. Kale, and M. Sundararajan, “Estimating training data influence by tracing gradient descent,” in Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., 2020, pp. 19 920–19 930. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/e6385d39ec9394f2f3a354d9d2b88eec-Paper.pdf
  80. H. Guo, N. Rajani, P. Hase, M. Bansal, and C. Xiong, “Fastif: Scalable influence functions for efficient model interpretation and debugging,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 10 333–10 350.
  81. U. Bhatt, A. Weller, and G. Cherubin, “Fast conformal classification using influence functions,” in Conformal and Probabilistic Prediction and Applications.   PMLR, 2021, pp. 303–305.
  82. R. Grosse, J. Bae, C. Anil, N. Elhage, A. Tamkin, A. Tajdini, B. Steiner, D. Li, E. Durmus, E. Perez et al., “Studying large language model generalization with influence functions,” arXiv preprint arXiv:2308.03296, 2023.
  83. R. Liu, Y. Lin, X. Yang, and J. S. Dong, “Debugging and explaining metric learning approaches: An influence function based perspective,” Advances in Neural Information Processing Systems, vol. 35, pp. 7824–7837, 2022.
  84. Z. Wang, H. Zhu, Z. Dong, X. He, and S.-L. Huang, “Less is better: Unweighted data subsampling via influence function,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 6340–6347.
  85. J. Bae, N. Ng, A. Lo, M. Ghassemi, and R. B. Grosse, “If influence functions are the answer, then what is the question?” Advances in Neural Information Processing Systems, vol. 35, pp. 17 953–17 967, 2022.
  86. A. Silva, R. Chopra, and M. Gombolay, “Cross-loss influence functions to explain deep network representations,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2022, pp. 1–17.
  87. S. Bates, T. Hastie, and R. Tibshirani, “Cross-validation: what does it estimate and how well does it do it?” Journal of the American Statistical Association, pp. 1–12, 2023.
  88. B. Efron, “Jackknife-after-bootstrap standard errors and influence functions,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 54, no. 1, pp. 83–111, 1992.
  89. A. Ghorbani and J. Zou, “Data shapley: Equitable valuation of data for machine learning,” in International conference on machine learning.   PMLR, 2019, pp. 2242–2251.
  90. Y. Kwon and J. Zou, “Beta shapley: a unified and noise-reduced data valuation framework for machine learning.” in International Conference on AI and Statistics, 2022.
  91. J. T. Wang and R. Jia, “Data banzhaf: A robust data valuation framework for machine learning,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2023, pp. 6388–6421.
  92. Y. Kwon, E. Wu, K. Wu, and J. Zou, “Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models,” arXiv preprint arXiv:2310.00902, 2023.
  93. S. Schoch, H. Xu, and Y. Ji, “Cs-shapley: Class-wise shapley values for data valuation in classification,” Advances in Neural Information Processing Systems, vol. 35, pp. 34 574–34 585, 2022.
  94. H. A. Just, F. Kang, T. Wang, Y. Zeng, M. Ko, M. Jin, and R. Jia, “Lava: Data valuation without pre-specified learning algorithms,” in The Eleventh International Conference on Learning Representations, 2022.
  95. X. Zhou and O. Wu, “Which samples should be learned first: Easy or hard?” IEEE Transactions on Neural Networks and Learning Systems, 2023.
  96. E. Smirnov, A. Melnikov, A. Oleinik, E. Ivanova, I. Kalinovskiy, and E. Luckyanets, “Hard example mining with auxiliary embeddings,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 37–46.
  97. M. Toneva, A. Sordoni, R. T. des Combes, A. Trischler, Y. Bengio, and G. J. Gordon, “An empirical study of example forgetting during deep neural network learning,” in International Conference on Learning Representations, 2018.
  98. A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 761–769.
  99. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988.
  100. G. Chechik, V. Sharma, U. Shalit, and S. Bengio, “Large scale online learning of image similarity through ranking.” Journal of Machine Learning Research, vol. 11, no. 3, 2010.
  101. S. Xiang, X. Ye, J. Xia, J. Wu, Y. Chen, and S. Liu, “Interactive correction of mislabeled training data,” in 2019 IEEE Conference on Visual Analytics Science and Technology (VAST).   IEEE, 2019, pp. 57–68.
  102. W. Zhang, A. Passerini, and F. Giunchiglia, “Dealing with mislabeling via interactive machine learning,” KI-Künstliche Intelligenz, vol. 34, pp. 271–278, 2020.
  103. C. Chen, J. Yuan, Y. Lu, Y. Liu, H. Su, S. Yuan, and S. Liu, “Oodanalyzer: Interactive analysis of out-of-distribution samples,” IEEE transactions on visualization and computer graphics, vol. 27, no. 7, pp. 3335–3349, 2020.
  104. J. Huang, L. Qu, R. Jia, and B. Zhao, “O2u-net: A simple noisy label detection approach for deep neural networks,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 3326–3334.
  105. H. Valizadegan and P.-N. Tan, “Kernel based detection of mislabeled training examples,” in Proceedings of the 2007 SIAM International Conference on Data Mining.   SIAM, 2007, pp. 309–319.
  106. M.-K. Xie and S.-J. Huang, “Partial multi-label learning with noisy label identification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3676–3687, 2021.
  107. D. Guan and W. Yuan, “A survey of mislabeled training data detection techniques for pattern classification,” IETE Technical Review, vol. 30, no. 6, pp. 524–530, 2013.
  108. S. Gupta and A. Gupta, “Dealing with noise problem in machine learning data-sets: A systematic review,” Procedia Computer Science, vol. 161, pp. 466–474, 2019.
  109. H. Song, M. Kim, D. Park, Y. Shin, and J.-G. Lee, “Learning from noisy labels with deep neural networks: A survey,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  110. M. Cinquini, F. Giannotti, R. Guidotti, and A. Mattei, “Handling missing values in local post-hoc explainability,” in World Conference on Explainable Artificial Intelligence.   Springer, 2023, pp. 256–278.
  111. J. H. Sejr and A. Schneider-Kamp, “Explainable outlier detection: What, for whom and why?” Machine Learning with Applications, vol. 6, p. 100172, 2021.
  112. R. Cao and G. Lugosi, “Goodness-of-fit tests based on the kernel density estimator,” Scandinavian Journal of Statistics, vol. 32, no. 4, pp. 599–616, 2005.
  113. A. Glazer, M. Lindenbaum, and S. Markovitch, “Learning high-density regions for a generalized kolmogorov-smirnov test in high-dimensional data,” Advances in neural information processing systems, vol. 25, 2012.
  114. A. Martinez-Seras, J. Del Ser, and P. Garcia-Bringas, “Can post-hoc explanations effectively detect out-of-distribution samples?” in 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).   IEEE, 2022, pp. 1–9.
  115. R. Dua, S. Yang, Y. Li, and E. Choi, “Task agnostic and post-hoc unseen distribution detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 1350–1359.
  116. L. Holmberg, “Exploring out-of-distribution in image classification for neural networks via concepts,” in International Congress on Information and Communication Technology.   Springer, 2023, pp. 155–171.
  117. G. De Bernardi, S. Narteni, E. Cambiaso, M. Muselli, and M. Mongelli, “Weighted mutual information for out-of-distribution detection,” in World Conference on Explainable Artificial Intelligence.   Springer, 2023, pp. 318–331.
  118. S. Krishnan and E. Wu, “Palm: Machine learning explanations for iterative debugging,” in Proceedings of the 2Nd workshop on human-in-the-loop data analytics, 2017, pp. 1–6.
  119. S. Grafberger, P. Groth, J. Stoyanovich, and S. Schelter, “Data distribution debugging in machine learning pipelines,” The VLDB Journal, vol. 31, no. 5, pp. 1103–1126, 2022.
  120. D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning adversarially fair and transferable representations,” in International Conference on Machine Learning.   PMLR, 2018, pp. 3384–3393.
  121. J. Zhou and T. Joachims, “How to explain and justify almost any decision: Potential pitfalls for accountability in ai decision-making,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 12–21.
  122. J. J. Smith, A. Buhayh, A. Kathait, P. Ragothaman, N. Mattei, R. Burke, and A. Voida, “The many faces of fairness: Exploring the institutional logics of multistakeholder microlending recommendation,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 1652–1663.
  123. J. Gardner, R. Yu, Q. Nguyen, C. Brooks, and R. Kizilcec, “Cross-institutional transfer learning for educational models: Implications for model performance, fairness, and equity,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 1664–1684.
  124. B. Benbouzid, “Fairness in machine learning from the perspective of sociology of statistics: How machine learning is becoming scientific by turning its back on metrological realism,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 35–43.
  125. H. Shrestha, K. Cachel, M. Alkhathlan, E. Rundensteiner, and L. Harrison, “Help or hinder? evaluating the impact of fairness metrics and algorithms in visualizations for consensus ranking,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 1685–1698.
  126. A. Castelnovo, L. Malandri, F. Mercorio, M. Mezzanzanica, and A. Cosentini, “Towards fairness through time,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2021, pp. 647–663.
  127. E. A. Stanley, M. Wilms, P. Mouches, and N. D. Forkert, “Fairness-related performance and explainability effects in deep learning models for brain image analysis,” Journal of Medical Imaging, vol. 9, no. 6, pp. 061 102–061 102, 2022.
  128. K. Alikhademi, B. Richardson, E. Drobina, and J. E. Gilbert, “Can explainable ai explain unfairness? a framework for evaluating explainable ai,” arXiv preprint arXiv:2106.07483, 2021.
  129. J. A. McDermid, Y. Jia, Z. Porter, and I. Habli, “Artificial intelligence explainability: the technical and ethical dimensions,” Philosophical Transactions of the Royal Society A, vol. 379, no. 2207, p. 20200363, 2021.
  130. B. Rakova and R. Dobbe, “Algorithms as social-ecological-technological systems: an environmental justice lens on algorithmic audits,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 491–491.
  131. M. Jakesch, Z. Buçinca, S. Amershi, and A. Olteanu, “How different groups prioritize ethical values for responsible ai,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 310–323.
  132. M. Mccradden, O. Odusi, S. Joshi, I. Akrout, K. Ndlovu, B. Glocker, G. Maicas, X. Liu, M. Mazwi, T. Garnett et al., “What’s fair is… fair? presenting justefab, an ethical framework for operationalizing medical ethics and social justice in the integration of clinical machine learning: Justefab,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 1505–1519.
  133. A. Kasirzadeh and A. Smart, “The use and misuse of counterfactuals in ethical machine learning,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 228–236.
  134. W. Hawkins and B. Mittelstadt, “The ethical ambiguity of ai data enrichment: Measuring gaps in research ethics norms and practices,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 261–270.
  135. S. Kacianka and A. Pretschner, “Designing accountable systems,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 424–437.
  136. B. Hutchinson, A. Smart, A. Hanna, E. Denton, C. Greer, O. Kjartansson, P. Barnes, and M. Mitchell, “Towards accountability for machine learning datasets: Practices from software engineering and infrastructure,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 560–575.
  137. J. A. Kroll, “Outlining traceability: A principle for operationalizing accountability in computing systems,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 758–771.
  138. G. Lima, N. Grgić-Hlača, J. K. Jeong, and M. Cha, “The conflict between explainable and accountable decision-making algorithms,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 2103–2113.
  139. J. Donia, “Normative logics of algorithmic accountability,” in 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 598–598.
  140. N. Rostamzadeh, D. Mincu, S. Roy, A. Smart, L. Wilcox, M. Pushkarna, J. Schrouff, R. Amironesei, N. Moorosi, and K. Heller, “Healthsheet: development of a transparency artifact for health datasets,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 1943–1961.
  141. T. Schmude, L. Koesten, T. Möller, and S. Tschiatschek, “On the impact of explanations on understanding of algorithmic decision-making,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 959–970.
  142. E. Grünewald and F. Pallas, “Tilt: A gdpr-aligned transparency information language and toolkit for practical privacy engineering,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 636–646.
  143. A. Marian, “Algorithmic transparency and accountability through crowdsourcing: A study of the nyc school admission lottery,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 434–443.
  144. J. P. Lapostol Piderit, R. Garrido Iglesias, and M. P. Hermosilla Cornejo, “Algorithmic transparency from the south: Examining the state of algorithmic transparency in chile’s public administration algorithms,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 227–235.
  145. J. Jiménez-Luna, M. Skalic, N. Weskamp, and G. Schneider, “Coloring molecules with explainable artificial intelligence for preclinical relevance assessment,” Journal of Chemical Information and Modeling, vol. 61, no. 3, pp. 1083–1094, 2021.
  146. M. Proietti, A. Ragno, B. L. Rosa, R. Ragno, and R. Capobianco, “Explainable ai in drug discovery: self-interpretable graph neural network for molecular property prediction using concept whitening,” Machine Learning, pp. 1–32, 2023.
  147. P. Lemos, N. Jeffrey, M. Cranmer, S. Ho, and P. Battaglia, “Rediscovering orbital mechanics with machine learning,” Machine Learning: Science and Technology, vol. 4, no. 4, p. 045002, 2023.
  148. J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko et al., “Highly accurate protein structure prediction with alphafold,” Nature, 2021.
  149. H. Heberle, L. Zhao, S. Schmidt, T. Wolf, and J. Heinrich, “Xsmiles: interactive visualization for molecules, smiles and xai attribution scores,” Journal of Cheminformatics, vol. 15, no. 1, pp. 1–12, 2023.
  150. S. El-Sappagh, J. M. Alonso, S. R. Islam, A. M. Sultan, and K. S. Kwak, “A multilayer multimodal detection and prediction model based on explainable artificial intelligence for alzheimer’s disease,” Scientific reports, vol. 11, no. 1, p. 2660, 2021.
  151. A. Essemlali, E. St-Onge, M. Descoteaux, and P.-M. Jodoin, “Understanding alzheimer disease’s structural connectivity through explainable ai,” in Medical Imaging with Deep Learning.   PMLR, 2020, pp. 217–229.
  152. D. Ardila, A. P. Kiraly, S. Bharadwaj, B. Choi, J. J. Reicher, L. Peng, D. Tse, M. Etemadi, W. Ye, G. Corrado et al., “End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography,” Nature medicine, vol. 25, no. 6, pp. 954–961, 2019.
  153. S. Basu, L. O. Hall, D. B. Goldgof, Y. Gu, V. Kumar, J. Choi, R. J. Gillies, and R. A. Gatenby, “Developing a classifier model for lung tumors in ct-scan images,” in 2011 IEEE International Conference on Systems, Man, and Cybernetics.   IEEE, 2011, pp. 1306–1312.
  154. K. Borys, Y. A. Schmitt, M. Nauta, C. Seifert, N. Krämer, C. M. Friedrich, and F. Nensa, “Explainable ai in medical imaging: An overview for clinical practitioners–beyond saliency-based xai approaches,” European journal of radiology, p. 110786, 2023.
  155. K. A. Tarnowska, B. C. Dispoto, and J. Conragan, “Explainable ai-based clinical decision support system for hearing disorders,” AMIA Summits on Translational Science Proceedings, vol. 2021, p. 595, 2021.
  156. C. Gross and I. Bloch, “Quantum simulations with ultracold atoms in optical lattices,” Science, vol. 357, no. 6355, pp. 995–1001, 2017.
  157. R. G. Melko, G. Carleo, J. Carrasquilla, and J. I. Cirac, “Restricted boltzmann machines in quantum physics,” Nature Physics, vol. 15, no. 9, pp. 887–892, 2019.
  158. P. Keyl, P. Bischoff, G. Dernbach, M. Bockmayr, R. Fritz, D. Horst, N. Blüthgen, G. Montavon, K.-R. Müller, and F. Klauschen, “Single-cell gene regulatory network prediction by explainable ai,” Nucleic Acids Research, vol. 51, no. 4, pp. e20–e20, 2023.
  159. H. Park, K. Maruhashi, R. Yamaguchi, S. Imoto, and S. Miyano, “Global gene network exploration based on explainable artificial intelligence approach,” Plos one, vol. 15, no. 11, p. e0241508, 2020.
  160. Z. Sahin, M. Vubangsi, A. S. Mubarak, and F. Al-Turjman, “Xai uncovers an underlying quantum phenomenon in a photovoltaic materials dataset,” SSNR, 2021.
  161. A. Anguita-Ruiz, A. Segura-Delgado, R. Alcalá, C. M. Aguilera, and J. Alcalá-Fdez, “explainable artificial intelligence (xai) for the identification of biologically relevant gene expression patterns in longitudinal human studies, insights from obesity research,” PLoS computational biology, vol. 16, no. 4, p. e1007792, 2020.
  162. A. Li, S. Wu, H. Chen, Z. Du, and Q. Xia, “Explainable machine learning to uncover hydrogen diffusion mechanism in clinopyroxene,” Chemical Geology, vol. 641, p. 121776, 2023.
  163. B. Rozemberczki, L. Watson, P. Bayer, H.-T. Yang, O. Kiss, S. Nilsson, and R. Sarkar, “The shapley value in machine learning,” in The 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence.   International Joint Conferences on Artificial Intelligence Organization, 2022, pp. 5572–5579.
  164. K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034, 2013.
  165. C. H. Martin and M. W. Mahoney, “Heavy-tailed universality predicts trends in test accuracies for very large pre-trained deep neural networks,” in Proceedings of the 2020 SIAM International Conference on Data Mining.   SIAM, 2020, pp. 505–513.
  166. ——, “Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning,” The Journal of Machine Learning Research, vol. 22, no. 1, pp. 7479–7551, 2021.
  167. M. Ancona, E. Ceolini, C. Öztireli, and M. Gross, “Towards better understanding of gradient-based attribution methods for deep neural networks,” in Proceedings of the International Conference on Learning Representations, 2018.
  168. S. Basu, P. Pope, and S. Feizi, “Influence functions in deep learning are fragile,” in International Conference on Learning Representations, 2020.
  169. T. Yuan, X. Li, H. Xiong, H. Cao, and D. Dou, “Explaining information flow inside vision transformers using markov chain,” in Neural Information Processing Systems XAI4Debugging Workshop, 2021.
  170. A. Ignatiev, N. Narodytska, and J. Marques-Silva, “On relating explanations and adversarial examples,” Advances in neural information processing systems, vol. 32, 2019.
  171. R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,” Nature Machine Intelligence, 2020.
  172. Z. Hammoudeh and D. Lowd, “Training data influence analysis and estimation: A survey,” arXiv preprint arXiv:2212.04612, 2022.
  173. S. Mohseni, N. Zarei, and E. D. Ragan, “A multidisciplinary survey and framework for design and evaluation of explainable ai systems,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 11, no. 3-4, pp. 1–45, 2021.
  174. A. Notovich, H. Chalutz-Ben Gal, and I. Ben-Gal, “Explainable artificial intelligence (xai): Motivation, terminology, and taxonomy,” in Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook.   Springer, 2023, pp. 971–985.
  175. K. Preuer, G. Klambauer, F. Rippmann, S. Hochreiter, and T. Unterthiner, “Interpretable deep learning in drug discovery,” in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning.   Springer, 2019.
  176. A. Noack, I. Ahern, D. Dou, and B. Li, “An empirical study on the relation between network interpretability and adversarial robustness,” SN Computer Science, vol. 2, pp. 1–13, 2021.
  177. L. S. Shapley et al., “A value for n-person games,” 1953.
  178. E. Štrumbelj and I. Kononenko, “Explaining prediction models and individual predictions with feature contributions,” Knowledge and information systems, vol. 41, pp. 647–665, 2014.
  179. S. Srinivas and F. Fleuret, “Rethinking the role of gradient-based attribution methods for model interpretability,” in International Conference on Learning Representations, 2020.
  180. C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” 2011.
  181. A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.   Portland, Oregon, USA: Association for Computational Linguistics, June 2011, pp. 142–150. [Online]. Available: http://www.aclweb.org/anthology/P11-1015
  182. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  183. Y. Hao, L. Dong, F. Wei, and K. Xu, “Self-attention attribution: Interpreting information interactions inside transformer,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 14, 2021, pp. 12 963–12 971.
  184. X. Li, H. Xiong, S. Huang, S. Ji, and D. Dou, “Cross-model consensus of explanations and beyond for image classification models: An empirical study,” Machine Learning, vol. 112, no. 5, pp. 1627–1662, 2023.
  185. X. Li, H. Xiong, Y. Liu, D. Zhou, Z. Chen, Y. Wang, and D. Dou, “Distilling ensemble of explanations for weakly-supervised pre-training of image segmentation models,” Machine Learning, vol. 112, no. 6, pp. 2193–2209, 2023.
  186. R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,” ACM computing surveys (CSUR), vol. 51, no. 5, pp. 1–42, 2018.
  187. C. Strobl, A.-L. Boulesteix, T. Kneib, T. Augustin, and A. Zeileis, “Conditional variable importance for random forests,” BMC Bioinformatics, vol. 9, pp. 1–11, 2008.
  188. S. M. Lundberg, G. G. Erion, and S.-I. Lee, “Consistent individualized feature attribution for tree ensembles,” Methods, vol. 5, no. 13, p. 25, 2018.
  189. C. Molnar, “Global surrogate (lime),” 2020. [Online]. Available: https://christophm.github.io/interpretable-ml-book/global.html
  190. ——, “Local surrogate (lime),” 2020. [Online]. Available: https://christophm.github.io/interpretable-ml-book/lime.html
  191. A. Mahendran and A. Vedaldi, “Understanding deep image representations by inverting them,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
  192. A. Nguyen, J. Yosinski, and J. Clune, “Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks,” arXiv preprint arXiv:1602.03616, 2016.
  193. S. T. Barlow and P. Neville, “Case study: Visualization for decision tree analysis in data mining.” in infovis, 2001, pp. 149–152.
  194. J. R. Quinlan, “Generating production rules from decision trees.” in IJCAI, vol. 87.   Citeseer, 1987, pp. 304–307.
  195. B. De Ville, “Decision trees,” Wiley Interdisciplinary Reviews: Computational Statistics, vol. 5, no. 6, pp. 448–455, 2013.
  196. C. Bénard, G. Biau, S. Da Veiga, and E. Scornet, “Interpretable random forests via rule extraction,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2021, pp. 937–945.
  197. M. Golfarelli, S. Rizzi, and A. Proli, “Designing what-if analysis: towards a methodology,” in Proceedings of the 9th ACM International Workshop on Data Warehousing and OLAP, 2006, pp. 51–58.
  198. R. Moraffah, M. Karami, R. Guo, A. Raglin, and H. Liu, “Causal interpretability for machine learning-problems, methods and evaluation,” ACM SIGKDD Explorations Newsletter, 2020.
  199. G. Xu, T. D. Duong, Q. Li, S. Liu, and X. Wang, “Causality learning: A new perspective for interpretable machine learning,” arXiv preprint arXiv:2006.16789, 2020.
  200. J. Wexler, M. Pushkarna, T. Bolukbasi, M. Wattenberg, F. Viégas, and J. Wilson, “The what-if tool: Interactive probing of machine learning models,” IEEE transactions on visualization and computer graphics, vol. 26, no. 1, pp. 56–65, 2019.
  201. J. Pearl et al., “Causal inference in statistics: An overview,” Statistics surveys, 2009.
  202. R. Guidotti, A. Monreale, S. Matwin, and D. Pedreschi, “Black box explanation by learning image exemplars in the latent feature space,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2019.
  203. M. Nauta, A. Jutte, J. Provoost, and C. Seifert, “This looks like that, because… explaining prototypes for interpretable image recognition,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases.   Springer, 2021, pp. 441–456.
  204. S. Mohammadjafari, M. Cevik, M. Thanabalasingam, and A. Basar, “Using protopnet for interpretable alzheimer’s disease classification.” in Canadian Conference on AI, 2021.
  205. G. Carloni, A. Berti, C. Iacconi, M. A. Pascali, and S. Colantonio, “On the applicability of prototypical part learning in medical images: breast masses classification using protopnet,” in International Conference on Pattern Recognition.   Springer, 2022, pp. 539–557.
  206. S. Gautam, M. M.-C. Höhne, S. Hansen, R. Jenssen, and M. Kampffmeyer, “This looks more like that: Enhancing self-explaining models by prototypical relevance propagation,” Pattern Recognition, vol. 136, p. 109172, 2023.
  207. S. S. Kim, N. Meister, V. V. Ramaswamy, R. Fong, and O. Russakovsky, “Hive: Evaluating the human interpretability of visual explanations,” in European Conference on Computer Vision.   Springer, 2022, pp. 280–298.
  208. O. Davoodi, S. Mohammadizadehsamakosh, and M. Komeili, “On the interpretability of part-prototype based classifiers: A human centric analysis,” arXiv preprint arXiv:2310.06966, 2023.
  209. Q. Huang, M. Xue, W. Huang, H. Zhang, J. Song, Y. Jing, and M. Song, “Evaluation and improvement of interpretability for self-explainable part-prototype networks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 2011–2020.
  210. H. Behzadi-Khormouji and J. Oramas, “A protocol for evaluating model interpretation methods from visual explanations,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 1421–1429.
  211. R. Fong, M. Patrick, and A. Vedaldi, “Understanding deep networks via extremal perturbations and smooth masks,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019.
  212. M. Robnik-Šikonja and M. Bohanec, “Perturbation-based explanations of prediction models,” Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent, pp. 159–175, 2018.
  213. A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,” in Proceedings of the Winter Conference on Applications of Computer Vision, 2018.
  214. H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, and X. Hu, “Score-cam: Score-weighted visual explanations for convolutional neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
  215. P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, “Layercam: Exploring hierarchical class activation maps for localization,” IEEE Transactions on Image Processing, 2021.
  216. W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. Müller, “Evaluating the visualization of what a deep neural network has learned,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 11, pp. 2660–2673, 2016.
  217. A. Søgaard et al., “Revisiting methods for finding influential examples,” arXiv preprint arXiv:2111.04683, 2021.
  218. R. D. Cook and S. Weisberg, “Characterizations of an empirical influence function for detecting influential cases in regression,” Technometrics, vol. 22, no. 4, pp. 495–508, 1980.
  219. V. Feldman, “Does learning require memorization? a short tale about a long tail,” in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, 2020.
  220. Z. Kong and K. Chaudhuri, “Understanding instance-based interpretability of variational auto-encoders,” in Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34.   Curran Associates, Inc., 2021, pp. 2400–2412. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2021/file/13d7dc096493e1f77fb4ccf3eaf79df1-Paper.pdf
  221. J. Lin, A. Zhang, M. Lécuyer, J. Li, A. Panda, and S. Sen, “Measuring the effect of training data on deep learning predictions via randomized experiments,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato, Eds., vol. 162.   PMLR, 17–23 Jul 2022, pp. 13 468–13 504. [Online]. Available: https://proceedings.mlr.press/v162/lin22h.html
  222. J. Brophy, Z. Hammoudeh, and D. Lowd, “Adapting and evaluating influence-estimation methods for gradient-boosted decision trees,” Journal of Machine Learning Research, vol. 24, no. 154, pp. 1–48, 2023. [Online]. Available: http://jmlr.org/papers/v24/22-0449.html
  223. E. Black and M. Fredrikson, “Leave-one-out unfairness,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 285–295.
  224. J. Ye, A. Borovykh, S. Hayou, and R. Shokri, “Leave-one-out distinguishability in machine learning,” arXiv preprint arXiv:2309.17310, 2023.
  225. A. Alaa and M. Van Der Schaar, “Discriminative jackknife: Quantifying uncertainty in deep learning via higher-order influence functions,” in International Conference on Machine Learning.   PMLR, 2020, pp. 165–174.
  226. ——, “Frequentist uncertainty in recurrent neural networks via blockwise influence functions,” in International Conference on Machine Learning.   PMLR, 2020, pp. 175–190.
  227. P. Sattigeri, S. Ghosh, I. Padhi, P. Dognin, and K. R. Varshney, “Fair infinitesimal jackknife: Mitigating the influence of biased training data points without refitting,” Advances in Neural Information Processing Systems, vol. 35, pp. 35 894–35 906, 2022.
  228. H. Friedl and E. Stampfer, “Jackknife resampling,” Encyclopedia of environmetrics, vol. 2, pp. 1089–1098, 2002.
  229. C.-P. Tsai, C.-K. Yeh, and P. Ravikumar, “Sample based explanations via generalized representers,” arXiv preprint arXiv:2310.18526, 2023.
  230. T. Hastie, R. Tibshirani, J. Friedman, T. Hastie, R. Tibshirani, and J. Friedman, “Boosting and additive trees,” The elements of statistical learning: data mining, inference, and prediction, pp. 337–387, 2009.
  231. R. E. Schapire, “Explaining adaboost,” in Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik.   Springer, 2013, pp. 37–52.
  232. J. H. Friedman, “Greedy function approximation: a gradient boosting machine,” Annals of statistics, pp. 1189–1232, 2001.
  233. T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794.
  234. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” Advances in neural information processing systems, vol. 30, 2017.
  235. C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, pp. 273–297, 1995.
  236. O. Canévet and F. Fleuret, “Large scale hard sample mining with monte carlo tree search,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5128–5137.
  237. Q. Dong, S. Gong, and X. Zhu, “Class rectification hard mining for imbalanced deep learning,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 1851–1860.
  238. Y. Suh, B. Han, W. Kim, and K. M. Lee, “Stochastic class-based hard example mining for deep metric learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7251–7259.
  239. Y.-B. Tang, K. Yan, Y.-X. Tang, J. Liu, J. Xiao, and R. M. Summers, “Uldor: a universal lesion detector for ct scans with pseudo masks and hard negative example mining,” in 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019).   IEEE, 2019, pp. 833–836.
  240. H. Sheng, Y. Zheng, W. Ke, D. Yu, X. Cheng, W. Lyu, and Z. Xiong, “Mining hard samples globally and efficiently for person reidentification,” IEEE Internet of Things Journal, vol. 7, no. 10, pp. 9611–9622, 2020.
  241. C. Zhao, X. Lv, Z. Zhang, W. Zuo, J. Wu, and D. Miao, “Deep fusion feature representation learning with hard mining center-triplet loss for person re-identification,” IEEE Transactions on Multimedia, vol. 22, no. 12, pp. 3180–3195, 2020.
  242. M. Paul, S. Ganguli, and G. K. Dziugaite, “Deep learning on a data diet: Finding important examples early in training,” Advances in Neural Information Processing Systems, vol. 34, pp. 20 596–20 607, 2021.
  243. R. J. Baldock, H. Maennel, and B. Neyshabur, “Deep learning through the lens of example difficulty,” arXiv preprint arXiv:2106.09647, 2021.
  244. Y. Liu, X. Yang, S. Zhou, X. Liu, Z. Wang, K. Liang, W. Tu, L. Li, J. Duan, and C. Chen, “Hard sample aware network for contrastive deep graph clustering,” in Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 7, 2023, pp. 8914–8922.
  245. T. Formal, C. Lassance, B. Piwowarski, and S. Clinchant, “From distillation to hard negative sampling: Making sparse neural ir models more effective,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 2353–2359.
  246. S. Zhang, M. Liu, J. Yan, H. Zhang, L. Huang, X. Yang, and P. Lu, “M-mix: Generating hard negatives via multi-sample mixing for contrastive learning,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 2461–2470.
  247. C. E. Brodley and M. A. Friedl, “Identifying mislabeled training data,” Journal of artificial intelligence research, vol. 11, pp. 131–167, 1999.
  248. J.-w. Sun, F.-y. Zhao, C.-j. Wang, and S.-f. Chen, “Identifying and correcting mislabeled training instances,” in Future generation communication and networking (FGCN 2007), vol. 1.   IEEE, 2007, pp. 244–250.
  249. G. Pleiss, T. Zhang, E. Elenberg, and K. Q. Weinberger, “Identifying mislabeled data using the area under the margin ranking,” Advances in Neural Information Processing Systems, vol. 33, pp. 17 044–17 056, 2020.
  250. V. Pulastya, G. Nuti, Y. K. Atri, and T. Chakraborty, “Assessing the quality of the datasets by identifying mislabeled samples,” in Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2021, pp. 18–22.
  251. D. Ponnoprat, P. Pattarapanitchai, P. Taninpong, and S. Suantai, “An explainable machine learning approach to visual-interactive labeling: A case study on non-communicable disease data,” arXiv preprint arXiv:2209.12778, 2022.
  252. H. Xiong, X. Li, B. Yu, Z. Zhu, D. Wu, and D. Dou, “Doubly stochastic models: Learning with unbiased label noises and inference stability,” arXiv preprint arXiv:2304.00320, 2023.
  253. A. Jain, H. Patel, L. Nagalapatti, N. Gupta, S. Mehta, S. Guttula, S. Mujumdar, S. Afzal, R. Sharma Mittal, and V. Munigala, “Overview and importance of data quality for machine learning tasks,” in Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 3561–3562.
  254. N. Gupta, S. Mujumdar, H. Patel, S. Masuda, N. Panwar, S. Bandyopadhyay, S. Mehta, S. Guttula, S. Afzal, R. Sharma Mittal et al., “Data quality for machine learning tasks,” in Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2021, pp. 4040–4041.
  255. F. Bießmann, T. Rukat, P. Schmidt, P. Naidu, S. Schelter, A. Taptunov, D. Lange, and D. Salinas, “Datawig: Missing value imputation for tables,” Journal of Machine Learning Research, vol. 20, pp. 1–6, 2019.
  256. V. Yepmo, G. Smits, and O. Pivert, “Anomaly explanation: A review,” Data & Knowledge Engineering, vol. 137, p. 101946, 2022.
  257. F. Gralinski, A. Wróblewska, T. Stanisławek, K. Grabowski, and T. Górecki, “Geval: Tool for debugging nlp datasets and models,” in Proceedings of the 2019 ACL workshop BlackboxNLP: analyzing and interpreting neural networks for NLP, 2019, pp. 254–262.
  258. R. F. MARTINEZ, “A visual empirical region of influence pattern recognition tool for leave-one-out data analysis,” Sandia National Lab.(SNL-NM), Albuquerque, NM (United States); Sandia …, Tech. Rep., 2002.
  259. B. Blili-Hamelin and L. Hancox-Li, “Making intelligence: Ethical values in iq and ml benchmarks,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 271–284.
  260. M. Krenn, R. Pollice, S. Y. Guo, M. Aldeghi, A. Cervera-Lierta, P. Friederich, G. dos Passos Gomes, F. Häse, A. Jinich, A. Nigam et al., “On scientific understanding with artificial intelligence,” Nature Reviews Physics, vol. 4, no. 12, pp. 761–769, 2022.
  261. Z. Li, J. Ji, and Y. Zhang, “From kepler to newton: Explainable ai for science discovery,” arXiv preprint arXiv:2111.12210, 2021.
  262. C. Kamath, “On mining scientific datasets,” Data Mining for Scientific and Engineering Applications, pp. 1–21, 2001.
  263. A. Karpatne, G. Atluri, J. H. Faghmous, M. Steinbach, A. Banerjee, A. Ganguly, S. Shekhar, N. Samatova, and V. Kumar, “Theory-guided data science: A new paradigm for scientific discovery from data,” IEEE Transactions on knowledge and data engineering, vol. 29, no. 10, pp. 2318–2331, 2017.
  264. G. P. Wellawatte, H. A. Gandhi, A. Seshadri, and A. D. White, “A perspective on explanations of molecular prediction models,” Journal of Chemical Theory and Computation, vol. 19, no. 8, pp. 2149–2160, 2023.
  265. C. M. Weston, E. B. Bass, D. E. Ford, and J. B. Segal, “Faculty involvement in translational research and interdisciplinary collaboration at a us academic medical center,” Journal of Investigative Medicine, vol. 58, no. 6, pp. 770–776, 2010.
  266. E. Guerrero, E. Hahn, T. Khachikian, E. Chuang, A. Brown et al., “Interdisciplinary dissemination and implementation research to advance translational science: challenges and opportunities,” Journal of clinical and translational science, vol. 1, no. 1, pp. 67–72, 2017.
  267. N. Burkart and M. F. Huber, “A survey on the explainability of supervised machine learning,” Journal of Artificial Intelligence Research, vol. 70, pp. 245–317, 2021.
  268. G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborová, “Machine learning and the physical sciences,” Reviews of Modern Physics, vol. 91, no. 4, p. 045002, 2019.
  269. A. Lavin, D. Krakauer, H. Zenil, J. Gottschlich, T. Mattson, J. Brehmer, A. Anandkumar, S. Choudry, K. Rocki, A. G. Baydin et al., “Simulation intelligence: Towards a new generation of scientific methods,” arXiv preprint arXiv:2112.03235, 2021.
  270. Y.-N. Chuang, G. Wang, F. Yang, Z. Liu, X. Cai, M. Du, and X. Hu, “Efficient xai techniques: A taxonomic survey,” arXiv preprint arXiv:2302.03225, 2023.
  271. C. Agarwal, E. Saxena, S. Krishna, M. Pawelczyk, N. Johnson, I. Puri, M. Zitnik, and H. Lakkaraju, “Openxai: Towards a transparent evaluation of model explanations,” arXiv preprint arXiv:2206.11104, 2022.
  272. X. Li, M. Du, J. Chen, Y. Chai, H. Lakkaraju, and H. Xiong, “M4: A unified xai benchmark for faithfulness evaluation of feature attribution methods across metrics, modalities and models,” in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  273. H. Zhao, H. Chen, F. Yang, N. Liu, H. Deng, H. Cai, S. Wang, D. Yin, and M. Du, “Explainability for large language models: A survey,” ACM Transactions on Intelligent Systems and Technology (TIST), 2023.
  274. M. Haddouchi and A. Berrado, “A survey of methods and tools used for interpreting random forest,” in 2019 1st International Conference on Smart Systems and Data Science (ICSSD).   IEEE, 2019, pp. 1–6.
  275. E. Smirnov, A. Melnikov, S. Novoselov, E. Luckyanets, and G. Lavrentyeva, “Doppelganger mining for face representation learning,” in Proceedings of the IEEE international conference on computer vision workshops, 2017, pp. 1916–1923.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Haoyi Xiong (98 papers)
  2. Xiaofei Zhang (36 papers)
  3. Jiamin Chen (14 papers)
  4. Xinhao Sun (1 paper)
  5. Yuchen Li (85 papers)
  6. Zeyi Sun (16 papers)
  7. Mengnan Du (90 papers)
  8. Xuhong Li (40 papers)
Citations (5)
View on Semantic Scholar

Summary

Introduction

The field of AI is shaped by Deep Neural Networks (DNNs), which, due to their opaque nature, pose interpretability challenges. As AI gains prevalence in high-stakes scenarios, being able to trust and understand these systems becomes imperative. This is where Explainable AI (XAI) steps in, offering tools and techniques to make AI decisions more transparent. But beyond the algorithms themselves, this exploration emphasizes the 'data-centric' aspects of XAI – considering the role of data in explainability, from its collection to the way insights are gleaned from it – and categorizes existing work into three key purposes.

Interpretations of Deep Models

Understanding the rationale behind a model's predictions is vital for trust and transparency. Interpretations of deep models dive into feature attributions, highlighting specific data points responsible for predictions. Techniques here may include perturbation-based methods like LIME, which adjust input data and observe the effect on predictions, or differentiation-based methods like Integrated Gradients, which compute gradients as a measure of input feature importance. Activation and attention-based methods further enrich interpretability for models, making sense of how input data is weighted in decision-making.

Influences of Training Data

The quality of machine learning outcomes is a function of training data quality. XAI acknowledges this by pinpointing influential data points, aiding in the refinement of training processes for enhanced simplicity and performance. Methods for evaluating sample contributions include gradient-based approaches for assessing data point influence, and resampling techniques like Leave-One-Out, which measure the impact of removing certain data points from the training set. Game theoretical methods like Shapley values, traditionally applied in cooperative games, are also adapted to quantify the valuation of individual training samples.

Insights of Domain Knowledge

XAI's third purpose is to extract and elucidate domain-specific patterns from data, translating these into actionable insights. This facet of XAI not only serves to interpret AI processes but also aims to advance human understanding within various domains, such as healthcare and finance. XAI enables the discovery of latent patterns which can be crucial for scientific exploration and social value promotion, ensuring AI's decisions align with societal norms and contribute to the broader knowledge base.

Bridging the Interpretability Gap

In summary, the role of XAI extends beyond explaining model decisions to understanding the influence of data and drawing out domain insights that can inform social values and expediate scientific discovery. Through a structured, data-centric approach, this work categorizes and analyzes XAI methodologies, ultimately aiming to better integrate human-centric analyses with AI, for improved human-AI synergy. This comprehensive view of XAI within the context of data mining aids in the ongoing development of AI systems that are not only powerful but also accountable, trustworthy, and transparent.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown