Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A comprehensive study on fidelity metrics for XAI (2401.10640v1)

Published 19 Jan 2024 in cs.CV and cs.AI

Abstract: The use of eXplainable Artificial Intelligence (XAI) systems has introduced a set of challenges that need resolution. Herein, we focus on how to correctly select an XAI method, an open questions within the field. The inherent difficulty of this task is due to the lack of a ground truth. Several authors have proposed metrics to approximate the fidelity of different XAI methods. These metrics lack verification and have concerning disagreements. In this study, we proposed a novel methodology to verify fidelity metrics, using a well-known transparent model, namely a decision tree. This model allowed us to obtain explanations with perfect fidelity. Our proposal constitutes the first objective benchmark for these metrics, facilitating a comparison of existing proposals, and surpassing existing methods. We applied our benchmark to assess the existing fidelity metrics in two different experiments, each using public datasets comprising 52,000 images. The images from these datasets had a size a 128 by 128 pixels and were synthetic data that simplified the training process. All metric values, indicated a lack of fidelity, with the best one showing a 30 \% deviation from the expected values for perfect explanation. Our experimentation led us to conclude that the current fidelity metrics are not reliable enough to be used in real scenarios. From this finding, we deemed it necessary to development new metrics, to avoid the detected problems, and we recommend the usage of our proposal as a benchmark within the scientific community to address these limitations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE access, 6:52138–52160.
  2. Fair and explainable depression detection in social media. Information Processing & Management, 60(1):103168.
  3. Sanity checks for saliency maps. Advances in neural information processing systems, 31.
  4. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31.
  5. Towards better understanding of gradient-based attribution methods for deep neural networks. In 6th International Conference on Learning Representations (ICLR), number 1711.06104, pages 0–0. Arxiv-Computer Science.
  6. Explainable agents and robots: Results from a systematic literature review. In 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, May 13–17, 2019, pages 1078–1088. International Foundation for Autonomous Agents and Multiagent Systems.
  7. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140.
  8. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58:82–115.
  9. Evaluating and aggregating feature-based model explanations. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3016–3022.
  10. Breiman, L. (1984). Classification and regression trees. Routledge.
  11. A survey on xai and natural language explanations. Information Processing & Management, 60(1):103111.
  12. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
  13. Explainable artificial intelligence: A survey. In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), pages 0210–0215. IEEE.
  14. Testing the robustness of attribution methods for convolutional neural networks in mri-based alzheimer’s disease classification. In Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support: Second International Workshop, iMIMIC 2019, and 9th International Workshop, ML-CDS 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Proceedings 9, pages 3–11. Springer.
  15. Statistics (international student edition). Pisani, R. Purves, 4th edn. WW Norton & Company, New York.
  16. Metrics for saliency map evaluation of deep learning explanation methods. In International Conference on Pattern Recognition and Artificial Intelligence, pages 84–95. Springer.
  17. Guidotti, R. (2021). Evaluating local explanation methods on ground truth. Artificial Intelligence, 291:103428.
  18. The meta-evaluation problem in explainable ai: Identifying reliable estimators with metaquantus. arXiv preprint arXiv:2302.07265.
  19. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond. Journal of Machine Learning Research, 24(34):1–11.
  20. The disagreement problem in explainable machine learning: A practitioner’s perspective. arXiv preprint arXiv:2202.01602.
  21. Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38.
  22. Explainable artificial intelligence: a comprehensive review. Artificial Intelligence Review, pages 1–66.
  23. A novel approach to generate datasets with xai ground truth to evaluate image models. arXiv preprint arXiv:2302.05624.
  24. Evaluating explainable artificial intelligence for x-ray image analysis. Applied Sciences, 12(9):4459.
  25. Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasets. arXiv:2311.01961 [cs].
  26. A multidisciplinary survey and framework for design and evaluation of explainable ai systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 11(3-4):1–45.
  27. Methods for interpreting and understanding deep neural networks. Digital signal processing, 73:1–15.
  28. Interpretable machine learning: definitions, methods, and applications. arXiv preprint arXiv:1901.04592.
  29. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.
  30. Generating perturbation-based explanations with robustness to out-of-distribution data. In Proceedings of the ACM Web Conference 2022, pages 3594–3605.
  31. Irof: a low resource evaluation metric for explanation methods. In Workshop AI for Affordable Healthcare at ICLR 2020.
  32. Evaluating the Visualization of What a Deep Neural Network Has Learned. IEEE Transactions on Neural Networks and Learning Systems, 28(11):2660–2673.
  33. Sanity checks for saliency metrics. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 6021–6029.
  34. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Medical Image Analysis, 79:102470.
  35. Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images. Scientific reports, 10(1):19549.
  36. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2097–2106.
  37. On the (in) fidelity and sensitivity of explanations. Advances in Neural Information Processing Systems, 32.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets