Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Solving the enigma: Deriving optimal explanations of deep networks (2405.10008v1)

Published 16 May 2024 in cs.CV

Abstract: The accelerated progress of AI has popularized deep learning models across domains, yet their inherent opacity poses challenges, notably in critical fields like healthcare, medicine and the geosciences. Explainable AI (XAI) has emerged to shed light on these "black box" models, helping decipher their decision making process. Nevertheless, different XAI methods yield highly different explanations. This inter-method variability increases uncertainty and lowers trust in deep networks' predictions. In this study, for the first time, we propose a novel framework designed to enhance the explainability of deep networks, by maximizing both the accuracy and the comprehensibility of the explanations. Our framework integrates various explanations from established XAI methods and employs a non-linear "explanation optimizer" to construct a unique and optimal explanation. Through experiments on multi-class and binary classification tasks in 2D object and 3D neuroscience imaging, we validate the efficacy of our approach. Our explanation optimizer achieved superior faithfulness scores, averaging 155% and 63% higher than the best performing XAI method in the 3D and 2D applications, respectively. Additionally, our approach yielded lower complexity, increasing comprehensibility. Our results suggest that optimal explanations based on specific criteria are derivable and address the issue of inter-method variability in the current XAI literature.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. \bibcommenthead
  2. Deep learning for multi-year enso forecasts. Nature 573, 568–572 (2019). URL https://doi.org/10.1038/s41586-019-1559-7.
  3. Reichstein, M. et al. Deep learning and process understanding for data-driven earth system science. Nature 566, 195–204 (2019). URL https://doi.org/10.1038/s41586-019-0912-1.
  4. Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE 109, 247–278 (2021).
  5. Mamalakis, M. et al. Artificial intelligence framework with traditional computer vision and deep learning approaches for optimal automatic segmentation of left ventricle with scar. Artificial Intelligence in Medicine 143, 102610 (2023). URL https://www.sciencedirect.com/science/article/pii/S0933365723001240.
  6. Mamalakis, M. et al. A 3d explainability framework to uncover learning patterns and crucial sub-regions in variable sulci recognition (2023). 2309.00903.
  7. Viñas, R. et al. Hypergraph factorization for multi-tissue gene expression imputation. Nature Machine Intelligence 5, 739–753 (2023). URL https://doi.org/10.1038/s42256-023-00684-8.
  8. A novel framework employing deep multi-attention channels network for the autonomous detection of metastasizing cells through fluorescence microscopy (2023). 2309.00911.
  9. van der Velden, B. H. M. Explainable ai: current status and future potential. European Radiology (2023). URL https://doi.org/10.1007/s00330-023-10121-4.
  10. Longo, L. et al. Explainable artificial intelligence (xai) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion 106, 102301 (2024). URL https://www.sciencedirect.com/science/article/pii/S1566253524000794.
  11. Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE 10, 1–46 (2015).
  12. ”why should i trust you?”: Explaining the predictions of any classifier (2016). 1602.04938.
  13. Explain yourself! leveraging language models for commonsense reasoning (2019). 1906.02361.
  14. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Transactions on Neural Networks and Learning Systems 1–21 (2020).
  15. Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geoscience. Artificial Intelligence for the Earth Systems 1, e220012 (2022). URL https://journals.ametsoc.org/view/journals/aies/1/4/AIES-D-22-0012.1.xml.
  16. Explainable deep learning models in medical image analysis (2020). 2005.13799.
  17. Explainable Artificial Intelligence in Meteorology and Climate Science: Model Fine-Tuning, Calibrating Trust and Learning New Science, 315–339 (Springer International Publishing, Cham, 2022). URL https://doi.org/10.1007/978-3-031-04083-2_16.
  18. Neural network attribution methods for problems in geoscience: A novel synthetic benchmark dataset. Environmental Data Science 1, e8 (2022).
  19. Bach, S. et al. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE 10, 1–46 (2015). URL https://doi.org/10.1371/journal.pone.0130140.
  20. Kohlbrenner, M. et al. Towards best practice in explaining neural network decisions with lrp (2020). 1910.09840.
  21. A unified approach to interpreting model predictions. CoRR abs/1705.07874 (2017). URL http://arxiv.org/abs/1705.07874.
  22. Sampling permutations for shapley value estimation (2022). 2104.12199.
  23. Fastshap: Real-time shapley value estimation (2022). 2107.07436.
  24. Approximating the shapley value without marginal contributions (2024). 2302.00736.
  25. Hedström, A. et al. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond. Journal of Machine Learning Research 24, 1–11 (2023). URL http://jmlr.org/papers/v24/22-0142.html.
  26. Mørch-Johnsen, L. et al. Auditory cortex characteristics in schizophrenia: Associations with auditory hallucinations 43, 75–83. URL https://academic.oup.com/schizophreniabulletin/article/43/1/75/2503785.
  27. Fornito, A. et al. Morphology of the paracingulate sulcus and executive cognition in schizophrenia 88, 192–197. URL https://linkinghub.elsevier.com/retrieve/pii/S0920996406003021.
  28. Garrison, J. R. et al. Paracingulate sulcus morphology is associated with hallucinations in the human brain 6, 8956. URL http://www.nature.com/articles/ncomms9956.
  29. Krizhevsky, A. Learning multiple layers of features from tiny images (2009). URL https://api.semanticscholar.org/CorpusID:18268744.
  30. Simon, H. A. A Behavioral Model of Rational Choice. The Quarterly Journal of Economics 69, 99–118 (1955). URL https://doi.org/10.2307/1884852.
  31. Simon, H. A. Invariants of human behavior. Annual Review of Psychology 41, 1–20 (1990). URL https://www.annualreviews.org/content/journals/10.1146/annurev.ps.41.020190.000245.
  32. Simon, H. A. Rational choice and the structure of the environment. Psychological Review 63, 129–138 (1956).
  33. Mamalakis, M. et al. An explainable three dimension framework to uncover learning patterns: A unified look in variable sulci recognition (2024). 2309.00903.
  34. Mitchell, S. C. et al. Paracingulate sulcus measurement protocol v2 (2023). URL https://www.repository.cam.ac.uk/handle/1810/358381.
  35. On the (in)fidelity and sensitivity for explanations (2019). URL https://arxiv.org/abs/1901.09392.
  36. Evaluating and aggregating feature-based model explanations (2020). URL https://arxiv.org/abs/2005.00631.
  37. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 600–612 (2004).
  38. Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). URL http://arxiv.org/abs/1512.03385.
  39. Hatamizadeh, A. et al. Crimi, A. & Bakas, S. (eds) Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. (eds Crimi, A. & Bakas, S.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, 272–284 (Springer International Publishing, Cham, 2022).
  40. U-net: Convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015). URL http://arxiv.org/abs/1505.04597.
  41. Dosovitskiy, A. et al. An image is worth 16x16 words: Transformers for image recognition at scale. CoRR abs/2010.11929 (2020). URL https://arxiv.org/abs/2010.11929.
  42. Axiomatic attribution for deep networks (2017). 1703.01365.
  43. Striving for simplicity: The all convolutional net (2015). 1412.6806.
  44. Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision 128, 336–359 (2019). URL http://dx.doi.org/10.1007/s11263-019-01228-7.
  45. Adam: A method for stochastic optimization (2014). URL https://arxiv.org/abs/1412.6980.
  46. The “independent components” of natural scenes are edge filters. Vision Research 37, 3327–3338 (1997). URL https://www.sciencedirect.com/science/article/pii/S0042698997001211.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Michail Mamalakis (10 papers)
  2. Antonios Mamalakis (7 papers)
  3. Ingrid Agartz (4 papers)
  4. Graham Murray (12 papers)
  5. John Suckling (9 papers)
  6. Lynn Egeland Mørch-Johnsen (2 papers)
  7. Pietro Lio (69 papers)
Citations (1)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube