Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey (1911.12116v1)

Abstract: Deep Learning is a state-of-the-art technique to make inference on extensive or complex data. As a black box model due to their multilayer nonlinear structure, Deep Neural Networks are often criticized to be non-transparent and their predictions not traceable by humans. Furthermore, the models learn from artificial datasets, often with bias or contaminated discriminating content. Through their increased distribution, decision-making algorithms can contribute promoting prejudge and unfairness which is not easy to notice due to lack of transparency. Hence, scientists developed several so-called explanators or explainers which try to point out the connection between input and output to represent in a simplified way the inner structure of machine learning black boxes. In this survey we differ the mechanisms and properties of explaining systems for Deep Neural Networks for Computer Vision tasks. We give a comprehensive overview about taxonomy of related studies and compare several survey papers that deal with explainability in general. We work out the drawbacks and gaps and summarize further research ideas.

• The paper provides a comparative evaluation of explainers like deconvnet, Grad-CAM, and LIME to improve the interpretability of DNN models.
• It categorizes methods based on specificity, locality, and timing (ante-hoc vs post-hoc), offering structured approaches for practical application.
• The study underscores challenges in quantifying explanation quality and calls for robust metrics to ensure fidelity and interpretability.

Survey of Explainability in Deep Neural Networks for Computer Vision

The paper "Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey" by Vanessa Buhrmester, David Münch, and Michael Arens provides a detailed examination of the current methods developed to shed light on the decision-making processes of deep neural networks (DNNs), especially within the context of computer vision. The authors identify the core challenges associated with DNNs, namely their opacity due to multilayer nonlinear structures and the attendant risks of biased decision-making resulting from biased training data. This challenge necessitates the implementation of explainability methods to enhance transparency and trust in AI systems used for critical applications.

The introductory sections underscore the need for explanation systems or "explainers" that bridge the gap between complex input data and the final predictions made by DNNs. Given the intricate architectures of models like VGG-19 and ResNet, which contain millions of parameters, the demand for methods that offer insight into these mechanisms is clear. The paper further contextualizes this need within ethical considerations and recent regulatory developments, such as the European Union's General Data Protection Regulation (GDPR), which emphasizes the right to an explanation of decisions made by AI systems affecting individuals.

The authors proceed to introduce an array of contemporary methods devised to provide explanations of DNN behavior, focusing on their application in computer vision tasks. These include:

• Deconvolutional Networks (Deconvnet): This method involves the reverse engineering of a CNN to visualize activations that lead to a specific decision, enhancing understanding of the network's focal points during the classification process.
• Grad-CAM (Gradient-weighted Class Activation Mapping): Utilizes gradient information to elucidate the importance of each neuron in the last convolutional layer, thus providing class-specific visual explanations of model predictions.
• LIME (Local Interpretable Model-agnostic Explanations): A popular method that facilitates understanding of model predictions by approximating the model around a specific prediction using an interpretable model that highlights relevant features.

The paper methodically categorizes these explainers based on criteria such as locality versus globality of explanation, model-specific versus model-agnostic approaches, and ante-hoc versus post-hoc methodologies. There is an emphasis on tools that generate visual explanations due to their intuitive appeal in computer vision contexts, though the limitations of these tools, such as their inherent black-box nature in the case of model-agnostic explainers like LIME, are acknowledged.

To evaluate the efficacy of these explaining systems, the authors explore metrics for assessing explanations, which include fidelity to the original model, interpretability, and completeness. They also discuss emerging challenges, such as quantifying explanation quality and addressing the computational complexity associated with certain explainer methods.

In their concluding remarks, the authors express the necessity for more robust definitions and standards for assessing the qualities of an explanation, which remains a significant research frontier. The survey lays a foundation for future work aiming to standardize explainability across DNN applications in computer vision.

The implications of this research are crucial for practical and theoretical domains. Practically, improving the interpretability of DNNs helps in building safer AI systems that align with ethical norms and legal standards. Theoretically, the explanations gleaned from these methods can lead to the development of more inherently interpretable models, reducing reliance on post-hoc approaches. With explainability as a burgeoning aspect of AI research, the anticipation remains high for developments that enhance the symbiosis between machine learning efficacy and human comprehensibility.