Faithful Attention Explainer: Verbalizing Decisions Based on Discriminative Features (2405.13032v2)
Abstract: In recent years, model explanation methods have been designed to interpret model decisions faithfully and intuitively so that users can easily understand them. In this paper, we propose a framework, Faithful Attention Explainer (FAE), capable of generating faithful textual explanations regarding the attended-to features. Towards this goal, we deploy an attention module that takes the visual feature maps from the classifier for sentence generation. Furthermore, our method successfully learns the association between features and words, which allows a novel attention enforcement module for attention explanation. Our model achieves promising performance in caption quality metrics and a faithful decision-relevance metric on two datasets (CUB and ACT-X). In addition, we show that FAE can interpret gaze-based human attention, as human gaze indicates the discriminative features that humans use for decision-making, demonstrating the potential of deploying human gaze for advanced human-AI interaction.
- Interpreting chest x-rays via cnns that exploit hierarchical disease dependencies and uncertainty labels, Neurocomputing (2021).
- E. Tjoa, C. Guan, A survey on explainable artificial intelligence (XAI): towards medical XAI, CoRR (2019). URL: http://arxiv.org/abs/1907.07374. arXiv:1907.07374.
- User trust on an explainable ai-based medical diagnosis support system, arXiv preprint arXiv:2204.12230 (2022).
- Towards human-centered explainable ai: A survey of user studies for model explanations, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
- Flex: Faithful linguistic explanations for neural net based model decisions, in: AAAI, 2019.
- Generating visual explanations, CoRR (2016). URL: http://arxiv.org/abs/1603.08507. arXiv:1603.08507.
- A consistent and efficient evaluation strategy for attribution methods, in: International Conference on Machine Learning, PMLR, 2022, pp. 18770–18795.
- Multimodal explanations: Justifying decisions and pointing to the evidence, CoRR (2018). URL: http://arxiv.org/abs/1802.08129. arXiv:1802.08129.
- Rise: Randomized input sampling for explanation of black-box models, BMVC (2018).
- Learning deep features for discriminative localization, in: CVPR, 2016.
- Grad-cam: Visual explanations from deep networks via gradient-based localization, in: ICCV, 2017.
- Axiomatic attribution for deep networks, in: ICML, 2017.
- Learning important features through propagating activation differences, in: ICML, 2017.
- Textual explanations for self-driving vehicles, CoRR (2018). URL: http://arxiv.org/abs/1807.11546. arXiv:1807.11546.
- Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization, CoRR (2016).
- M. I. Posner, S. E. Petersen, The attention system of the human brain, Annual review of neuroscience (1990).
- Human attention in visual question answering: Do humans and deep networks look at the same regions?, Computer Vision and Image Understanding (2017).
- Human attention in fine-grained classification, arXiv preprint arXiv:2111.01628 (2021).
- Prophet attention: Predicting attention with future attention, in: NeurIPS, 2020. URL: https://proceedings.neurips.cc/paper/2020/file/13fe9d84310e77f13a6d184dbf1232f3-Paper.pdf.
- Show, attend and tell: Neural image caption generation with visual attention, CoRR (2015). URL: http://arxiv.org/abs/1502.03044. arXiv:1502.03044.
- Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning, in: CVPR, 2017.
- Knowing when to look: Adaptive attention via a visual sentinel for image captioning, in: CVPR, 2017.
- Image captioning with semantic attention, in: CVPR, 2016.
- Attention correctness in neural image captioning, in: AAAI, 2017.
- Grounded video description, in: CVPR, 2019.
- S. Barratt, Interpnet: Neural introspection for interpretable deep learning, ArXiv (2017).
- M. Schuster, K. K. Paliwal, Bidirectional recurrent neural networks, IEEE transactions on Signal Processing (1997).
- Learning deep representations of fine-grained visual descriptions, CoRR (2016). URL: http://arxiv.org/abs/1605.05395. arXiv:1605.05395.
- 2d human pose estimation: New benchmark and state of the art analysis, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.