Path Choice Matters for Clear Attribution in Path Methods
Abstract: Rigorousness and clarity are both essential for interpretations of DNNs to engender human trust. Path methods are commonly employed to generate rigorous attributions that satisfy three axioms. However, the meaning of attributions remains ambiguous due to distinct path choices. To address the ambiguity, we introduce \textbf{Concentration Principle}, which centrally allocates high attributions to indispensable features, thereby endowing aesthetic and sparsity. We then present \textbf{SAMP}, a model-agnostic interpreter, which efficiently searches the near-optimal path from a pre-defined set of manipulation paths. Moreover, we propose the infinitesimal constraint (IC) and momentum strategy (MS) to improve the rigorousness and optimality. Visualizations show that SAMP can precisely reveal DNNs by pinpointing salient image pixels. We also perform quantitative experiments and observe that our method significantly outperforms the counterparts. Code: https://github.com/zbr17/SAMP.
- Neural additive models: Interpretable machine learning with neural nets. In NeurIPS, pp. 4699–4711, 2021.
- Towards better understanding of gradient-based attribution methods for deep neural networks. In ICLR, 2018.
- Values of non-atomic games. Princeton University Press, 1974.
- On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
- Entropy-based logic explanations of neural networks. In AAAI, pp. 6046–6054, 2022.
- Benchmarking and survey of explanation methods for black box models. arXiv, abs/2102.13076, 2021.
- Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In WACV, pp. 839–847, 2018.
- Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248–255, 2009.
- Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141–142, 2012.
- Interpretable explanations of black boxes by meaningful perturbation. In ICCV, 2017.
- Eric J Friedman. Paths and consistency in additive cost sharing. International Journal of Game Theory, 32(4):501–518, 2004.
- Explaining explanations: An overview of interpretability of machine learning. In DSAA, pp. 80–89, 2018.
- Deep residual learning for image recognition. In CVPR, pp. 770–778, 2016.
- Xrai: Better attributions through regions. In ICCV, pp. 4948–4957, 2019.
- Guided integrated gradients: An adaptive path method for removing noise. In CVPR, pp. 5050–5058, 2021.
- Learning multiple layers of features from tiny images. 2009.
- Training interpretable convolutional neural networks by differentiating class-specific filters. In ECCV, pp. 622–638, 2020.
- Decoupled weight decay regularization. arXiv, abs/1711.05101, 2017.
- Dance: Enhancing saliency maps using decoys. In ICML, pp. 7124–7133, 2021.
- A unified approach to interpreting model predictions. In NeurIPS, volume 30, 2017.
- Methods for interpreting and understanding deep neural networks. Digital signal processing, 73:1–15, 2018.
- From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. arXiv, abs/2201.08164, 2022.
- Making sense of dependence: Efficient black-box explanations using dependence measure. Advances in Neural Information Processing Systems, 35:4344–4357, 2022.
- Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 32, 2019.
- Rise: Randomized input sampling for explanation of black-box models. In BMVC, 2018.
- ” why should i trust you?” explaining the predictions of any classifier. In KDD, pp. 1135–1144, 2016.
- Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, pp. 618–626, 2017.
- Interpretable compositional convolutional neural networks. arXiv, abs/2107.04474, 2021.
- Not just a black box: Learning important features through propagating activation differences. arXiv, abs/1605.01713, 2016.
- Learning important features through propagating activation differences. In ICML, pp. 3145–3153, 2017.
- Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR (Workshop), 2014.
- Smoothgrad: removing noise by adding noise. arXiv, abs/1706.03825, 2017.
- Striving for simplicity: The all convolutional net. In ICLR (Workshop), 2015.
- Axiomatic attribution for deep networks. In ICML, pp. 3319–3328, 2017.
- The caltech-ucsd birds-200-2011 dataset. 2011.
- Nbdt: neural-backed decision trees. arXiv, abs/2004.00221, 2020.
- Self-interpretable model with transformation equivariant interpretation. In NeurIPS, pp. 2359–2372, 2021.
- Attribution in scale and space. In CVPR, pp. 9680–9689, 2020.
- Visualizing and understanding convolutional networks. In ECCV, pp. 818–833, 2014.
- Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126(10):1084–1102, 2018a.
- Visual interpretability for deep learning: a survey. Frontiers of Information Technology & Electronic Engineering, 19(1):27–39, 2018.
- Interpretable convolutional neural networks. In CVPR, pp. 8827–8836, 2018b.
- Learning deep features for discriminative localization. In CVPR, pp. 2921–2929, 2016.
- Visualizing deep neural network decisions: Prediction difference analysis. arXiv, abs/1702.04595, 2017.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.