Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SPADE: Sparsity-Guided Debugging for Deep Neural Networks (2310.04519v2)

Published 6 Oct 2023 in cs.LG

Abstract: It is known that sparsity can improve interpretability for deep neural networks. However, existing methods in the area either require networks that are pre-trained with sparsity constraints, or impose sparsity after the fact, altering the network's general behavior. In this paper, we demonstrate, for the first time, that sparsity can instead be incorporated into the interpretation process itself, as a sample-specific preprocessing step. Unlike previous work, this approach, which we call SPADE, does not place constraints on the trained model and does not affect its behavior during inference on the sample. Given a trained model and a target sample, SPADE uses sample-targeted pruning to provide a "trace" of the network's execution on the sample, reducing the network to the most important connections prior to computing an interpretation. We demonstrate that preprocessing with SPADE significantly increases the accuracy of image saliency maps across several interpretability methods. Additionally, SPADE improves the usefulness of neuron visualizations, aiding humans in reasoning about network behavior. Our code is available at https://github.com/IST-DASLab/SPADE.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Gradient-based attribution methods. Explainable AI: Interpreting, explaining and visualizing deep learning, pp.  169–191, 2019.
  2. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 2015.
  3. Food-101 - mining discriminative components with random forests. In European Conference on Computer Vision, 2014.
  4. Gender shades: Intersectional accuracy disparities in commercial gender classification. In FAT, 2018.
  5. Low-complexity probing via finding subnetworks. In North American Chapter of the Association for Computational Linguistics (NAACL-HLT), 2021.
  6. Benchmarking interpretability tools for deep neural networks. arXiv preprint arXiv:2302.10894, 2023.
  7. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In IEEE Winter Conference on Applications of Computer Vision, WACV, 2018.
  8. Transformer interpretability beyond attention visualization. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
  9. Imagenet: A large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
  10. Towards a rigorous science of interpretable machine learning. arXiv, 2017.
  11. Visualizing higher-layer features of a deep network. 2009.
  12. Optimal brain compression: A framework for accurate post-training quantization and pruning. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
  13. Massive language models can be accurately pruned in one-shot. arXiv preprint arXiv:2301.00774, 2023.
  14. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2, 2020.
  15. Don’t trust your eyes: on the (un)reliability of feature visualizations, 2023.
  16. What do vision transformers learn? A visual exploration. arXiv preprint arXiv:2212.06727, 2022.
  17. Multimodal neurons in artificial neural networks. Distill, 6(3):e30, 2021.
  18. Metrics for saliency map evaluation of deep learning explanation methods. In Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI. Springer, 2022.
  19. Finding neurons in a haystack: Case studies with sparse probing. ArXiv, abs/2305.01610, 2023.
  20. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  21. A benchmark for interpretability methods in deep neural networks, 2019.
  22. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.
  23. Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks. Advances in neural information processing systems, 2021.
  24. Measuring the tendency of cnns to learn surface statistical regularities. ArXiv, abs/1711.11561, 2017.
  25. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896, 2020.
  26. Do better imagenet models transfer better? In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  27. Cap: Correlation-aware pruning for highly-accurate sparse vision models, 2023.
  28. Explainable ai: A review of machine learning interpretability methods. Entropy, 2020.
  29. A convnet for the 2020s. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  30. Seeing is believing: Brain-inspired modular training for mechanistic interpretability. arXiv preprint arXiv:2305.08746, 2023.
  31. Deep learning face attributes in the wild. In International Conference on Computer Vision (ICCV), 2015.
  32. A unified approach to interpreting model predictions. In Conference on Neural Information Processing Systems (NeurIPS), 2017.
  33. Locating and editing factual associations in GPT. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
  34. Deepdream-a code example for visualizing neural networks. Google Research.
  35. Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks. arXiv preprint arXiv:1602.03616, 2016.
  36. Robust explainability: A tutorial on gradient-based attribution methods for deep neural networks. IEEE Signal Processing Magazine, 2022.
  37. Feature visualization. Distill, 2(11), 2017.
  38. AC/DC: Alternating compressed/decompressed training of deep neural networks. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
  39. There and back again: Revisiting backpropagation saliency methods. In Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  40. ”why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
  41. Grad-cam: Visual explanations from deep networks via gradient-based localization. In International Conference on Computer Vision (ICCV), 2017.
  42. Not using the car to see the sidewalk – quantifying and controlling the effects of context in classification and segmentation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  43. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713, 2016.
  44. Learning important features through propagating activation differences. In International Conference on Machine Learning (ICML), 2017.
  45. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
  46. Deep inside convolutional networks: Visualising image classification models and saliency maps. In International Conference on Machine Learning (ICML), 2014.
  47. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.
  48. Axiomatic attribution for deep networks. In International Conference on Machine Learning (ICML), 2017.
  49. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, 2019.
  50. Score-cam: Score-weighted visual explanations for convolutional neural networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019.
  51. Leveraging sparse linear layers for debuggable deep networks. In International Conference on Machine Learning (ICML), 2021.
  52. Understanding neural networks through deep visualization. ArXiv, abs/1506.06579, 2015.
  53. Lu Yu and Wei Xiang. X-pruner: explainable pruning for vision transformers. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  54. Visualizing and understanding convolutional networks. In European Conference on Computer Vision (ICCV), 2014.
  55. Opti-cam: Optimizing saliency maps for interpretability. ArXiv, abs/2301.07002, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets