Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pixel-Grounded Prototypical Part Networks (2309.14531v1)

Published 25 Sep 2023 in cs.CV

Abstract: Prototypical part neural networks (ProtoPartNNs), namely PROTOPNET and its derivatives, are an intrinsically interpretable approach to machine learning. Their prototype learning scheme enables intuitive explanations of the form, this (prototype) looks like that (testing image patch). But, does this actually look like that? In this work, we delve into why object part localization and associated heat maps in past work are misleading. Rather than localizing to object parts, existing ProtoPartNNs localize to the entire image, contrary to generated explanatory visualizations. We argue that detraction from these underlying issues is due to the alluring nature of visualizations and an over-reliance on intuition. To alleviate these issues, we devise new receptive field-based architectural constraints for meaningful localization and a principled pixel space mapping for ProtoPartNNs. To improve interpretability, we propose additional architectural improvements, including a simplified classification head. We also make additional corrections to PROTOPNET and its derivatives, such as the use of a validation set, rather than a test set, to evaluate generalization during training. Our approach, PIXPNET (Pixel-grounded Prototypical part Network), is the only ProtoPartNN that truly learns and localizes to prototypical object parts. We demonstrate that PIXPNET achieves quantifiably improved interpretability without sacrificing accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (89)
  1. Computing receptive fields of convolutional neural networks. Distill, 2019. https://distill.pub/2019/computing-receptive-fields.
  2. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
  3. Hubert Baniecki. Adversarial explainable AI. https://hbaniecki.com/adversarial-explainable-ai/, 2023. Accessed: 2023-01-28.
  4. This looks like that there: Interpretable neural networks for image tasks when location matters. Artificial Intelligence for the Earth Systems, 1(3):e220001, 2022.
  5. Mapping the ictal-interictal-injury continuum using interpretable machine learning. arXiv, 2022.
  6. Concept-level debugging of part-prototype networks. In International Conference on Learning Representations, ICLR. OpenReview, 2023.
  7. Post-hoc explanations fail to achieve their purpose in adversarial contexts. ACM Conference on Fairness, Accountability, and Transparency, 5, 2022.
  8. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Conference on Fairness, Accountability and Transparency, pages 77–91. PMLR, Jan. 2018.
  9. Can i trust the explainer? verifying post-hoc explanatory methods. In NeurIPS 2019 Workshop on Safety and Robustness in Decision Making. arXiv, 2019.
  10. A framework for evaluating post hoc feature-additive explainers. arXiv, abs/2106.08376, 2021.
  11. Unfooling perturbation-based post hoc explainers. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI, 2023.
  12. Explainable ai helps bridge the ai skills gap: Evidence from a large bank. Economics Faculty Articles and Research, 276, 2022.
  13. This looks like that: Deep learning for interpretable image recognition. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Neural Information Processing Systems, NeurIPS, pages 8928–8939, 2019.
  14. Towards prototype-based self-explainable graph neural network. arXiv, 2022.
  15. On the tractability of SHAP explanations. In AAAI Conference on Innovative Applications of Artificial Intelligence, IAAI, pages 6505–6513. AAAI Press, 2021.
  16. Imagenet: A large-scale hierarchical image database. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE Computer Society, 2009.
  17. You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods. Frontiers in Artificial Intelligence and Applications: ECAI, 2020.
  18. Explanations can be manipulated and geometry is to blame. Advances in neural information processing systems, 32, 2019.
  19. Nas-bench-201: Extending the scope of reproducible neural architecture search. In International Conference on Learning Representations, ICLR. OpenReview.net, 2020.
  20. Deformable ProtoPNet: An interpretable image classifier using deformable prototypes. In Conference on Computer Vision and Pattern Recognition, CVPR, pages 10255–10265. IEEE/CVF, 2022.
  21. Towards a rigorous science of interpretable machine learning. arXiv, 2017.
  22. Council of the EU and European Parliament. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L 119:1–88, 2016.
  23. European Commission. Proposal for a regulation of the European Parliament and the Council: Laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206, 4 2021.
  24. William Falcon and The PyTorch Lightning team. PyTorch Lightning, 3 2019. https://github.com/Lightning-AI/lightning.
  25. FastAI. Imagenette. https://github.com/fastai/imagenette, 2020.
  26. Damien Garreau and Ulrike von Luxburg. Explaining the explainer: A first theoretical analysis of LIME. In Silvia Chiappa and Roberto Calandra, editors, International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1287–1296. PMLR, Aug. 2020.
  27. ProtoVAE: A trustworthy self-explainable prototypical variational model. In Neural Information Processing Systems, NeurIPS, 2022.
  28. This looks more like that: Enhancing self-explaining models by prototypical relevance propagation. Pattern Recognition, 136:1–13, 2023.
  29. Kamaledin Ghiasi-Shirazi. Generalizing the convolution operator in convolutional neural networks. Neural Processing Letters, 50(3):2627–2646, 2019.
  30. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 770–778. IEEE Computer Society, 2016.
  31. Natural adversarial examples. In Conference on Computer Vision and Pattern Recognition, pages 15262–15271. IEEE/Computer Vision Foundation, 2021.
  32. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15262–15271, 2021.
  33. FunnyBirds: A synthetic vision dataset for a part-based analysis of explainable AI methods. In IEEE/CVF International Conference on Computer Vision, ICCV, pages 1–18. IEEE, 2023.
  34. This looks like that… does it? Shortcomings of latent space prototype interpretability in deep networks. In ICML Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI, 2022.
  35. Densely connected convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 2261–2269. IEEE Computer Society, 2017.
  36. Is ProtoPNet really explainable? evaluating and improving the interpretability of prototypes. arXiv, 2022.
  37. Comparing effects of attribution-based, example-based, and feature-based explanation methods on ai-assisted decision-making. OSF Preprints, 2022.
  38. How can i choose an explainer? an application-grounded evaluation of post-hoc explanations. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, FAccT’21, page 805–815, New York, NY, USA, 2021. Association for Computing Machinery.
  39. Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. In Regina Bernhaupt, Florian ’Floyd’ Mueller, David Verweij, Josh Andres, Joanna McGrenere, Andy Cockburn, Ignacio Avellino, Alix Goguey, Pernille Bjøn, Shengdong Zhao, Briane Paul Samson, and Rafal Kocielnik, editors, Conference on Human Factors in Computing Systems, pages 1–14. ACM, Apr. 2020.
  40. Proto2Proto: Can you recognize the car, the way I do? In Conference on Computer Vision and Pattern Recognition, CVPR, pages 10223–10233. IEEE/CVF, 2022.
  41. XProtoNet: Diagnosis in chest radiography with global and local explanations. In Conference on Computer Vision and Pattern Recognition, CVPR, pages 15719–15728. CVF/IEEE, 2021.
  42. Vit-net: Interpretable vision transformers with neural tree decoder. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML, volume 162 of Proceedings of Machine Learning Research, pages 11162–11172. PMLR, 2022.
  43. HIVE: evaluating the human interpretability of visual explanations. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, European Conference on Computer Vision, ECCV, volume 13672 of Lecture Notes in Computer Science, pages 280–298. Springer, 2022.
  44. Do better imagenet models transfer better? In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 2661–2671. Computer Vision Foundation / IEEE, 2019.
  45. The receptive field as a regularizer in deep convolutional neural networks for acoustic scene classification. In 27th European Signal Processing Conference, EUSIPCO 2019, A Coruña, Spain, September 2-6, 2019, pages 1–5. IEEE, 2019.
  46. 3D object representations for fine-grained categorization. In 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013.
  47. The disagreement problem in explainable machine learning: A practitioner’s perspective. arXiv, pages 1–46, 2022.
  48. Towards falsifiable interpretability research. In NeurIPS Workshop on ML-Retrospectives, Surveys & Meta-Analyses, pages 1–15. arXiv, 2020.
  49. Library of Congress. H.R.6580 - 117th Congress (2021-2022): Algorithmic accountability act of 2022. https://www.congress.gov/bill/117th-congress/house-bill/6580/text, 2 2022.
  50. Zachary C. Lipton. The mythos of model interpretability. ACM Queue, 16(3):30, July 2018.
  51. SGDR: Stochastic gradient descent with warm restarts. arXiv, 2016.
  52. Understanding the effective receptive field in deep convolutional neural networks. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 4898–4906, 2016.
  53. Sean McGregor. Preventing repeated real world AI failures by cataloging incidents: The AI incident database. In AAAI Conference on Innovative Applications of Artificial Intelligence, IAAI, pages 15458–15463. AAAI Press, 2021.
  54. Artificial intelligence in medical practice: The question to the answer? The American Journal of Medicine, 131(2):129–133, 2018.
  55. Interpretable and steerable sequence learning via prototypes. In Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis, editors, SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD, pages 903–913. ACM, 2019.
  56. mpmath Contributors. Mpmath: A python library for arbitrary-precision floating-point arithmetic, 2022. https://github.com/mpmath/mpmath.
  57. Efficient implementation of a generalized convolutional neural networks based on weighted Euclidean distance. In International Conference on Computer and Knowledge Engineering, ICCKE, pages 211–216, 2017.
  58. This looks like that, because… explaining prototypes for interpretable image recognition. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases - International Workshops of ECML PKDD, volume 1524 of Communications in Computer and Information Science, pages 441–456. Springer, 2021.
  59. Neural prototype trees for interpretable fine-grained image recognition. In Conference on Computer Vision and Pattern Recognition, CVPR, pages 14933–14943. CVF/IEEE, 2021.
  60. Cathy O’Neil. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, Sept. 2016.
  61. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
  62. Prototype-based interpretable graph neural networks. IEEE Transactions on Artificial Intelligence, pages 1–11, 2022.
  63. Tim Räz. ML interpretability: Simple isn’t easy. arXiv, 2022.
  64. Do CIFAR-10 classifiers generalize to CIFAR-10? arXiv, 2018.
  65. Do ImageNet classifiers generalize to ImageNet? In International conference on machine learning, pages 5389–5400. PMLR, 2019.
  66. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215, May 2019.
  67. Interpretable image classification with differentiable prototypes assignment. In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, European Conference on Computer Vision, ECCV, volume 13672 of Lecture Notes in Computer Science, pages 351–368. Springer, 2022.
  68. Protopshare: Prototypical parts sharing for similarity discovery in interpretable image classification. In Feida Zhu, Beng Chin Ooi, and Chunyan Miao, editors, SIGKDD Conference on Knowledge Discovery and Data Mining, KDD, pages 1420–1430. ACM, 2021.
  69. Interpretability benchmark for evaluating spatial misalignment of prototypical parts explanations. arXiv, 2023.
  70. ProtoSeg: Interpretable semantic segmentation with prototypical parts. In Winter Conference on Applications of Computer Vision (WACV), pages 1481–1492. IEEE/CVF, January 2023.
  71. A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery, Jan 2023.
  72. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11):1359–1366, 2011.
  73. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, International Conference on Learning Representations, ICLR, 2015.
  74. Gurmail Singh. Think positive: An interpretable neural network for image recognition. Neural Networks, 151:178–189, 2022.
  75. An interpretable deep learning model for covid-19 detection with chest x-ray images. IEEE Access, 9:85198–85208, 2021.
  76. Object or background: An interpretable deep learning model for COVID-19 detection from CT-scan images. Diagnostics, 11(9):1732, Sep 2021.
  77. These do not look like those: An interpretable deep learning model for image recognition. IEEE Access, 9:41482–41493, 2021.
  78. Towards human-interpretable prototypes for visual assessment of image classification models. arXiv, 2022.
  79. Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods. In Annette N. Markham, Julia Powles, Toby Walsh, and Anne L. Washington, editors, AAAI/ACM Conference on AI, Ethics, and Society (AIES), pages 180–186. ACM, 2020.
  80. Semi-ProtoPNet deep neural network for the classification of defective power grid distribution structures. Sensors, 22(13):4859, 2022.
  81. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun, editors, International Conference on Learning Representations, pages 1–10, Apr. 2014.
  82. A survey on explainable artificial intelligence (XAI): toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems, 32(11):4793–4813, 2020.
  83. Interpretable and trustworthy deepfake detection via dynamic prototypes. In Winter Conference on Applications of Computer Vision, WACV, pages 1972–1982. IEEE, 2021.
  84. U.S.-EU TTC. U.S.-EU joint statement of the Trade and Technology Council. https://www.commerce.gov/news/press-releases/2022/05/us-eu-joint-statement-trade-and-technology-council, 5 2022.
  85. The Caltech-UCSD Birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011.
  86. Learning support and trivial prototypes for interpretable image classification. arXiv, 2023.
  87. Interpretable image recognition by constructing transparent embedding space. In International Conference on Computer Vision, ICCV, pages 875–884. IEEE/CVF, 2021.
  88. ProtoPFormer: Concentrating on prototypical parts in vision transformers for interpretable image recognition. arXiv, 2022.
  89. ProtGNN: Towards self-explaining graph neural networks. In Conference on Innovative Applications of Artificial Intelligence, IAAI, pages 9127–9135. AAAI Press, 2022.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now