Papers
Topics
Authors
Recent
2000 character limit reached

Trade-off Between Efficiency and Consistency for Removal-based Explanations (2210.17426v3)

Published 31 Oct 2022 in cs.LG, cs.AI, cs.CV, and stat.ML

Abstract: In the current landscape of explanation methodologies, most predominant approaches, such as SHAP and LIME, employ removal-based techniques to evaluate the impact of individual features by simulating various scenarios with specific features omitted. Nonetheless, these methods primarily emphasize efficiency in the original context, often resulting in general inconsistencies. In this paper, we demonstrate that such inconsistency is an inherent aspect of these approaches by establishing the Impossible Trinity Theorem, which posits that interpretability, efficiency, and consistency cannot hold simultaneously. Recognizing that the attainment of an ideal explanation remains elusive, we propose the utilization of interpretation error as a metric to gauge inefficiencies and inconsistencies. To this end, we present two novel algorithms founded on the standard polynomial basis, aimed at minimizing interpretation error. Our empirical findings indicate that the proposed methods achieve a substantial reduction in interpretation error, up to 31.8 times lower when compared to alternative techniques. Code is available at https://github.com/trusty-ai/efficient-consistent-explanations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (79)
  1. Explaining individual predictions when features are dependent: More accurate approximations to shapley values. Artificial Intelligence, 298:103502, 2021.
  2. Slic superpixels. Technical report, EPFL, 2010.
  3. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
  4. Improved svrg for non-strongly-convex or sum-of-non-convex objectives. In International Conference on Machine Learning, pp. 1080–1089. PMLR, 2016.
  5. Explaining deep neural networks with a polynomial time algorithm for shapley value approximation. In International Conference on Machine Learning, pp. 272–281. PMLR, 2019.
  6. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
  7. Randall Balestriero. Neural decision trees. ArXiv, abs/1702.07360, 2017.
  8. Interpretability via model extraction. ArXiv, abs/1706.09773, 2017.
  9. William Beckner. Inequalities in fourier analysis. Annals of Mathematics, 102(1):159–182, 1975.
  10. Leo Breiman. Random forests. Machine learning, 45:5–32, 2001.
  11. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  12. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  9650–9660, 2021.
  13. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  782–791, 2021.
  14. An interpretable model with globally consistent explanations for credit risk. ArXiv, abs/1811.12615, 2018.
  15. L-shapley and c-shapley: Efficient model interpretation for structured data. ArXiv, abs/1808.02610, 2019.
  16. Improving kernelshap: Practical shapley value estimation using linear regression. In International Conference on Artificial Intelligence and Statistics, pp.  3457–3465. PMLR, 2021.
  17. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33:17212–17223, 2020.
  18. Explaining by removing: A unified framework for model explanation. The Journal of Machine Learning Research, 22:209–1, 2021.
  19. Information theoretic inequalities. IEEE Transactions on Information theory, 37(6):1501–1518, 1991.
  20. What does explainable ai really mean? a new conceptualization of perspectives. ArXiv, abs/1710.00794, 2017.
  21. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE international conference on computer vision, pp.  3429–3437, 2017.
  22. Three methods to share joint costs or surplus. Journal of Economic Theory, 87(2):275–312, 1999.
  23. Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. Advances in Neural Information Processing Systems, 33:1229–1239, 2020.
  24. An axiomatic approach to the concept of interaction among players in cooperative games. International Journal of Game Theory, 28:547–565, 1999.
  25. Testing booleanity and the uncertainty principle. arXiv preprint arXiv:1204.0944, 2012.
  26. Axiomatic explanations for visual search, retrieval, and similarity learning. In International Conference on Learning Representations, 2022.
  27. Hyperparameter optimization: A spectral approach. ArXiv, abs/1706.00764, 2018.
  28. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  29. Isidore I Hirschman. A note on entropy. American journal of mathematics, 79(1):152–156, 1957.
  30. Explaining explanations: Axiomatic feature interactions for deep networks. J. Mach. Learn. Res., 22:104:1–104:54, 2021.
  31. Fastshap: Real-time shapley value estimation. In International Conference on Learning Representations, 2021.
  32. Adam: A method for stochastic optimization. Arxiv, abs/1412.6980, 2015.
  33. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60:84 – 90, 2012.
  34. Faithful and customizable explanations of black box models. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp.  131–138, 2019.
  35. Microsoft coco: Common objects in context, 2015.
  36. Constant depth circuits, fourier transform, and learnability. Journal of the ACM (JACM), 40(3):607–620, 1993.
  37. Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31–57, 2018.
  38. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, 2017.
  39. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888, 2018.
  40. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pp.  142–150, 2011.
  41. Explanations of black-box models based on directional feature interactions. In International Conference on Learning Representations, 2022.
  42. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern recognition, 65:211–222, 2017.
  43. Ryan O’Donnell. Analysis of boolean functions. Cambridge University Press, 2014.
  44. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
  45. Guillermo Owen. Multilinear extensions of games. Management Science, 18(5-part-2):64–79, 1972.
  46. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing, pp.  1532–1543, 2014.
  47. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.  1135–1144, 2016.
  48. Cxplain: Causal explanations for model interpretation under uncertainty. Advances in Neural Information Processing Systems, 32, 2019.
  49. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128:336–359, 2017.
  50. Lloyd S. Shapley. Quota solutions of n-person games. Edited by Emil Artin and Marston Morse, pp.  343, 1953.
  51. Not just a black box: Learning important features through propagating activation differences. CoRR, abs/1605.01713, 2016.
  52. Learning important features through propagating activation differences. In International Conference on Machine Learning, pp. 3145–3153. PMLR, 2017.
  53. Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/1312.6034, 2014.
  54. Smoothgrad: removing noise by adding noise. ArXiv, abs/1706.03825, 2017.
  55. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pp.  1631–1642, 2013.
  56. Real analysis: measure theory, integration, and Hilbert spaces. Princeton University Press, 2009.
  57. Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3):647–665, 2014.
  58. Interpretable two-level boolean rule learning for classification. ArXiv, abs/1511.07361, 2015.
  59. The many shapley values for model explanation. In International Conference on Machine Learning, pp. 9269–9278. PMLR, 2020.
  60. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pp. 3319–3328. PMLR, 2017.
  61. The shapley taylor interaction index. In International conference on machine learning, pp. 9259–9268. PMLR, 2020.
  62. Faith-shap: The faithful shapley shapley interaction index. arXiv preprint arXiv:2203.00870, 2022.
  63. How does this interaction affect me? interpretable attribution for feature interactions. Advances in Neural Information Processing Systems, 33:6147–6159, 2020.
  64. Methods and models for interpretable linear classification. ArXiv, abs/1405.4047, 2014.
  65. Supersparse linear integer models for interpretable classification. ArXiv, abs/1306.6677, 2014.
  66. Trading interpretability for accuracy: Oblique treed sparse additive models. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.  1245–1254, 2015a.
  67. Shapley flow: A graph-based approach to interpreting model predictions. In International Conference on Artificial Intelligence and Statistics, pp.  721–729. PMLR, 2021a.
  68. Shapley explanation networks. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021b.
  69. Or’s of and’s for interpretable classification, with application to context-aware recommender systems. ArXiv, abs/1504.07614, 2015b.
  70. A bayesian framework for learning rule sets for interpretable classification. The Journal of Machine Learning Research, 18(1):2357–2393, 2017.
  71. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022.
  72. Robert J Weber. Probabilistic values for games, the shapley value. essays in honor of lloyd s. shapley (ae roth, ed.), 1988.
  73. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.
  74. Attribution in scale and space. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 9677–9686. Computer Vision Foundation / IEEE, 2020. doi: 10.1109/CVPR42600.2020.00970.
  75. Deep neural decision trees. ArXiv, abs/1806.06988, 2018.
  76. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023.
  77. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning, pp. 12241–12252. PMLR, 2021.
  78. Interpreting multivariate shapley interactions in dnns. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  10877–10886, 2021.
  79. Cumulative reasoning with large language models. arXiv preprint arXiv:2308.04371, 2023.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Video Overview

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up