Explaining the Model and Feature Dependencies by Decomposition of the Shapley Value (2306.10880v1)
Abstract: Shapley values have become one of the go-to methods to explain complex models to end-users. They provide a model agnostic post-hoc explanation with foundations in game theory: what is the worth of a player (in machine learning, a feature value) in the objective function (the output of the complex machine learning model). One downside is that they always require outputs of the model when some features are missing. These are usually computed by taking the expectation over the missing features. This however introduces a non-trivial choice: do we condition on the unknown features or not? In this paper we examine this question and claim that they represent two different explanations which are valid for different end-users: one that explains the model and one that explains the model combined with the feature dependencies in the data. We propose a new algorithmic approach to combine both explanations, removing the burden of choice and enhancing the explanatory power of Shapley values, and show that it achieves intuitive results on simple problems. We apply our method to two real-world datasets and discuss the explanations. Finally, we demonstrate how our method is either equivalent or superior to state-to-of-art Shapley value implementations while simultaneously allowing for increased insight into the model-data structure.
- Explaining individual predictions when features are dependent: More accurate approximations to shapley values. Artificial Intelligence, 298:103502.
- Explaining predictive models using shapley values and non-parametric vine copulas. Dependence Modeling, 9(1):62–81.
- Predicting forest fire in algeria using data mining techniques: Case study of the decision tree algorithm. In International Conference on Advanced Intelligent Systems for Sustainable Development, pages 363–370. Springer.
- True to the model or true to the data? arXiv preprint arXiv:2006.16234.
- Shapley explainability on the data manifold. arXiv preprint arXiv:2006.01272.
- Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. Advances in Neural Information Processing Systems, 33:1229–1239.
- Hedonic housing prices and the demand for clean air. Journal of environmental economics and management, 5(1):81–102.
- Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models. Advances in Neural Information Processing Systems, 33.
- Feature relevance quantification in explainable ai: A causal problem. In International Conference on Artificial Intelligence and Statistics, pages 2907–2916. PMLR.
- Shapley residuals: Quantifying the limits of the shapley value for explanations. Advances in Neural Information Processing Systems, 34:26598–26608.
- Problems with shapley-value-based explanations as feature importance measures. In International Conference on Machine Learning, pages 5491–5500. PMLR.
- Racial disparities and similarities in post-release recidivism and employment among ex-prisoners with a different level of education. Journal of Prison Education and Reentry, 2(1):16–31.
- Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888.
- A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, pages 4765–4774.
- Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering, 2(10):749.
- ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
- Impact of obesity on autonomic modulation, heart rate and blood pressure in obese young people. Autonomic neuroscience, 193:138–141.
- Shapley, L. S. (1953). A value for n-person games. Contributions to the Theory of Games, 2(28):307–317.
- Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 180–186.
- Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems, 41(3):647–665.
- The shapley taylor interaction index. In International conference on machine learning, pages 9259–9268. PMLR.
- The many shapley values for model explanation. In International Conference on Machine Learning, pages 9269–9278. PMLR.
- Takeishi, N. (2019). Shapley values of reconstruction errors of pca for explaining anomaly detection. In 2019 International Conference on Data Mining Workshops (ICDMW), pages 793–798. IEEE.
- Shapley flow: A graph-based approach to interpreting model predictions. In International Conference on Artificial Intelligence and Statistics, pages 721–729. PMLR.
- Threading the needle of on and off-manifold value functions for shapley explanations. In International Conference on Artificial Intelligence and Statistics, pages 1485–1502. PMLR.