Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SHAP-XRT: The Shapley Value Meets Conditional Independence Testing (2207.07038v5)

Published 14 Jul 2022 in cs.LG

Abstract: The complex nature of artificial neural networks raises concerns on their reliability, trustworthiness, and fairness in real-world scenarios. The Shapley value -- a solution concept from game theory -- is one of the most popular explanation methods for machine learning models. More traditionally, from a statistical perspective, feature importance is defined in terms of conditional independence. So far, these two approaches to interpretability and feature importance have been considered separate and distinct. In this work, we show that Shapley-based explanation methods and conditional independence testing are closely related. We introduce the SHAPley EXplanation Randomization Test (SHAP-XRT), a testing procedure inspired by the Conditional Randomization Test (CRT) for a specific notion of local (i.e., on a sample) conditional independence. With it, we prove that for binary classification problems, the marginal contributions in the Shapley value provide lower and upper bounds to the expected $p$-values of their respective tests. Furthermore, we show that the Shapley value itself provides an upper bound to the expected $p$-value of a global (i.e., overall) null hypothesis. As a result, we further our understanding of Shapley-based explanation methods from a novel perspective and characterize the conditions under which one can make statistically valid claims about feature importance via the Shapley value.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (91)
  1. Explaining individual predictions when features are dependent: More accurate approximations to shapley values. Artificial Intelligence, 298:103502, 2021.
  2. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
  3. The conditional permutation test for independence while controlling for confounders. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(1):175–197, 2020.
  4. A better bound on the variance. The american mathematical monthly, 107(4):353–357, 2000.
  5. Selection of relevant features and examples in machine learning. Artificial intelligence, 97(1-2):245–271, 1997.
  6. Transparency, auditability, and explainability of machine learning models in credit scoring. Journal of the Operational Research Society, 73(1):70–90, 2022.
  7. Interpreting black box models via hypothesis testing. In Proceedings of the 2020 ACM-IMS on Foundations of Data Science Conference, pages 47–57, 2020.
  8. Panning for gold: ‘model-x’ knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(3):551–577, 2018.
  9. Rethinking explainable machines: The gdpr’s’ right to explanation’debate and the rise of algorithmic audits in enterprise. Berkeley Tech. LJ, 34:143, 2019.
  10. True to the model or true to the data? arXiv preprint arXiv:2006.16234, 2020.
  11. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, pages 1–12, 2023.
  12. L-shapley and c-shapley: Efficient model interpretation for structured data. arXiv preprint arXiv:1808.02610, 2018.
  13. L-shapley and c-shapley: Efficient model interpretation for structured data. In International Conference on Learning Representations, 2019.
  14. Feature selection via coalitional game theory. Neural Computation, 19(7):1939–1961, 2007.
  15. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33:17212–17223, 2020.
  16. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209):1–90, 2021.
  17. Learning to estimate shapley values with vision transformers. arXiv preprint arXiv:2206.05282, 2022.
  18. Deepstarr predicts enhancer activity from dna sequence and enables the de novo design of synthetic enhancers. Nature Genetics, 54(5):613–624, 2022.
  19. Explainable ai for interpretable credit scoring. arXiv preprint arXiv:2012.03749, 2020.
  20. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  21. Value theory without efficiency. Mathematics of Operations Research, 6(1):122–128, 1981.
  22. R. A. Fisher. Questions and answers. The American Statistician, 2(5):30–31, 1948. ISSN 00031305. URL http://www.jstor.org/stable/2681650.
  23. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE international conference on computer vision, pages 3429–3437, 2017.
  24. Shapley explainability on the data manifold. arXiv preprint arXiv:2006.01272, 2020a.
  25. Asymmetric shapley values: incorporating causal knowledge into model-agnostic explainability. Advances in Neural Information Processing Systems, 33:1229–1239, 2020b.
  26. Shapley values for feature selection: The good, the bad, and the axioms. IEEE Access, 9:144352–144360, 2021.
  27. An omnibus test for the global null hypothesis. Statistical methods in medical research, 28(8):2292–2304, 2019.
  28. A distributional framework for data valuation. In International Conference on Machine Learning, pages 3535–3544. PMLR, 2020.
  29. An introduction to variable and feature selection. Journal of machine learning research, 3(Mar):1157–1182, 2003.
  30. Explainable ai under contract and tort law: legal incentives and technical challenges. Artificial Intelligence and Law, 28(4):415–439, 2020.
  31. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  32. Choosing between methods of combining-values. Biometrika, 105(1):239–246, 2018.
  33. Gerhard Hommel. Tests of the overall hypothesis for arbitrary dependence structures. Biometrical Journal, 25(5):423–430, 1983.
  34. Multiple hypotheses testing based on ordered p values—a historical survey with applications to medical research. Journal of biopharmaceutical statistics, 21(4):595–609, 2011.
  35. Giles Hooker. Generalized functional anova diagnostics for high-dimensional functions of dependent variables. Journal of Computational and Graphical Statistics, 16(3):709–732, 2007.
  36. Missingness bias in model debugging. arXiv preprint arXiv:2204.08945, 2022.
  37. An introduction to statistical learning, volume 112. Springer, 2013.
  38. Feature relevance quantification in explainable ai: A causal problem. In International Conference on artificial intelligence and statistics, pages 2907–2916. PMLR, 2020.
  39. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing, 30:5875–5888, 2021.
  40. Algorithmic impact assessments under the gdpr: producing multi-layered explanations. International Data Privacy Law, pages 19–28, 2020.
  41. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  42. Cartoon explanations of image classifiers. arXiv preprint arXiv:2110.03485, 2021.
  43. A rate-distortion framework for explaining black-box model decisions. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers, pages 91–115. Springer, 2022.
  44. Weightedshap: analyzing and improving shapley based feature attributions. arXiv preprint arXiv:2209.13429, 2022.
  45. Deep learning. nature, 521(7553):436–444, 2015.
  46. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018.
  47. Fast and powerful conditional randomization testing via distillation. Biometrika, 109(2):277–293, 2022.
  48. Evaluating eligibility criteria of oncology trials using real-world data and ai. Nature, 592(7855):629–633, 2021.
  49. Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures. Journal of the American Statistical Association, 115(529):393–402, 2020.
  50. Annotated high-throughput microscopy image sets for validation. Nature methods, 9(7):637–637, 2012.
  51. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  52. From local explanations to global understanding with explainable ai for trees. Nature machine intelligence, 2(1):56–67, 2020.
  53. Predictive and causal implications of using shapley value for model interpretation. In Proceedings of the 2020 KDD Workshop on Causal Discovery, pages 23–38. PMLR, 2020.
  54. The explanation game: Explaining machine learning models using shapley values. In Machine Learning and Knowledge Extraction: 4th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2020, Dublin, Ireland, August 25–28, 2020, Proceedings 4, pages 17–38. Springer, 2020.
  55. Explainable machine learning can outperform cox regression predictions and provide insights in breast cancer survival. Scientific Reports, 11(1):1–13, 2021.
  56. Shap-based explanation methods: A review for nlp interpretability. In COLING, 2022.
  57. Rectified linear units improve restricted boltzmann machines. In Icml, 2010.
  58. Ecko: Ensemble of clustered knockoffs for robust multivariate inference on fmri data. In International Conference on Information Processing in Medical Imaging, pages 454–466. Springer, 2019.
  59. On shapley value for measuring importance of dependent inputs. SIAM/ASA Journal on Uncertainty Quantification, 5(1):986–1002, 2017.
  60. Guillermo Owen. Game theory. Emerald Group Publishing, 2013.
  61. On some series of functions,(3). In Mathematical Proceedings of the Cambridge Philosophical Society, volume 28, pages 190–205. Cambridge University Press, 1932.
  62. Karl Pearson. On a method of determining whether a sample of size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika, pages 379–410, 1933.
  63. Introduction to the theory of cooperative games, volume 34. Springer Science & Business Media, 2007.
  64. "why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  65. The shapley value in machine learning. arXiv preprint arXiv:2202.05594, 2022.
  66. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215, 2019.
  67. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  68. Individualized conditional independence testing under model-x with heterogeneous samples and interactions. arXiv preprint arXiv:2205.08653, 2022.
  69. False discovery rate control in genome-wide association studies with population structure. Proceedings of the National Academy of Sciences, 118(40):e2105841118, 2021.
  70. Juliet Popper Shaffer. Multiple hypothesis testing. Annual review of psychology, 46(1):561–584, 1995.
  71. Do input gradients highlight discriminative features? Advances in Neural Information Processing Systems, 34, 2021.
  72. Lloyd S Shapley. Notes on the N-person Game. Rand Corporation, 1951.
  73. Donghee Shin. The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable ai. International Journal of Human-Computer Studies, 146:102551, 2021.
  74. Learning important features through propagating activation differences. In International conference on machine learning, pages 3145–3153. PMLR, 2017.
  75. The many shapley values for model explanation. In International conference on machine learning, pages 9269–9278. PMLR, 2020.
  76. The holdout randomization test for feature selection in black box models. Journal of Computational and Graphical Statistics, 31(1):151–162, 2022.
  77. Fast hierarchical games for image explanations. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–11, 2022a. doi: 10.1109/TPAMI.2022.3189849.
  78. Weakly supervised learning significantly reduces the number of labels required for intracranial hemorrhage detection on head ct. arXiv preprint arXiv:2211.15924, 2022b.
  79. Leonard Henry Caleb Tippett et al. The methods of statistics. The Methods of Statistics., 1931.
  80. Interpretable to whom? a role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552, 2018.
  81. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  82. Decorrelated variable importance. arXiv preprint arXiv:2111.10853, 2021.
  83. Feature importance: A closer look at shapley values and loco. arXiv preprint arXiv:2303.05981, 2023.
  84. Combining p-values via averaging. Biometrika, 107(4):791–808, 2020.
  85. Data banzhaf: A robust data valuation framework for machine learning. In International Conference on Artificial Intelligence and Statistics, pages 6388–6421. PMLR, 2023.
  86. Explaining predictive uncertainty with information theoretic shapley values. arXiv preprint arXiv:2306.05724, 2023.
  87. Daniel J Wilson. The harmonic mean p-value for combining dependent tests. Proceedings of the National Academy of Sciences, 116(4):1195–1200, 2019.
  88. Deep sets. Advances in neural information processing systems, 30, 2017.
  89. Carlos Zednik. Solving the black box problem: a normative framework for explainable artificial intelligence. Philosophy & Technology, 34(2):265–288, 2021.
  90. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.
  91. Machine learning-based prediction of covid-19 diagnosis based on symptoms. npj digital medicine, 4(1):1–5, 2021.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now