Papers
Topics
Authors
Recent
2000 character limit reached

I Bet You Did Not Mean That: Testing Semantic Importance via Betting

Published 29 May 2024 in stat.ML and cs.LG | (2405.19146v2)

Abstract: Recent works have extended notions of feature importance to semantic concepts that are inherently interpretable to the users interacting with a black-box predictive model. Yet, precise statistical guarantees, such as false positive rate and false discovery rate control, are needed to communicate findings transparently and to avoid unintended consequences in real-world scenarios. In this paper, we formalize the global (i.e., over a population) and local (i.e., for a sample) statistical importance of semantic concepts for the predictions of opaque models by means of conditional independence, which allows for rigorous testing. We use recent ideas of sequential kernelized independence testing (SKIT) to induce a rank of importance across concepts, and showcase the effectiveness and flexibility of our framework on synthetic datasets as well as on image classification tasks using several and diverse vision-LLMs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American mathematical society, 68(3):337–404, 1950.
  2. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6541–6549, 2017.
  3. Reproducing kernel Hilbert spaces in probability and statistics. Springer Science & Business Media, 2011.
  4. The conditional permutation test for independence while controlling for confounders. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(1):175–197, 2020.
  5. Interpreting clip with sparse linear concept embeddings (splice). arXiv preprint arXiv:2402.10376, 2024.
  6. Interpreting black box models via hypothesis testing. In Proceedings of the 2020 ACM-IMS on foundations of data science conference, pages 47–57, 2020.
  7. Panning for gold:‘model-x’knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80(3):551–577, 2018.
  8. Information maximization perspective of orthogonal matching pursuit with applications to explainable ai. Advances in Neural Information Processing Systems, 36, 2024.
  9. Invite: Interpret and control vision-language models with text explanations. In The Twelfth International Conference on Learning Representations.
  10. L-shapley and c-shapley: Efficient model interpretation for structured data. arXiv preprint arXiv:1808.02610, 2018.
  11. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2818–2829, 2023.
  12. Thomas M Cover. Universal portfolios. Mathematical finance, 1(1):1–29, 1991.
  13. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33:17212–17223, 2020.
  14. Black-box reductions for parameter-free online learning in banach spaces. In Conference On Learning Theory, pages 1493–1529. PMLR, 2018.
  15. Skincon: A skin disease dataset densely annotated by domain experts for fine-grained debugging and analysis. Advances in Neural Information Processing Systems, 35:18157–18167, 2022.
  16. A survey of the state of explainable ai for natural language processing. arXiv preprint arXiv:2010.00711, 2020.
  17. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  18. Christiane Fellbaum. WordNet: An electronic lexical database. MIT press, 1998.
  19. Sequential permutation testing by betting. arXiv preprint arXiv:2401.07365, 2024.
  20. Kernel measures of conditional dependence. Advances in neural information processing systems, 20, 2007.
  21. Interpreting clip’s image representation via text-based decomposition. arXiv preprint arXiv:2310.05916, 2023.
  22. Large sample analysis of the median heuristic. arXiv preprint arXiv:1707.07269, 2017.
  23. A kernel statistical test of independence. Advances in neural information processing systems, 20, 2007.
  24. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
  25. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  26. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668–2677. PMLR, 2018.
  27. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  28. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR, 2020.
  29. Learning interpretable queries for explainable image classification with information pursuit. arXiv preprint arXiv:2312.11548, 2023.
  30. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  31. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  32. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020.
  33. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  34. A rate-distortion framework for explaining neural network decisions. arXiv preprint arXiv:1905.11092, 2019.
  35. Diconstruct: Causal concept-based explanations through black-box distillation. arXiv preprint arXiv:2401.08534, 2024.
  36. Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probability, 29(2):429–443, 1997.
  37. Clip-dissect: Automatic description of neuron representations in deep vision networks. arXiv preprint arXiv:2204.10965, 2022.
  38. Label-free concept bottleneck models. arXiv preprint arXiv:2304.06129, 2023.
  39. A measure-theoretic approach to kernel conditional mean embeddings. Advances in neural information processing systems, 33:21247–21259, 2020.
  40. Sequential kernelized independence testing. In International Conference on Machine Learning, pages 27957–27993. PMLR, 2023.
  41. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  42. Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10932–10941, 2023.
  43. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.
  44. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206–215, 2019.
  45. Explaining the behavior of black-box prediction algorithms with causal learning. arXiv preprint arXiv:2006.02482, 2020.
  46. David W Scott. Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, 2015.
  47. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  48. Model-x sequential testing for conditional independence via testing by betting. In International Conference on Artificial Intelligence and Statistics, pages 2054–2086. PMLR, 2023.
  49. Glenn Shafer. Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(2):407–431, 2021.
  50. Game-theoretic foundations for probability and finance, volume 455. John Wiley & Sons, 2019.
  51. Kernel methods for pattern analysis. Cambridge university press, 2004.
  52. Nonparametric two-sample testing by betting. IEEE Transactions on Information Theory, 2023.
  53. On distance and kernel measures of conditional dependence. Journal of Machine Learning Research, 24(7):1–16, 2023.
  54. Cavli-using image associations to produce local concept-based explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3749–3754, 2023.
  55. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 961–968, 2009.
  56. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  57. Charles Spearman. The proof and measurement of association between two things. 1961.
  58. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research, 11:1517–1561, 2010.
  59. The holdout randomization test for feature selection in black box models. Journal of Computational and Graphical Statistics, 31(1):151–162, 2022.
  60. Fast hierarchical games for image explanations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4494–4503, 2022.
  61. Shap-xrt: The shapley value meets conditional independence testing. Transactions on Machine Learning Research, 2023.
  62. Feature importance: A closer look at shapley values and loco. arXiv preprint arXiv:2303.05981, 2023.
  63. Decorrelated variable importance. Journal of Machine Learning Research, 25(7):1–27, 2024.
  64. Jean Ville. Etude critique de la notion de collectif. Gauthier-Villars Paris, 1939.
  65. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736–1754, 2021.
  66. Pre-trained language models in biomedical domain: A systematic survey. ACM Computing Surveys, 56(3):1–52, 2023.
  67. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(3):822–852, 2022.
  68. Medclip: Contrastive learning from unpaired medical images and text. arXiv preprint arXiv:2210.10163, 2022.
  69. Discover and cure: Concept-aware mitigation of spurious correlation. In International Conference on Machine Learning, pages 37765–37786. PMLR, 2023.
  70. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187–19197, 2023.
  71. Explainability in graph neural networks: A taxonomic survey. IEEE transactions on pattern analysis and machine intelligence, 45(5):5782–5799, 2022.
  72. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15480, 2022.
  73. Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv:1202.3775, 2012.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Jacopo Teneggi 
  2. Jeremias Sulam 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 3 tweets with 12 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free