Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation (2402.08845v4)

Published 13 Feb 2024 in cs.LG and stat.ME

Abstract: We investigate the problem of explainability for machine learning models, focusing on Feature Attribution Methods (FAMs) that evaluate feature importance through perturbation tests. Despite their utility, FAMs struggle to distinguish the contributions of different features, when their prediction changes are similar after perturbation. To enhance FAMs' discriminative power, we introduce Feature Attribution with Necessity and Sufficiency (FANS), which find a neighborhood of the input such that perturbing samples within this neighborhood have a high Probability of being Necessity and Sufficiency (PNS) cause for the change in predictions, and use this PNS as the importance of the feature. Specifically, FANS compute this PNS via a heuristic strategy for estimating the neighborhood and a perturbation test involving two stages (factual and interventional) for counterfactual reasoning. To generate counterfactual samples, we use a resampling-based approach on the observed samples to approximate the required conditional distribution. We demonstrate that FANS outperforms existing attribution methods on six benchmarks. Please refer to the source code via \url{https://github.com/DMIRLAB-Group/FANS}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Graphframex: Towards systematic evaluation of explainability methods for graph neural networks. In Rieck, B. and Pascanu, R. (eds.), Learning on Graphs Conference, LoG 2022, 9-12 December 2022, Virtual Event, volume 198 of Proceedings of Machine Learning Research, pp.  44. PMLR, 2022. URL https://proceedings.mlr.press/v198/amara22a.html.
  2. Towards better understanding of gradient-based attribution methods for deep neural networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/forum?id=Sy21R9JAW.
  3. On the probability of necessity and sufficiency of explaining graph neural networks: A lower bound optimization approach. arXiv preprint arXiv:2212.07056, 2022.
  4. Concise explanations of neural networks using adversarial training. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp.  1383–1391. PMLR, 2020. URL http://proceedings.mlr.press/v119/chalasani20a.html.
  5. Is this the right neighborhood? accurate and query efficient model agnostic explanations. Advances in Neural Information Processing Systems, 35:9499–9511, 2022.
  6. Interpretable explanations of black boxes by meaningful perturbation. In Proceedings of the IEEE international conference on computer vision, pp.  3429–3437, 2017.
  7. Explaining black-box algorithms using probabilistic contrastive counterfactuals. In Proceedings of the 2021 International Conference on Management of Data, pp.  577–590, 2021.
  8. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  9. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond. Journal of Machine Learning Research, 24(34):1–11, 2023. URL http://jmlr.org/papers/v24/22-0142.html.
  10. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  11. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885–1894. PMLR, 2017.
  12. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896, 2020.
  13. Learning multiple layers of features from tiny images. 2009.
  14. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  15. Cf-gnnexplainer: Counterfactual explanations for graph neural networks. In International Conference on Artificial Intelligence and Statistics, pp.  4499–4511. PMLR, 2022.
  16. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  17. Parameterized explainer for graph neural network. Advances in neural information processing systems, 33:19620–19631, 2020.
  18. Causes of effects: Learning individual responses from population data. arXiv preprint arXiv:2104.13730, 2021.
  19. Pearl, J. Structural counterfactuals: A brief introduction. Cognitive science, 37(6):977–985, 2013.
  20. Pearl, J. Probabilities of causation: three counterfactual interpretations and their identification. In Probabilistic and Causal Inference: The Works of Judea Pearl, pp.  317–372. 2022.
  21. ” why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp.  1135–1144, 2016.
  22. IROF: a low resource evaluation metric for explanation methods. CoRR, abs/2003.08747, 2020. URL https://arxiv.org/abs/2003.08747.
  23. Collective classification in network data. AI magazine, 29(3):93–93, 2008.
  24. Shapley, L. S. et al. A value for n-person games. 1953.
  25. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
  26. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Bengio, Y. and LeCun, Y. (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings, 2014. URL http://arxiv.org/abs/1312.6034.
  27. Striving for simplicity: The all convolutional net. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6806.
  28. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319–3328. PMLR, 2017.
  29. Learning and evaluating graph neural network explanations based on counterfactual and factual reasoning. In Proceedings of the ACM Web Conference 2022, pp. 1018–1027, 2022.
  30. Towards multi-grained explainability for graph neural networks. Advances in Neural Information Processing Systems, 34:18446–18458, 2021.
  31. Local explanations via necessity and sufficiency: Unifying theory and practice. In Uncertainty in Artificial Intelligence, pp.  1382–1392. PMLR, 2021.
  32. A causality inspired framework for model interpretation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  2731–2741, 2023.
  33. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
  34. Invariant learning via probability of sufficient and necessary causes. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a. URL https://openreview.net/forum?id=K5e5tFZuur.
  35. Idgi: A framework to eliminate explanation noise from integrated gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  23725–23734, 2023b.
  36. On the (in) fidelity and sensitivity of explanations. Advances in Neural Information Processing Systems, 32, 2019.
  37. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
  38. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp. 818–833. Springer, 2014.
  39. Mixupexplainer: Generalizing explanations for graph neural networks with data augmentation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.  3286–3296, 2023.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets