Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 88 tok/s
Gemini 2.5 Pro 47 tok/s Pro
GPT-5 Medium 21 tok/s Pro
GPT-5 High 13 tok/s Pro
GPT-4o 81 tok/s Pro
Kimi K2 175 tok/s Pro
GPT OSS 120B 450 tok/s Pro
Claude Sonnet 4 39 tok/s Pro
2000 character limit reached

Sparse and Faithful Explanations Without Sparse Models (2402.09702v3)

Published 15 Feb 2024 in cs.LG and stat.ML

Abstract: Even if a model is not globally sparse, it is possible for decisions made from that model to be accurately and faithfully described by a small number of features. For instance, an application for a large loan might be denied to someone because they have no credit history, which overwhelms any evidence towards their creditworthiness. In this work, we introduce the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models. In the loan denial example above, the SEV is 1 because only one factor is needed to explain why the loan was denied. SEV is a measure of decision sparsity rather than overall model sparsity, and we are able to show that many machine learning models -- even if they are not sparse -- actually have low decision sparsity, as measured by SEV. SEV is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints. We proposed the algorithms that reduce SEV without sacrificing accuracy, providing sparse and completely faithful explanations, even without globally sparse models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. George Miller. The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63:81–97, 1956.
  2. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44):22071–22080, 2019.
  3. Sparse additive models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(5):1009–1030, November 2009.
  4. Learning structured sparsity in deep neural networks. Advances in neural information processing systems, 29, 2016.
  5. Review of sparse methods in regression and classification with application to chemometrics. Journal of Chemometrics, 26(3-4), 2012.
  6. Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (Series B), 1996.
  7. Optimization with sparsity-inducing penalties. Foundations and Trends in Machine Learning, 4(1):1–106, 2012.
  8. A survey of sparse representation: algorithms and applications. IEEE Access, 3:490–530, 2015.
  9. William S Cleveland and E. Grosse. Computational methods for local regression. Statistics and Computing, 1991.
  10. Local classification: Locally weighted–partial least squares-discriminant analysis (LW–PLS-DA). Analytica Chimica Acta, 2014.
  11. Datum-wise classification: A sequential approach to sparsity. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2011, Athens, Greece, September 5-9, 2011. Proceedings, Part I, volume 6911 of Lecture Notes in Computer Science, pages 375–390. Springer, 2011.
  12. A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), 2018.
  13. “Why should I trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd International Conference on Knowledge Discovery and Data Mining, 2016.
  14. Anchors: High-precision model-agnostic explanations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
  15. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems, pages 4765–4774, 2017.
  16. How to explain individual classification decisions. Journal of Machine Learning Research, 11:1803–1831, aug 2010. ISSN 1532-4435.
  17. Contrastive explanations to classification systems using sparse dictionaries. In Lecture Notes in Computer Science, pages 207–218. Springer International Publishing, 2019.
  18. Middle-level features for the explanation of classification systems by sparse dictionary methods. International Journal of Neural Systems, 30(08):2050040, July 2020.
  19. Sparse visual counterfactual explanations in image space. In Pattern Recognition: 44th DAGM German Conference, DAGM GCPR 2022, Konstanz, Germany, September 27–30, 2022, Proceedings, pages 133–148. Springer, 2022.
  20. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016.
  21. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220, 2016.
  22. The explanation game: Towards prediction explainability through sparse communication. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020.
  23. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2963–2977, 2019.
  24. Rethinking cooperative rationalization: Introspective extraction and complement control. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4094––4103, 2019.
  25. Understanding interlocking dynamics of cooperative rationalization. Advances in Neural Information Processing Systems, 34:12822–12835, 2021.
  26. Learning to explain: An information-theoretic perspective on model interpretation. In International Conference on Machine Learning, pages 883–892. PMLR, 2018.
  27. Explaining data-driven document classifications. MIS quarterly, 38(1):73–100, 2014.
  28. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 10–19, 2019.
  29. Optimal action extraction for random forests and boosted trees. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 179–188, 2015.
  30. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 465–474, 2017.
  31. Focus: Flexible optimizable counterfactual explanations for tree ensembles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 5313–5322, 2022.
  32. Equalizing recourse across groups. arXiv preprint arXiv:1909.03166, 2019.
  33. Interpreting neural network judgments via minimal, stable, and symbolic corrections. Advances in Neural Information Processing Systems, 31, 2018.
  34. Learning models for actionable recourse. Advances in Neural Information Processing Systems, 34:18734–18746, 2021.
  35. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31:841, 2017.
  36. Interpretable credit application predictions with counterfactual explanations. In NIPS 2018-Workshop on Challenges and Opportunities for AI in Financial Services: the Impact of Fairness, Explainability, Accuracy, and Privacy, 2018.
  37. Chris Russell. Efficient search for diverse coherent explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 20–28, 2019.
  38. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615, 2019.
  39. A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: SEDC, LIME-C and SHAP-C. Advances in Data Analysis and Classification, 14:801–819, 2020.
  40. Comparison-based inverse classification for interpretability in machine learning. In Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations: 17th International Conference, IPMU 2018, Cádiz, Spain, June 11-15, 2018, Proceedings, Part I 17, pages 100–111. Springer, 2018.
  41. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. Advances in Neural Information Processing Systems, 31, 2018.
  42. Interpretable counterfactual explanations guided by prototypes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 650–665. Springer, 2021.
  43. Dece: Decision explorer with counterfactual explanations for machine learning models. IEEE Transactions on Visualization and Computer Graphics, 27(2):1438–1447, 2020.
  44. Generalized inverse classification. In Proceedings of the 2017 SIAM International Conference on Data Mining, pages 162–170. SIAM, 2017a.
  45. A budget-constrained inverse classification framework for smooth classifiers. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pages 1184–1193. IEEE, 2017b.
  46. Explaining data-driven decisions made by ai systems: The counterfactual approach. MIS Quarterly, 46(3):1635–1660, 2022.
  47. Counterfactual explanations for misclassified images: How human and machine explanations differ. Artificial Intelligence, 324:103995, 2023.
  48. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
  49. FICO. Explainable Machine Learning Challenge, 2018. URL https://community.fico.com/s/explainable-machine-learning-challenge. Accessed: 2018-11-02.
  50. MIMIC-III clinical database. https://physionet.org/content/mimiciii/, 2016a.
  51. MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1), May 2016b. doi: 10.1038/sdata.2016.35. URL https://doi.org/10.1038/sdata.2016.35.
  52. Lauren Kirchner Jeff Larson, Surya Mattu and Julia Angwin. How we analyzed the COMPAS recidivism algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm, 2016. Accessed: 2023-02-01.
  53. In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction. Journal of Quantitative Criminology, pages 1–63, 2022. ISSN 0748-4518. doi: 10.1007/s10940-022-09545-w.
  54. Impact of HbA1c measurement on hospital readmission rates: Analysis of 70, 000 clinical database patient records. BioMed Research International, 2014:1–11, 2014.
  55. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 607–617, 2020.
  56. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  57. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019. URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 2 posts and received 9 likes.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up to View