Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Supervised Feature Compression based on Counterfactual Analysis (2211.09894v4)

Published 17 Nov 2022 in cs.LG and math.OC

Abstract: Counterfactual Explanations are becoming a de-facto standard in post-hoc interpretable machine learning. For a given classifier and an instance classified in an undesired class, its counterfactual explanation corresponds to small perturbations of that instance that allows changing the classification outcome. This work aims to leverage Counterfactual Explanations to detect the important decision boundaries of a pre-trained black-box model. This information is used to build a supervised discretization of the features in the dataset with a tunable granularity. Using the discretized dataset, an optimal Decision Tree can be trained that resembles the black-box model, but that is interpretable and compact. Numerical results on real-world datasets show the effectiveness of the approach in terms of accuracy and sparsity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Strong optimal classification trees. arXiv preprint arXiv:2103.15965.
  2. Model extraction from counterfactual explanations. arXiv preprint arXiv:2009.01884.
  3. Beware explanations from AI in health care. Science, 373(6552):284–286.
  4. Explainable models of credit losses. European Journal of Operational Research, 301(1):386–394.
  5. Optimal classification trees. Machine Learning, 106:1039–1082.
  6. Breiman, L. (2001). Random forests. Machine Learning, 45(1):5–32.
  7. Classification and Regression Trees. Chapman and Hall/CRC.
  8. Mathematical optimization in classification and regression trees. TOP, 29(1):5–33.
  9. Generating collective counterfactual explanations in score-based classification via mathematical optimization. Technical report, IMUS, Sevilla, Spain, https://www.researchgate.net/publication/353073138_Generating_Collective_Counterfactual_Explanations_in_Score-Based_Classification_via_Mathematical_Optimization.
  10. Mathematical optimization modelling for group counterfactual explanations. Technical report, IMUS, Sevilla, Spain, https://www.researchgate.net/publication/368958766_Mathematical_Optimization_Modelling_for_Group_Counterfactual_Explanations.
  11. XGBoost: extreme gradient boosting. R package version 0.4-2, 1(4):1–4.
  12. Comparative analysis of supervised and unsupervised discretization techniques. International Journal of Advances in Science and Technology, 2(3):29–37.
  13. Supervised and unsupervised discretization of continuous features. In Prieditis, A. and Russell, S., editors, Machine Learning Proceedings 1995, pages 194–202. Morgan Kaufmann, San Francisco (CA).
  14. Operational research and artificial intelligence methods in banking. European Journal of Operational Research, 306(1):1–16.
  15. Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects. European Journal of Operational Research, 297(3):1178–1192.
  16. Dunn, J. W. (2018). Optimal trees for prediction and prescription. PhD thesis, Massachusetts Institute of Technology.
  17. European Commission (2020). White Paper on Artificial Intelligence : a European approach to excellence and trust. https://ec.europa.eu/info/publications/white-paper-artificial-intelligence-european-approach-excellence-and-trust_en.
  18. Factual and counterfactual explanations in fuzzy classification trees. IEEE Transactions on Fuzzy Systems, 30(12):5484–5495.
  19. Assessing bank efficiency and performance with operational research and artificial intelligence techniques: A survey. European Journal of Operational Research, 204(2):189–198.
  20. Handling numeric attributes when comparing bayesian network classifiers: does the discretization method matter? Applied Intelligence, 34(3):372–385.
  21. Robust counterfactual explanations for random forests. arXiv preprint arXiv:2205.14116.
  22. A survey of discretization techniques: Taxonomy and empirical analysis in supervised learning. IEEE Transactions on Knowledge and Data Engineering, 25(4):734–750.
  23. PreCoF: counterfactual explanations for fairness. Forthcoming in Machine Learning.
  24. European union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine, 38(3):50–57.
  25. Guidotti, R. (2022). Counterfactual explanations and how to find them: literature review and benchmarking. Forthcoming in Data Mining and Knowledge Discovery.
  26. Optimal decision trees for categorical data via integer programming. Journal of Global Optimization, 81(1):233–260.
  27. Gurobi (2021). Gurobi optimizer reference manual. http://www.gurobi.com.
  28. A survey of algorithmic recourse: contrastive explanations and consequential recommendations. ACM Computing Surveys, 55(5):1–29.
  29. Fairness in credit scoring: Assessment, implementation and profit implications. European Journal of Operational Research, 297(3):1083–1094.
  30. Adversarial XAI methods in cybersecurity. IEEE Transactions on Information Forensics and Security, 16:4924–4938.
  31. Counterfactual Fairness. Advances in Neural Information Processing Systems, 30:4066–4076.
  32. Constructing optimal binary decision trees is NP-complete. Information Processing Letters, 5(1):15–17.
  33. Generalized and scalable optimal sparse decision trees. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 6150–6160. PMLR.
  34. Finding regions of counterfactual explanations via robust optimization. arXiv preprint arXiv:2301.11113.
  35. Counterfactual explanations using optimization with constraint learning. arXiv preprint arXiv:2209.10997.
  36. Explaining data-driven document classifications. MIS Quarterly, 38(1):73–99.
  37. Fast sparse decision tree optimization via reference ensembles. Proceedings of the AAAI Conference on Artificial Intelligence, 36(9):9604–9613.
  38. Interpretable machine learning - a brief history, state-of-the-art and challenges. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 417–431. Springer.
  39. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 607–617. Association for Computing Machinery.
  40. Optimal counterfactual explanations in tree ensembles. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 8422–8431. PMLR.
  41. Piramuthu, S. (2004). Evaluating feature selection methods for learning in data mining applications. European Journal of Operational Research, 156(2):483–494.
  42. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1):81–106.
  43. Quinlan, J. R. (2014). C4.5: programs for machine learning. Elsevier.
  44. Data discretization: taxonomy and big data challenge. WIREs Data Mining and Knowledge Discovery, 6(1):5–21.
  45. Ridgeway, G. (2013). The pitfalls of prediction. National Institute of Justice Journal, 271:34–40.
  46. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistics Surveys, 16:1–85.
  47. A class of categorization methods for credit scoring models. European Journal of Operational Research, 296(1):323–331.
  48. Counterfactual explanations of machine learning predictions: opportunities and challenges for AI safety. SafeAI@ AAAI.
  49. Counterfactual explanations and algorithmic recourses for machine learning: A review. arXiv preprint arXiv:2010.10596.
  50. Learning optimal classification trees using a binary linear program formulation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1625–1632.
  51. Born-again tree ensembles. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 9743–9753. PMLR.
  52. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2):841–887.
  53. Discretization for naive-bayes learning: managing discretization bias and variance. Machine Learning, 74(1):39–74.
  54. Exploiting explanations for model inversion attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 682–692.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Veronica Piccialli (14 papers)
  2. Dolores Romero Morales (7 papers)
  3. Cecilia Salvatore (3 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now