Probabilistic Scoring Lists for Interpretable Machine Learning (2407.21535v1)
Abstract: A scoring system is a simple decision model that checks a set of features, adds a certain number of points to a total score for each feature that is satisfied, and finally makes a decision by comparing the total score to a threshold. Scoring systems have a long history of active use in safety-critical domains such as healthcare and justice, where they provide guidance for making objective and accurate decisions. Given their genuine interpretability, the idea of learning scoring systems from data is obviously appealing from the perspective of explainable AI. In this paper, we propose a practically motivated extension of scoring systems called probabilistic scoring lists (PSL), as well as a method for learning PSLs from data. Instead of making a deterministic decision, a PSL represents uncertainty in the form of probability distributions, or, more generally, probability intervals. Moreover, in the spirit of decision lists, a PSL evaluates features one by one and stops as soon as a decision can be made with enough confidence. To evaluate our approach, we conduct a case study in the medical domain.
- \bibcommenthead
- Accuracy of symptoms and signs for coronary heart disease assessed in primary care. British Journal of General Practice 60(575): 246–257 .
- Brier, G.W. 1950. Verification of forecasts expressed in terms of probability. Monthly Weather Review 78(1): 1–3 .
- Rounding methods for discrete linear classification. In Proc. ICML, International Conference on Machine Learning, pp. 651–659.
- Interpretable cascade classifiers with abstention. In Proc. AISTATS, 22nd Int. Conf. on Artificial Intelligence and Statistics.
- The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4): 404–413 .
- Classifier calibration: a survey on how to assess and improve predicted class probabilities. Machine Learning 112(9): 3211–3260 .
- Fawcett, T. 2006. An introduction to ROC analysis. Pattern Recognit. Lett. 27(8): 861–874 .
- Fayyad, U.M. and K.B. Irani 1993. Multi-interval discretization of continuous-valued attributes for classification learning. In International Joint Conference on Artificial Intelligence.
- The limits of distribution-free conditional predictive inference. Information and Inference 10(2): 455–482 .
- Fürnkranz, J. 1999. Separate-and-conquer rule learning. Artificial Intelligence Review 13(1): 3–54 .
- Foundations of Rule Learning. Cognitive Technologies. Heidelberg, Germany: Springer.
- Probabilistic scoring lists for interpretable machine learning. In Discovery Science, Porto, Portugal, pp. 189–203. Springer Nature Switzerland.
- Hastie, T.J. 2017. Generalized Additive Models. New York, USA: Routledge.
- Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning 110(3): 457–506 .
- Bipartite ranking through minimization of univariate loss. In Proc. ICML 2011, 28th Int. Conf. on Machine Learning, Washington, USA.
- Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers. In Proc. AISTATS, 20th International Conference on Artificial Intelligence and Statistics, Volume 54 of PMLR, pp. 623–631.
- Weighing harm in therapeutic decisions of smear-negative pulmonary tuberculosis. Medical Decision Making 29: 380 – 390 .
- Extreme value correction: A method for correcting optimistic estimations in rule learning. Machine Learning 108(2): 297–329 .
- Predicting good probabilities with supervised learning. In Proc. ICML, 22nd International Conference on Machine Learning, New York, USA, pp. 625–632.
- Breast Cancer Coimbra. UCI Machine Learning Repository.
- Tree induction for probability-based ranking. Machine Learning 52(3): 199–215 .
- ILPD (Indian Liver Patient Dataset). UCI Machine Learning Repository.
- Rivest, R.L. 1987. Learning decision lists. Machine Learning 2: 229–246 .
- Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty. Inf. Sci. 255: 16–29 .
- On Learning Decision Heuristics. In Imperfect Decision Makers: Admitting Real-World Rationality, pp. 75–85.
- Chest pain in the emergency room: value of the heart score. Netherlands Heart Journal 16(6): 191–196 .
- Developing a clinical prediction score: Comparing prediction accuracy of integer scores to statistical regression models. Anesthesia and Analgesia 132(6): 1603–1613 .
- An empirical comparison of probability estimation techniques for probabilistic rules. In Proceedings of the 12th International Conference on Discovery Science (DS-09), Porto, Portugal, pp. 317–331. Springer-Verlag.
- A regret theory approach to decision curve analysis: a novel method for eliciting decision makers’ preferences and decision-making. BMC Medical Informatics and Decision Making 10: 51 .
- Supersparse linear integer models for optimized medical scoring systems. Machine Learning 102(3): 349–391 .
- Optimized risk scores. In Proc. 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1125–1134.
- Learning optimized risk scores. Journal of Machine Learning Research 20(150): 1–75 .
- In pursuit of interpretable, fair and accurate machine learning for criminal recidivism prediction. Journal of Quantitative Criminology 39(2): 519–581 .
- Webb, G.I. 1994. Recent progress in learning decision lists by prepending inferred rules. In Proceedings of the 2nd Singapore International Conference on Intelligent Systems, pp. B280–B285.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.