Interpretable Classification Models for Recidivism Prediction (1503.07810v6)

Abstract: We investigate a long-debated question, which is how to create predictive models of recidivism that are sufficiently accurate, transparent, and interpretable to use for decision-making. This question is complicated as these models are used to support different decisions, from sentencing, to determining release on probation, to allocating preventative social services. Each use case might have an objective other than classification accuracy, such as a desired true positive rate (TPR) or false positive rate (FPR). Each (TPR, FPR) pair is a point on the receiver operator characteristic (ROC) curve. We use popular machine learning methods to create models along the full ROC curve on a wide range of recidivism prediction problems. We show that many methods (SVM, Ridge Regression) produce equally accurate models along the full ROC curve. However, methods that designed for interpretability (CART, C5.0) cannot be tuned to produce models that are accurate and/or interpretable. To handle this shortcoming, we use a new method known as SLIM (Supersparse Linear Integer Models) to produce accurate, transparent, and interpretable models along the full ROC curve. These models can be used for decision-making for many different use cases, since they are just as accurate as the most powerful black-box machine learning models, but completely transparent, and highly interpretable.

• The paper demonstrates that SLIM scoring systems rival black-box models in accuracy while providing clear, interpretable decision frameworks.
• It rigorously compares methods across full ROC curves, highlighting the trade-offs between traditional techniques and interpretable approaches.
• The study shows that transparent models can improve high-stakes decisions in criminal justice by aligning predictive precision with ethical standards.

Overview of Interpretable Classification Models for Recidivism Prediction

The paper "Interpretable Classification Models for Recidivism Prediction" by Zeng, Ustun, and Rudin investigates a crucial problem in criminology: how to construct predictive models of recidivism that maintain a balance between accuracy, transparency, and interpretability. Recidivism prediction models are used in various decision-making processes, from sentencing to probation releases and implementing preventive social services. These decisions often require considerations beyond mere classification accuracy, such as maintaining specified true positive and false positive rates.

To represent the trade-offs between different predictive models, the authors examine the full ROC curve using a wide range of recidivism prediction tasks. Notably, they demonstrate that several machine learning methods, including SVMs and stochastic gradient boosting, yield comparably accurate models along the entire ROC curve. However, techniques designed to enhance interpretability, like CART and C5.0, often fall short in accuracy or fail to be interpretable. Addressing this issue, the authors employ Supersparse Linear Integer Models (SLIM) to create scoring systems that excel along the ROC curve, combining high accuracy with transparency and interpretability.

Numerical and Methodological Insights

The paper conducts experiments showing that SLIM scoring systems rival the most accurate black-box models in different applications. The models achieve this by optimizing both accuracy and sparsity, offering interpretable scoring systems with integer coefficients, thereby facilitating understanding and application. In the context of recidivism prediction, they argue that simplifying models without compromising accuracy can significantly impact resource allocation and decision-making.

The empirical results reveal that while traditional methods like random forests produce precise black-box models, they lack transparency, making them less defensible and usable for certain applications. The authors provide evidence through rigorous experimentation across several datasets, demonstrating the ability of SLIM models to produce competitive ROC curves. The SLIM models achieve areas under the curve (AUC) comparable to more complex methods, even when constrained to a sparse model structure.

Implications and Future Directions

The theoretical contributions of this work underline the importance of integrating interpretability into predictive models deployed in high-stakes environments, such as criminal justice. Practically, the implementation of interpretable models, like SLIM, provides stakeholders with insights into the model's decision processes, making them actionable and align with ethical standards and legal requirements.

The paper invites future research to explore the intersection of interpretability and other domains that deploy predictive analytics for decision-making. One area of interest is extending the method to incorporate fairness and bias mitigation explicitly. As AI continues to integrate into societal infrastructures, embedding transparency and interpretability will be critical to fostering trust and accountability.

The computational demands of solving integer programs for SLIM can be considered a limitation, albeit one that is becoming increasingly manageable given advancements in computational optimization. Future advancements might focus on refining the SLIM method to accommodate even larger and more complex datasets, common in contemporary machine learning applications.

In summary, this research contributes significantly to the field by offering a methodological framework to develop models that strike a critical balance between interpretability and accuracy in recidivism prediction, a balance necessary for ethical and effective decision-making in criminal justice systems.