Explainable AI using expressive Boolean formulas (2306.03976v1)

Abstract: We propose and implement an interpretable machine learning classification model for Explainable AI (XAI) based on expressive Boolean formulas. Potential applications include credit scoring and diagnosis of medical conditions. The Boolean formula defines a rule with tunable complexity (or interpretability), according to which input data are classified. Such a formula can include any operator that can be applied to one or more Boolean variables, thus providing higher expressivity compared to more rigid rule-based and tree-based approaches. The classifier is trained using native local optimization techniques, efficiently searching the space of feasible formulas. Shallow rules can be determined by fast Integer Linear Programming (ILP) or Quadratic Unconstrained Binary Optimization (QUBO) solvers, potentially powered by special purpose hardware or quantum devices. We combine the expressivity and efficiency of the native local optimizer with the fast operation of these devices by executing non-local moves that optimize over subtrees of the full Boolean formula. We provide extensive numerical benchmarking results featuring several baselines on well-known public datasets. Based on the results, we find that the native local rule classifier is generally competitive with the other classifiers. The addition of non-local moves achieves similar results with fewer iterations, and therefore using specialized or quantum hardware could lead to a speedup by fast proposal of non-local moves.

• The paper introduces an interpretable classification model using expressive Boolean formulas that balance simplicity and predictive performance.
• The methodology employs ILP and QUBO techniques with non-local moves to enhance rule expressiveness and hint at quantum computing applications.
• The model demonstrates competitive benchmarks against classical classifiers while ensuring transparent decision processes for critical applications.

Interpretable Machine Learning with Expressive Boolean Formulas

Introduction

The quest for interpretable machine learning models has become increasingly important, especially in domains where understanding the decision-making process of the model is crucial. This has led to a growing interest in developing methods that not only perform well but are also explainable by design. In light of this, the paper presents an interpretable machine learning classification model based on expressive Boolean formulas. The proposed model aims to balance the complexity and performance of the classifier, providing a rule-based approach to classification that can be easily interpreted.

Method

The core of the proposed approach hinges on the use of expressive Boolean formulas to define classification rules. These rules are designed to be as simple as possible while retaining predictive power. The complexity of these formulas is carefully controlled through a combination of Integer Linear Programming (ILP) and Quadratic Unconstrained Binary Optimization (QUBO) techniques. The model leverages local optimization strategies, enriched by the inclusion of non-local moves to enhance search efficiency and rule expressiveness. These non-local moves are especially significant as they offer a pathway to utilizing quantum computing techniques for optimization, presenting a bridge between current machine learning models and future quantum-enhanced algorithms.

Results

The performance of the proposed model is extensively benchmarked against classical machine learning classifiers on various datasets. Notably, the model demonstrates competitive performance, often matching or closely approaching the benchmarks set by more complex, less interpretable models. This performance is particularly impressive given the inherently interpretable nature of the model, which does not sacrifice clarity for accuracy. Further, the inclusion of non-local move proposals in the optimization process shows potential for reducing the number of iterations needed to achieve optimal or near-optimal rules, suggesting an area where specialized hardware could significantly speed up model training.

Discussion

The research underscores the feasibility of creating an interpretable machine learning model that does not compromise on performance. By utilizing expressive Boolean formulas, the proposed model offers a transparent view into its decision-making process, a critical feature for applications in sensitive fields such as healthcare and finance. Moreover, the exploration of non-local moves opens exciting prospects for integrating quantum computing into machine learning, offering a glimpse into how future advancements in quantum technology could revolutionize model optimization techniques.

Future Work

The paper opens several avenues for future research, including exploring additional datasets, integrating more complex operators into the Boolean formulas, and refining the optimization process to further leverage quantum computing advancements. Additionally, fine-tuning the model to handle larger datasets through sampling or other scalability techniques could extend the applicability and utility of this approach.

Conclusively, this paper presents a significant step forward in the development of interpretable machine learning models. By balancing complexity and interpretability without sacrificing performance, it lays the groundwork for future research at the intersection of machine learning, optimization, and quantum computing.