Counterfactual Explanations of Black-box Machine Learning Models using Causal Discovery with Applications to Credit Rating (2402.02678v2)

Abstract: Explainable artificial intelligence (XAI) has helped elucidate the internal mechanisms of machine learning algorithms, bolstering their reliability by demonstrating the basis of their predictions. Several XAI models consider causal relationships to explain models by examining the input-output relationships of prediction models and the dependencies between features. The majority of these models have been based their explanations on counterfactual probabilities, assuming that the causal graph is known. However, this assumption complicates the application of such models to real data, given that the causal relationships between features are unknown in most cases. Thus, this study proposed a novel XAI framework that relaxed the constraint that the causal graph is known. This framework leveraged counterfactual probabilities and additional prior information on causal structure, facilitating the integration of a causal graph estimated through causal discovery methods and a black-box classification model. Furthermore, explanatory scores were estimated based on counterfactual probabilities. Numerical experiments conducted employing artificial data confirmed the possibility of estimating the explanatory score more accurately than in the absence of a causal graph. Finally, as an application to real data, we constructed a classification model of credit ratings assigned by Shiga Bank, Shiga prefecture, Japan. We demonstrated the effectiveness of the proposed method in cases where the causal graph is unknown.

Introduction to Counterfactual Explanations in XAI

Explainable Artificial Intelligence (XAI) represents an increasingly critical branch of AI research concerned with making machine learning models interpretable to humans. A growing body of research is focused on counterfactual explanations, which clarify model predictions by examining what might trigger a different outcome from a model if the input were altered in a specific way. This paper presents an innovative framework for generating counterfactual explanations using causal inference, even when the causal structure underpinning the model is not fully known.

Causal Structures in Explainable AI

The novel contribution of this work lies in relaxing the requirement of a known causal graph for counterfactual estimation. Instead, the paper proposes using causal discovery methods augmented with prior information about the causal structure. This approach contrasts with previous methods that presume complete understanding of the causal graph, which is rarely available in real-world settings.

Causal discovery techniques, paired with prior knowledge of causal relations, form the basis for the proposed XAI framework. The paper argues for two types of prior information that can be utilized: the assumption that a target variable is a "sink" variable, meaning it does not cause any other variable, and that all explanatory variables have a direct causal relationship to the target variable.

Evaluating the Model with Numerical Experiments

The framework's robustness is tested through numerical experiments with artificial data, showing its capability to accurately estimate explanatory scores better than previous methods without using causal graphs. For example, when examining a linear causal structure with error variables having a uniform distribution, DirectLiNGAM along with prior information resulted in lower mean absolute error (MAE) and higher Spearman's rank correlation coefficient (SPR) compared to situations where no graph was assumed.

Real-World Application to Credit Rating

Another strong numerical result showcased is the application of the proposed framework to credit rating data from Shiga Bank, Japan. This case underscores that useful counterfactual explanations can be derived even when the causal structure is unknown. By integrating causal discovery with the proposed method, researchers identified that key variables like 'industry type' and 'capital stock' could be critical in altering a credit rating when their values are changed.

Conclusion

This research represents a meaningful advancement in XAI, particularly in its capacity to leverage causal discovery for generating counterfactual explanations without a complete knowledge of underlying causal structures. The work not only provides a theoretically sound framework but also demonstrates practical effectiveness through real-world financial data, suggesting a wide applicability for this XAI method in various industries where accountability and explainability are paramount.