T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients (2404.16495v2)

Abstract: The development of machine learning applications has increased significantly in recent years, motivated by the remarkable ability of learning-powered systems to discover and generalize intricate patterns hidden in massive datasets. Modern learning models, while powerful, often have a level of complexity that renders them opaque black boxes, resulting in a notable lack of transparency that hinders our ability to decipher their reasoning. Opacity challenges the interpretability and practical application of machine learning, especially in critical domains where understanding the underlying reasons is essential for informed decision-making. Explainable Artificial Intelligence (XAI) rises to address that challenge, unraveling the complexity of black boxes by providing elucidating explanations. Among the various XAI approaches, feature attribution/importance stands out for its capacity to delineate the significance of input features in the prediction process. However, most existing attribution methods have limitations, such as instability, when divergent explanations may result from similar or even the same instance. This work introduces T-Explainer, a novel local additive attribution explainer based on Taylor expansion. It has desirable properties, such as local accuracy and consistency, making T-Explainer stable over multiple runs. We demonstrate T-Explainer's effectiveness in quantitative benchmark experiments against well-known attribution methods. Additionally, we provide several tools to evaluate and visualize explanations, turning T-Explainer into a comprehensive XAI framework.

Introducing T-Explainer: A Deterministic Local Additive Attribution Method for XAI

Overview

T-Explainer is a novel method introduced to enhance the interpretability of black-box models using a local additive attribution based on Taylor expansion. In contrast to existing methods like SHAP and LIME, T-Explainer is built on a deterministic approach which ensures stability and consistency in its explanations. This method estimates the importance of input features by approximating the gradient of the model at a given data point using centered finite differences, which calculates the derivative by evaluating the function at points around the input.

Key Contributions

1. Stable and Consistent Explanations: By leveraging Taylor expansion, T-Explainer provides a dependable mathematical framework for attribution, ensuring that results are both stable over multiple tests and consistent for similar or identical inputs.
2. Benchmarking with Established Methods: T-Explainer's effectiveness is demonstrated through quantitative comparisons against prevalent methods like SHAP and LIME, showcasing superior stability and reliability in attribution.
3. Framework Integration and Toolkit: The implementation of T-Explainer includes integration with the SHAP library for visualization tools, enhancing its practical application. It is accompanied by a Python package providing a robust suite for deploying the T-Explainer in various settings.

Methodology

The T-Explainer utilizes local, model-agnostic feature-importance attributions, approximating the function of the model near a query point using a first-order Taylor expansion. The method formally defines an optimization procedure for this approximation and addresses the calculation of gradients through centered finite differences. This strategic formulation captures the impact of slight perturbations in the input on the output, which inherently facilitates an understanding of feature attributions.

Evaluations and Results

T-Explainer was extensively evaluated against well-known attribution methods across various datasets, ranging from synthetic to real-world scenarios. Specifically, its stability was benchmarked via metrics such as Relative Input Stability (RIS) and Relative Output Stability (ROS), where T-Explainer frequently outperformed other methods. Additionally, its computational efficiency was demonstrated to be comparable to KernelSHAP, which is notable given the typically high resource demands of exact Shapley value computations.

Practical Implications

The deterministic nature of T-Explainer not only enhances the trustworthiness of the interpretations provided but also makes it suitable for critical applications where reliable and repeatable explanations are necessary. The integration with existing tools and the provision of a dedicated Python library facilitate its adoption in diverse AI projects and pipelines, promising broad usability across sectors where AI models need to be demystified.

Future Directions

Future development of T-Explainer includes extending its capabilities to multi-class classification problems and regression models, refining the optimization of the perturbation parameter h, and enhancing support for categorical data without needing retraining models. The ongoing enhancements aim to broaden the applicability of T-Explainer and cement its utility in providing transparent AI solutions.

Conclusion

T-Explainer represents a significant advancement in the toolkit available for XAI, offering a methodologically sound, stable, and consistent approach to understanding model decisions. Its introduction is timely, given the increasing complexity of AI models and the corresponding need for transparency in their decision-making processes.