KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactions (2405.10852v2)

Abstract: The Shapley value (SV) is a prevalent approach of allocating credit to ML entities to understand black box ML models. Enriching such interpretations with higher-order interactions is inevitable for complex systems, where the Shapley Interaction Index (SII) is a direct axiomatic extension of the SV. While it is well-known that the SV yields an optimal approximation of any game via a weighted least square (WLS) objective, an extension of this result to SII has been a long-standing open problem, which even led to the proposal of an alternative index. In this work, we characterize higher-order SII as a solution to a WLS problem, which constructs an optimal approximation via SII and $k$-Shapley values ($k$-SII). We prove this representation for the SV and pairwise SII and give empirically validated conjectures for higher orders. As a result, we propose KernelSHAP-IQ, a direct extension of KernelSHAP for SII, and demonstrate state-of-the-art performance for feature interactions.

• The paper introduces a novel weighted least square optimization formulation to compute Shapley interactions efficiently.
• It adapts the KernelSHAP method to capture complex feature interactions, validated by experiments in sentiment analysis and regression tasks.
• This approach provides faster and more accurate model interpretability, promising broader applications in real-world AI systems.

Understanding KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions

Introduction

If you've been working with machine learning models, you've probably heard about the Shapley value (SV) for interpreting model outputs. Whether it's feature attribution, feature importance, or data valuation, SV is a versatile tool for understanding how different entities (like features) contribute to a model's prediction. But what if you need to understand how combinations of features work together? That's where Shapley Interaction Index (SII) steps in.

The paper "KernelSHAP-IQ: Weighted Least Square Optimization for Shapley Interactions" introduces an extension of KernelSHAP, a popular tool for SV, to handle higher-order feature interactions through SII. Let's break down the key concepts and findings from this paper and why it's relevant for data scientists.

Shapley Values and Interactions

The Basics of Shapley Values

Shapley values distribute the payout (like a model's prediction) among different players (features) fairly, based on their contributions. The SV for a feature $i$ is calculated as the weighted average of its marginal contributions across all possible subsets of features.

Introducing Shapley Interaction Index (SII)

While the SV helps to understand the individual feature contributions, it may fall short for complex real-world problems where interactions between features are crucial. For instance, in sentiment analysis, words like "never" and "forget" might negate each other when they appear together, but this interaction won't be captured if we only consider their individual contributions.

The SII extends SV to account for these interactions. Essentially, the SII allows us to understand how groups of features work together to impact the model's output.

The Core Contributions of KernelSHAP-IQ

Traditional methods for calculating Shapley interactions are computationally prohibitive, requiring exponential time in the number of features. This paper's primary aim is to reduce this complexity by linking SII with a Weighted Least Square (WLS) optimization problem, similar to how SV is computed in KernelSHAP.

Optimal Approximations via Shapley Interactions

The paper shows that we can represent higher-order interactions (SII) as a solution to a WLS problem. Here's a simplified view:

• For pairwise interactions (order 2), the SII can be constructed iteratively, building on lower-order Shapley values.
• The proposed method introduces KernelSHAP-IQ, which uses this iterative approach to compute exact or approximate SII values efficiently.

Performance and Validation

KernelSHAP-IQ has been empirically demonstrated to outperform several existing methods in both approximation quality and efficiency across various datasets and machine learning models.

Practical Insights and Future Directions

Empirical Results

KernelSHAP-IQ shows state-of-the-art performance in capturing feature interactions. For example, in a sentiment analysis task, it correctly identified the critical interaction between "never" and "forget," which contributes significantly to the positive sentiment of a movie review.

Use Cases

1. Sentiment Analysis: As shown in the paper, understanding how word combinations impact sentiment can refine feature attributions and improve interpretability.
2. Regression Tasks: For predicting housing prices, it can reveal how combinations of geographical features (latitude and longitude) precisely dictate the model's prediction.

Looking Forward

Given its efficiency and effectiveness, KernelSHAP-IQ can be used for a variety of tasks in feature interaction analysis. Future research could further streamline this approach or even extend it to other types of model explanations. Additionally, integrating KernelSHAP-IQ with real-time systems might open up new avenues for dynamic model interpretability.

Conclusion

KernelSHAP-IQ represents a significant step in the field of model interpretability, particularly for capturing feature interactions. By extending the well-known KernelSHAP method to handle higher-order interactions efficiently, this paper provides a powerful tool for data scientists looking to explore how their models make decisions. Whether you're working with text, images, or tabular data, understanding these interactions can lead to more robust and explainable artificial intelligence systems.