SHAP: Additive Feature Explanations

• SHAP is a unified, theoretically grounded framework that assigns additive feature attributions based on Shapley values and game theory principles.
• The framework implements efficient methods like Kernel SHAP and Deep SHAP to reduce computational overhead while ensuring accurate, model-agnostic explanations.
• SHAP's axiomatic properties—local accuracy, missingness, and consistency—provide transparent and reliable insights, widely applied in finance, healthcare, and complex model analysis.

Shapley Additive Explanations (SHAP) are a unified, theoretically grounded framework for attributing the output of any predictive model to its input features. SHAP assigns each feature an importance value for an individual prediction, providing clarity and consistency in the interpretation of complex models (including ensembles and deep networks) by leveraging concepts from cooperative game theory. The SHAP framework is distinguished by its identification of a unique class of additive feature importance measures and a rigorous axiomatic foundation, offering a systematic approach that unifies multiple existing interpretability methods (Lundberg et al., 2017).

1. SHAP Framework and Theoretical Foundations

SHAP defines an explanation model as an additive feature attribution mechanism:

$g(z') = \phi_0 + \sum_{i=1}^M \phi_i z'_i$

where $z'$ is a binary vector indicating the presence or absence of each input variable, and $\phi_i$ quantifies the contribution of feature $i$.

The framework is characterized by three fundamental properties:

• Local Accuracy (Efficiency): The sum of the attributions equals the model's prediction at the instance, i.e., $f(x) = g(x') = \phi_0 + \sum_{i} \phi_i x'_i$.
• Missingness: Features absent from the input receive a zero contribution ($x'_i = 0 \implies \phi_i = 0$).
• Consistency: If, for all subsets, a change in the model increases the marginal effect of a feature, the assigned attribution should not decrease.

A central theoretical result (Theorem 1) proves that, within this additive framework and under these properties, there is a unique solution: the Shapley values. For feature $i$, the SHAP value is:

$$\phi_i(f, x) = \sum_{S \subseteq F \setminus \{i\}} \frac{ |S|! (M - |S| - 1)! }{ M! } [f_x(S \cup \{i\}) - f_x(S) ]$$

where $f_x(S)$ denotes the expected model output when only features in $S$ are observed. This solution directly connects game-theoretic fairness to model interpretability, providing additive allocation of predictive "credit" for each feature (Lundberg et al., 2017).

2. Connection with Other Interpretability Methods

The SHAP framework recasts several model explanation paradigms as special cases of the additive feature attribution class:

• LIME: Employs local surrogate models; however, heuristic choice of kernels and loss functions may violate local accuracy or consistency.
• DeepLIFT/Layer-Wise Relevance Propagation: Apply backpropagation-based techniques for deep models but do not guarantee the full set of SHAP properties.
• Classical Shapley Regression/Sampling: Adheres to fairness axioms but is computationally prohibitive for large-scale models.

The development of the Shapley kernel eliminates the pitfalls associated with ad-hoc parameterizations in other frameworks. By matching the correct kernel, loss function, and removing regularization ($\Omega(g) = 0$), SHAP is the only approach satisfying local accuracy, missingness, and consistency simultaneously. Thus, Kernel SHAP and Deep SHAP become the recommended, theoretically justified instantiations for model-agnostic and deep learning settings, respectively (Lundberg et al., 2017).

3. Efficient SHAP Computation: Kernel SHAP and Deep SHAP

Computing exact Shapley values is exponentially complex in the number of features. To address this, SHAP introduces two key scalable algorithms:

• Kernel SHAP: Reformulates SHAP as a weighted linear regression using the Shapley kernel,

$$\pi_{x'}(z') = \frac{M-1}{ \binom{M}{|z'|} |z'| (M - |z'|) }$$

and assigns infinite weight to the all-features-present and all-features-missing cases. This approach reduces the number of required model evaluations and is entirely model-agnostic.

• Deep SHAP: For compositional/deep architectures, Deep SHAP combines DeepLIFT’s backpropagation rules with the theoretical principles of SHAP, “backpropagating” SHAP values layer-wise to efficiently compute attributions through complex networks.

Empirical results indicate that SHAP methods not only deliver explanations that better align with human intuition but also require fewer model evaluations than previous Shapley sampling schemes (Lundberg et al., 2017).

4. SHAP Properties, Formulas, and Key Mathematical Results

The SHAP framework is underpinned by several formal results and explicit formulas:

Property	Mathematical Statement
Additive Model	$g(z') = \phi_0 + \sum_{i=1}^M \phi_i z'_i$
Local Accuracy	$f(x) = g(x') = \phi_0 + \sum_i \phi_i x'_i$
Missingness	$x'_i = 0 \implies \phi_i = 0$
Consistency	If feature effect increases in the model, so must $\phi_i$
Shapley Value (unique solution)	$$\phi_i(f, x) = \sum_{S \subseteq F \setminus \{i\}} \frac{|S|! (M - |S| - 1)!}{M!} [f_x(S \cup \{i\}) - f_x(S)]$$
Shapley Kernel	$$\pi_{x'}(z') = \frac{M-1}{ \binom{M}{|z'|} |z'| (M - |z'|) }$$

A loss function for weighted regression is used in Kernel SHAP:

$$L(f, g, \pi_{x'}) = \sum_{z' \in Z} [f(h_x(z')) - g(z')]^2 \pi_{x'}(z')$$

with no regularization ($\Omega(g) = 0$) (Lundberg et al., 2017).

5. Practical Implications and Real-World Applications

SHAP is now widely adopted for local and global interpretability in diverse domains:

• Finance and Healthcare: SHAP values are used to justify individual risk scores and clinical decisions, ensuring both transparency and auditability.
• User Studies: Explanations produced by SHAP correlate more strongly with human-judged feature importance than alternatives.
• Complex Model Analysis: SHAP exposes which features drive predictions in ensemble methods and deep networks, making them less "black box."
• Algorithm Selection: The SHAP framework advises practitioners on the theoretical trade-offs among available model-agnostic interpretation methods and highlights when to use specific variants (Kernel SHAP for arbitrary models, Deep SHAP for deep nets).

Furthermore, SHAP's explicit axiomatic foundation avoids the risk of inconsistent or misleading feature attributions and provides a clear solution to the feature importance allocation problem.

6. Limitations and Directions for Future Work

While SHAP offers a unifying solution for additive feature attribution, the exponential cost of exact value computation in high dimensions persists, motivating continued research on scalable approximations. Additionally, while theoretical guarantees are strong in the context of additive explanations, extending these guarantees to more general forms of model behavior (e.g., higher-order interactions beyond additive effects) remains an active area of research.

Recent research also investigates the stability of SHAP explanations with different choices of background samples and the implications of background dataset size on the robustness of interpretability outcomes, raising important practical considerations (Yuan et al., 2022).

7. Conclusion

SHAP provides a rigorous, unified, and axiomatic approach to assigning feature attributions in model predictions. By subsuming and improving prior interpretability methods, introducing efficient algorithms (Kernel SHAP and Deep SHAP), and enforcing key fairness and consistency properties, SHAP has established a new standard for local and global explanations in machine learning. Its theoretical guarantees, performance, and broad applicability have led to widespread adoption and ongoing methodological extensions in the field of explainable AI (Lundberg et al., 2017).