SHAP: A Unified Framework for ML Interpretability

• SHAP is a unified framework that explains machine learning predictions by attributing additive feature importance values based on cooperative game theory.
• It employs efficient approximations such as Kernel SHAP and Deep SHAP to handle computational challenges in estimating feature contributions in complex models.
• SHAP enhances interpretability by unifying previous methods and enabling practical applications in fields like healthcare, finance, and image processing.

SHapley Additive exPlanations (SHAP) is a unified framework for interpreting predictions of complex machine learning models by attributing additive feature importance values based on rigorous axiomatic foundations. SHAP leverages concepts from cooperative game theory to produce explanations that decompose model outputs into contributions from each feature, providing consistent, transparent, and theoretically optimal local attributions across a variety of model classes and applications.

1. Additive Feature Attribution and Theoretical Uniqueness

SHAP introduces additive feature attribution methods: explanation models of the form

$g(\mathbf{z}') = \phi_0 + \sum_{i=1}^M \phi_i z'_i$

where $\mathbf{z}' \in \{0, 1\}^M$ denotes the vector of simplified (present/missing) features, $\phi_0$ is a baseline, and $\phi_i$ quantifies the contribution of feature $i$. The mapping $h_x$ relates the simplified input $\mathbf{z}'$ to the original input $\mathbf{x}$. The SHAP framework identifies a unique solution in this class by enforcing three properties:

• Local accuracy: $g(\mathbf{x}') = f(\mathbf{x})$ for the explained input,
• Missingness: if $z'_i = 0$ then $\phi_i = 0$,
• Consistency: if a model change increases the marginal contribution of feature $i$, $\phi_i$ should not decrease, as proven in Theorem 1. This yields the Shapley value—a uniquely justified additive attribution:

$$\phi_i(f, x) = \sum_{S \subseteq F \setminus \{i\}} \frac{|S|! (M - |S| - 1)!}{M!} \left[ f_{x}(S \cup \{i\}) - f_{x}(S) \right]$$

with $f_x(S)$ representing the expected model output conditioned on features in $S$. Only Shapley values satisfy all three requirements (Lundberg et al., 2017).

2. Unification of Interpretability Methods

SHAP formalizes a broad family of feature attribution interpretability tools, unifying six previously disparate approaches:

• LIME,
• DeepLIFT,
• Layer-Wise Relevance Propagation,
• Shapley Regression/Sampling values,
• Quantitative Input Influence,

demonstrating that all can be interpreted as additive models with varying consistency, local accuracy, or missingness compliance. SHAP thus provides a comparative and organizing framework for both existing and future explanation methodologies, highlighting when certain techniques lack theoretical guarantees (Lundberg et al., 2017).

3. Algorithmic Approximations and Model Adaptations

Computing exact Shapley values is computationally infeasible for large $M$ ($2^M$ subsets). SHAP introduces two primary algorithmic instantiations:

• Kernel SHAP: A model-agnostic approximation formulating the attribution problem as a weighted linear regression, using the Shapley kernel $$\pi_{x'}(z') = (M - 1)/ \left[{M \choose |z'|}|z'|(M-|z'|)\right]$$. This dramatically reduces model evaluations via importance-weighted sampling (Lundberg et al., 2017).
• Deep SHAP: Tailored for deep networks, this method utilizes DeepLIFT-like modular compositionality and backpropagation to aggregate local Shapley estimates per network component, especially improving treatment of non-linearities such as max-pooling (Lundberg et al., 2017).

For linear models, the SHAP values are

$\phi_0 = b, \quad \phi_i = w_i(x_i - E[x_i]).$

4. Interpretability in Practice: Domains, Workflows, and Extensions

SHAP is applied broadly: in healthcare (identifying relevant biomarkers), finance (credit risk attribution), image processing (saliency in MNIST digits), general machine learning, and allocation problems. SHAP outputs permit tracing decision rationales in black-box models at the instance level, attributing “credit” to features according to their marginal effect on the model output. Further, SHAP's additive decompositions enable practitioners to aggregate, visualize, and statistically analyze feature impacts across samples.

Extensions have generalized the framework to more complex output functions:

• Generalized SHAP (G-SHAP): Allows explanations for arbitrary targets, such as inter-group differences or comparative classification, using arbitrary output functions $g(f, X, \Omega)$ in place of $f(x)$ (Bowen et al., 2020).
• Counterfactual SHAP (CF-SHAP): Integrates counterfactual background distributions for actionable recourse, yielding “derived trends” that point to the direction in which an individual could act to flip a model decision (Albini et al., 2021).
• Causal SHAP: Incorporates causality via learned structural graphs, discounting features that are merely correlated with the target but not causally relevant (Ng et al., 31 Aug 2025).
• Model Structure-Aware and Efficient SHAP: Algorithms enabling exact or polynomial-time computation of SHAP values by exploiting model decomposition, known interaction order, or iterative convergence (Hu et al., 2023).

5. Statistical, Algorithmic, and Validation Considerations

A range of practical and statistical issues affect the reliability and deployment of SHAP:

• Background Set Selection: For deep-learning models, the size and representativeness of the “background” dataset strongly influence SHAP value stability and variable ranking reliability. Larger, representative backgrounds are required for consistent results; ranking of most and least important variables is more robust than for those of middling importance (Yuan et al., 2022).
• Statistical Validity and Automated Interpretation: Superficial or subjective reporting of SHAP features is common; statistical validation, including significance assessment and formal interaction analysis, improves robustness and reproducibility of interpretations—as exemplified by CLE-SH (Lee et al., 19 Sep 2024).
• Causal Inference and Attribution: Standard SHAP can misattribute importance to confounded or merely correlated features. Integrating causal discovery algorithms (e.g., PC, IDA) leads to refined “causal SHAP” attributions, especially critical for high-stakes domains (Ng et al., 31 Aug 2025).
• Global versus Local Explanations: While SHAP is intrinsically local, tools like Shapley variable importance clouds aggregate SHAP attributions across the Rashomon set (ensemble of nearly optimal models) to provide more reliable uncertainty quantification and robust assessment of variable importance (Ning et al., 2021).

6. Impact, Limitations, and Future Directions

SHAP has established itself as a de facto standard for post-hoc model explanations, with broad utility in research and industry. Its adoption is driven by:

• Theoretical guarantees (local accuracy, consistency),
• Flexibility across model types and application domains,
• Algorithmic advances supporting both efficiency and practical deployment,
• Compatibility with domain-specific extensions (e.g., cyclic-spectral SHAP for cyclostationary signals (Chen et al., 10 Feb 2025), and interpretable surrogate models for instant Shapley computation via InstaSHAP (Enouen et al., 20 Feb 2025)).

Limitations relate to computational scaling for high-dimensional and highly interactive models, the interpretability of SHAP attributions in the presence of collinearity or complex interactions, and the challenge of distinguishing causation from correlation. Ongoing research focuses on computationally efficient variants (conditional expectation networks (Richman et al., 2023)), statistically valid explanations, hybridization with LLMs for accessibility (Zeng, 24 Aug 2024), and domain-specific adaptations.

SHAP remains a central, extensible tool for interpretable machine learning, combining an axiomatic foundation with practical methods to address the growing demand for transparent model decision-making in increasingly complex domains.