SHAP Analysis: Feature Attribution in ML

• SHAP Analysis is a framework that decomposes complex model predictions into feature contributions using Shapley values with guarantees of local accuracy, missingness, and consistency.
• It employs scalable algorithms such as KernelSHAP and TreeSHAP to compute either exact or approximate attributions across various models including ensembles and deep neural networks.
• SHAP is applied in diverse fields like materials science and industrial systems, enhancing interpretability and providing actionable insights for model validation and decision support.

SHAP (SHapley Additive exPlanations) is a principled framework for decomposing the prediction of a machine learning model into contributions from individual input features, grounded in the axioms of cooperative game theory. It provides both local and global interpretability for complex black-box models, such as ensemble methods and deep neural networks, by assigning a consistent, fair allocation of the predicted output to each feature. SHAP is characterized by its strong theoretical guarantees—efficiency (local accuracy), missingness, and consistency—and by a suite of scalable algorithms and extensions to accommodate a broad range of modeling scenarios.

1. Theoretical Foundations and Mathematical Definition

At its core, SHAP interprets the prediction task as a cooperative game, with input features as players and the model output as the "payout." The goal is to attribute the change in the prediction for a particular instance $x$ to each feature $i$ via the Shapley value:

$$\phi_i = \sum_{S \subseteq F \setminus \{i\}} \frac{|S|!\, (|F| - |S| - 1)!}{|F|!} \left[ f_{S \cup \{i\}}(x_{S \cup \{i\}}) - f_S(x_S) \right]$$

where $F$ is the feature set, $S$ is a subset of features not containing $i$, $f_S(x_S)$ is the model's output when only features in $S$ are known, and the rest are marginalized, integrated, or imputed. The result is an additive explanation:

$f(x) \approx \phi_0 + \sum_{i=1}^{|F|} \phi_i$

with $\phi_0$ as the expected model prediction on a background dataset. SHAP uniquely satisfies:

• Local Accuracy: $\sum_i \phi_i + \phi_0 = f(x)$.
• Missingness: If feature $i$ is missing, $\phi_i=0$.
• Consistency: If a model change increases feature $i$'s marginal contribution, $\phi_i$ cannot decrease (Lundberg et al., 2017).

2. Computational Algorithms: Exact and Approximate SHAP

The combinatorial nature of the Shapley value necessitates algorithmic strategies for practical computation:

• KernelSHAP: A model-agnostic weighted linear regression, with weights derived from the Shapley kernel. This enables estimation from $O(|F|)$ sampled coalitions but has exponential cost if all subsets are evaluated (Lundberg et al., 2017).
• TreeSHAP: Leverages tree ensemble structure to compute exact SHAP values in $O(T L^2)$ time, where $T$ is the number of trees and $L$ the number of leaves. It recursively accumulates weights over tree paths to aggregate marginal contributions efficiently (Lundberg et al., 2017, Yang, 2021).
• Fast TreeSHAP V1/V2: Systematic rearrangements and caching in TreeSHAP to achieve $1.5\times$ to $3\times$ speedups for large datasets or multiple explanation passes, with polynomial space and time complexity (Yang, 2021).
• Efficient Structure-Aware SHAP: For models with bounded interaction order $K$, exact SHAP can be computed in $O(p^{\lfloor K/2 \rfloor})$ or faster via additive decomposition, exploiting functional ANOVA structure (Hu et al., 2023, Herren et al., 2022).

3. Statistical and Algorithmic Issues in SHAP Estimation

Estimation of SHAP values is affected by choice of baseline distribution, high-order interactions, and computational constraints:

• Baseline Dependence: The conditional expectation $E[f(X) \mid X_S=x_S]$ can be computed using global marginal, global joint, local, or empirical data-driven distributions. Each presents trade-offs in bias and variance; results are sensitive to misspecification (Herren et al., 2022).
• ANOVA Decomposition: SHAP decomposes the model function into main effects and interactions; for a function $f(x) = \sum_{u \subseteq F} f_u(x_u)$, SHAP equally divides each $j$-order interaction among $j$ members (Herren et al., 2022).
• Sampling and Regression: Classical SHAP estimation employs paired-block sampling and constrained least squares over observed coalitions to recover feature attributions within computational budgets (Herren et al., 2022).

4. Extensions and Generalizations: Interactions, Model Classes, and Beyond

SHAP has been extended to address several core limitations and application scenarios:

• Feature Interactions: Classical SHAP provides only main effects, potentially misleading when high-order interactions dominate. NSHAP computes Shapley-style values for all exponent $2^d$ subsets, but is intractable and non-interpretable. Succinct interaction-aware explanations (iSHAP) partition features into interacting blocks, optimizing a fidelity-complexity objective over graph-constrained partitions, achieving high accuracy and interpretability without exponential blowup (Xu et al., 2024).
• Generalized SHAP (G-SHAP): Extends explanations to arbitrary functions $g$ of model outputs, enabling attribution for contrasts, intergroup differences, and performance metrics (e.g., loss functions), not just raw predictions (Bowen et al., 2020).
• Multiplicative Models (mSHAP): For two-part models ($z = f(x) g(x)$), mSHAP expands the product of SHAP decompositions, allocates cross-terms appropriately and preserves local accuracy. It achieves orders-of-magnitude speedups and accurate attributions compared to brute-force approaches (Matthews et al., 2021).
• Spectral and Tensor Approaches: Spectral SHAP represents attributions as linear functionals on the model’s general Fourier coefficients under a product measure, with Lipschitz stability, truncation error bounds, and probabilistic convergence analysis for infinite-width neural networks (Morales, 31 Oct 2025). Interaction Tensor SHAP (IT-SHAP) encodes Shapley–Taylor higher-order interactions as tensor contractions in TT format, scaling tractably in dimension for structured models (Hasegawa et al., 5 Dec 2025).

5. Practical Applications, Evaluation, and Visualization

SHAP analysis underpins diverse real-world applications and serves as a workflow for both model validation and domain-specific discovery:

• Materials Science: XGBoost models for magnetic entropy prediction use TreeSHAP to identify influential compositional descriptors, enabling extraction of actionable design rules for new compounds (Terashima et al., 2023).
• Atmospheric and Industrial Systems: SHAP (TreeSHAP) quantifies the importance of solar and geomagnetic indices in thermospheric modeling (Bard et al., 30 Sep 2025), and identifies root-cause process features for industrial fault detection, supporting both dimensionality reduction and causal DAG discovery in complex datasets (Santos et al., 27 Oct 2025).
• Supervised Clustering: SHAP vectors enable downstream clustering (UMAP + HDBSCAN) to discover mechanisms or subtypes within class predictions, mapping heterogeneous pathways in high-dimensional settings such as neuroimaging for Alzheimer’s (Lin et al., 9 Oct 2025).
• Statistical Validity: CLE-SH automates end-to-end SHAP interpretation—data-driven feature selection, trend curve fitting, and interaction testing—coupled to statistical significance assessment, improving reporting rigor (Lee et al., 2024).

6. Limitations, Robustness, and Theoretical Caveats

Despite theoretical appeal, standard SHAP analysis presents several pitfalls:

• Aggregate SHAP Values and Feature Removal: Direct removal of features based solely on low mean absolute SHAP can fail, as model dependence may be masked outside empirical support. Only when aggregate SHAP is computed over the product-of-marginals (extended support) does small global attribution indicate safe discard, formalized via operator-theoretic proofs and practical random permutation strategies (Bhattacharjee et al., 29 Mar 2025).
• Distributional Uncertainty: SHAP values are polynomials in feature marginals. Under uncertainty, feature attributions can fluctuate considerably; computing the tight interval for $\phi_j$ across all admissible product distributions is NP-complete. Empirical sampling and bounding methodologies can offer robust intervals and rankings (Cifuentes et al., 2024).
• Axiomatic Critique and Remediation: Standard SHAP may assign spurious importance to irrelevant features in certain classifier contexts. By redefining the characteristic function to reflect prediction-preserving coalitions, trustable (sSHAP, aSHAP, cSHAP) variants enforce zero attribution to provably irrelevant or non-abductively necessary features, with algorithms matching existing SHAP computational properties (Letoffe et al., 2024).
• Interaction and Additivity Tradeoffs: Methods such as NSHAP and iSHAP explicitly mediate between exhaustive interaction capture and parsimony, introducing partitioning and statistical testing (e.g., interaction index, Welch t-test on randomized interventions) to balance fidelity with complexity (Xu et al., 2024).

7. Implementation, Visualization, and Human-Centered Explanations

The practical impact of SHAP is supported by a proliferation of fast, robust libraries and integrated explainability suites:

• Software Ecosystem: Standard implementations (Python/R: shap, mshap, contextualshap, CLE-SH) deliver interoperable APIs and batch/parallel SHAP computation.
• Visualization: SHAP summary (beeswarm), waterfall, and dependence plots communicate global and local attributions. For multi-class and tensor-valued explanations, generalized and high-dimensional waterfall plots can be projected in PCA or class-pair spaces for interpretability (Lin et al., 9 Oct 2025).
• Human-Readable Reports: Packages such as ContextualSHAP and CLE-SH automatically generate natural-language interpretations, incorporating feature aliases, domain-specific metadata, and audience control, thereby connecting rigorous quantitative analyses to actionable, user-friendly insights (Dwiyanti et al., 8 Dec 2025, Lee et al., 2024).

---

SHAP has become a central methodology in explainable artificial intelligence, delivering a theoretically grounded, computationally scalable, and highly modular approach to feature attribution across a broad spectrum of models and domains. Its extensions, algorithmic advances, and robustness frameworks continue to expand interpretability, supporting both practitioners and researchers in the rigorous analysis of machine learning predictions.