KernelSHAP: Local Feature Attribution

• KernelSHAP is a model-agnostic framework that leverages weighted least-squares regression to approximate Shapley values, ensuring fair feature attributions.
• It unifies cooperative game theory with local surrogate modeling to interpret diverse black-box models in tabular, time series, and high-dimensional data.
• Recent advances enhance computational efficiency and stability through improved sampling, deterministic weighting, and interaction-aware surrogate extensions.

KernelSHAP is a model-agnostic, sampling-based framework for local feature attribution, originally formulated as a statistically justified, weighted least-squares approximation to the Shapley value from cooperative game theory. It is widely adopted for interpreting the predictions of black-box models (such as deep neural networks, ensembles, and autoencoders) and has become foundational in explainable AI (XAI), specifically for feature importance estimation in tabular, time series, and high-dimensional data domains. Methodologically, KernelSHAP unifies the Shapley attribution principle with local surrogate modeling ideas and is extensible to distributed, privacy-preserving, and interaction-aware settings.

1. Mathematical Foundations and Algorithm

KernelSHAP is built on the Shapley value, which provides a fair allocation of the total “payout” (e.g., a model's prediction) among features by averaging their marginal contributions across all possible feature subsets. For $f:\mathbb{R}^d\to\mathbb{R}$ and instance $x=(x_1,\ldots,x_d)$, the exact Shapley value for feature $i$ is

$$\phi_i(f, x) = \sum_{S \subseteq [d] \setminus \{i\}} \frac{|S|! (d - |S| - 1)!}{d!} \left[f_{S \cup \{i\}}(x_{S \cup \{i\}}) - f_S(x_S)\right],$$

where $f_S(x_S)$ is the expected model output when only $S$ is “present” (i.e., $x_S$ fixed, $x_{[d] \setminus S}$ replaced with background). This satisfies efficiency ($\sum_i \phi_i = f(x) - E[f(x)]$), symmetry, dummy, and linearity axioms.

Direct computation is intractable for moderate $d$ due to the $2^d$ subset enumeration. KernelSHAP circumvents this by fitting a local linear surrogate model on a limited number of sampled “coalitions” $z' \in \{0,1\}^d$, with

$g(z') = \phi_0 + \sum_{i=1}^d \phi_i z'_i,$

and optimizes the weighted squared error

$$L(\phi) = \sum_{z' \subseteq \{0,1\}^d} \pi_x(z') \left[f(h_x(z')) - g(z')\right]^2,$$

subject to the Shapley constraints (e.g., $\sum_i \phi_i = f(x) - \phi_0$). The kernel weight

$$\pi_x(z') = \frac{d - 1}{\binom{d}{|z'|} |z'| (d - |z'|)}$$

ensures the estimator obeys the Shapley axioms in expectation. In implementation, coalitions $z'$ are sampled according to the kernel, perturbed inputs $h_x(z')$ are generated by mixing $x$ and a background distribution $B$, model outputs $f(h_x(z'))$ are evaluated, and a weighted linear regression yields the local attributions $\{\phi_i\}$ (Roshan et al., 2023, Remman et al., 2021, Bogdanova et al., 2022, Covert et al., 2020).

Pseudocode Overview

1. Input: model $f$, instance $x$, background $B$, $M$ samples.
2. Compute $\phi_0 = E_{b\in B}[f(b)]$.
3. For $m = 1 \ldots M$: sample $z'^{(m)}$, build $h_x(z'^{(m)})$, evaluate $f(h_x(z'^{(m)}))$, weight with $\pi_x(z'^{(m)})$.
4. Solve weighted least-squares for $\phi$, under Shapley constraints.
5. Output local attributions.

2. Practical Implementation, Sampling, and Variants

KernelSHAP's core is a kernel-weighted regression, but practical implementation details have strong implications for speed, variance, and reliability. A primary computational challenge is the number and selection of coalitions:

• Sampling with replacement (standard): Draw $M$ coalitions $z'$ from the kernel and proceed as above. Stochasticity induces variance in attributions.
• Paired/antithetic sampling: For each sampled $z'$, include its complement; this halves estimator variance and yields results equivalent to a degree-2 polynomial fit (Covert et al., 2020, Fumagalli et al., 26 Jan 2026).
• Improved weighting/deterministic schemes: Replacing stochastic weights (arising from Monte Carlo frequencies) with deterministic, expectation-matched weights—such as C-kernel or CEL-kernel—reduces variance by up to 50%. Weighting corrections based on Wallenius’ distribution and finite-population inclusion probabilities further improve the estimator’s stability and efficiency (Olsen et al., 2024, Aanes, 13 Jan 2026, Covert et al., 2020).

Table: Key Sampling/Weighting Variants

Variant	Main Effect	Source
With replacement	Baseline, introduces variance	(Covert et al., 2020)
Paired sampling	Lowers variance, leverages symmetry	(Covert et al., 2020, Fumagalli et al., 26 Jan 2026)
C/CEL-kernel	Deterministic weights, variance reduction	(Olsen et al., 2024)
Wallenius alloc.	Without replacement, lower variance	(Aanes, 13 Jan 2026)

Practical choices (e.g., sampling granularity, budget size) directly affect runtime. For $d \sim 100$, $M$ is typically $10^3$–$10^4$. Computational complexity is $O(M d^2)$ for the regression, with model evaluation cost usually dominating (Chen et al., 5 Jun 2025, Olsen et al., 2024).

3. Theoretical Guarantees and Statistical Properties

The linear regression perspective yields non-asymptotic guarantees. The main result states that, with $M = O(d \log d / (\delta \epsilon^2))$ samples, the estimated attributions obey $\|\phi^{KS} - \phi^*\|_2 \leq \epsilon$ with probability $1 - \delta$ (Chen et al., 5 Jun 2025).

• Unbiasedness: Many variants (e.g., unbiased KernelSHAP) guarantee $E[\hat\phi] = \phi^*$, but the original (ordinary linear regression) form achieves lower variance at the expense of negligible bias.
• Variance estimation: Bootstrap schemes—most notably paired bootstrap, the doubled half bootstrap, and the new Symmetric bootstrap—enable finite-sample confidence intervals (Covert et al., 2020, Aanes, 13 Jan 2026).
• Sample complexity tradeoffs: KernelSHAP with leverage-score sampling can, in the worst case, remove a $\log d$ factor versus kernel-weight sampling, though kernel-weighting can be optimal for certain value functions (Chen et al., 5 Jun 2025).
• Efficiency in distributed and privacy-preserving settings is established by modified background/reference selection and orchestration of coalition evaluation (Bogdanova et al., 2022).

4. Limitations, Extensions, and Alternative Weighting Schemes

Despite its generality, KernelSHAP exhibits several limitations and motivates recent extensions:

• Marginal vs. causal attribution: By independently sampling absent features, KernelSHAP captures only direct/marginal effects, neglecting inter-feature causal chains. In domains with substantial feature dependence (e.g., robotic control), this can misestimate true attributions (Remman et al., 2021).
• Sampling-induced instability: Stochastic coalition selection introduces run-to-run variation in explanations. Deterministic sampling over full layers or layer-specific coalitions (e.g., ST-SHAP, layer-1 only) eliminates or sharply reduces instability with little fidelity loss (Kelodjou et al., 2023).
• Support mismatch: Standard KernelSHAP may ascribe low importance to genuinely functionally critical features if $f$ is adversarially defined outside the data support. KernelSHAP over the extended support (sampling from the product of marginal distributions) guarantees that small aggregate attributions imply safe feature removal (Bhattacharjee et al., 29 Mar 2025).
• Polynomial and interaction-aware surrogates: PolySHAP extends the linear surrogate to degree-$k$ polynomials, capturing non-linear and interaction effects, and is theoretically equivalent to paired sampling for $k=2$ (Fumagalli et al., 26 Jan 2026). KernelSHAP-IQ generalizes to explicit interaction index estimation within the weighted least-squares framework (Fumagalli et al., 2024).

5. Applications and Empirical Impact

KernelSHAP has seen adoption in a broad range of scientific and industrial tasks:

• Model interpretation and debugging: Post-hoc feature attribution for black-box models in health, finance, and security (Roshan et al., 2023, Ho et al., 2019).
• Unsupervised feature selection: In network anomaly detection, ranking features by mean $|\phi_i|$ and retraining on the top-ranked subset led to substantial gains in accuracy (0.71→0.90) and $F_1$ score (0.53→0.76) without using labels (Roshan et al., 2023).
• Distributed/explainable data collaboration: DC-SHAP yields consistent, privacy-preserving explanations across horizontal/vertical partitions and reduces RMSE between users by $1.75\times$ or more (Bogdanova et al., 2022).
• Time-series and sequential data: By flattening sequential inputs and supporting time-consistent attributions, KernelSHAP enables event detection and population-level feature aggregation over RNNs/LSTMs and classical models (Villani et al., 2022, Ho et al., 2019).
• Efficient PAC top-$k$ identification: Sampling and stopping condition improvements in KernelSHAP@k achieve up to $5\times$ reductions in sample and runtime in identifying the $k$ most-influential features for critical applications (Kariyappa et al., 2023).
• Benchmarking: KernelSHAP consistently outperforms model-agnostic alternatives in high-dimensional faithfulness and error metrics; deterministic and paired-weight variants further improve sample efficiency (Chen et al., 5 Jun 2025, Olsen et al., 2024).

6. Computational Considerations and Open Problems

KernelSHAP's bottleneck is the cost of model evaluations over perturbed inputs, which dominates the regression and sampling overheads. Performance is sensitive to:

• Number of features $d$ and background samples $B$.
• Number and method of coalition samplings $M$.
• Bias/variance tradeoffs in weight assignment and sampling regimes.
• Stability requirements and computational budgets in practical deployments.

Recent empirical studies suggest deterministic weighting (paired C-kernel or CEL-kernel) and sampling without replacement (Wallenius allocation) yield $20$–$50\%$ runtime reductions at fixed error, with only negligible additional implementation complexity over standard KernelSHAP (Olsen et al., 2024, Aanes, 13 Jan 2026). However, scaling to $d \gtrsim 1000$ or explaining models with highly structured or dependent features (e.g., images, text, graphs) remains a challenge. Theoretical optimality proofs for weighting/correction schemes and further adaptation to dependence-sensitive or causal objectives are active research areas.

7. Summary Table: Core Features and Advances

Aspect	Baseline KernelSHAP	Recent Advances	Key References
Surrogate Fit	Weighted linear	Polynomial (PolySHAP), interaction	(Fumagalli et al., 26 Jan 2026, Fumagalli et al., 2024)
Coalition Sampling	With replacement	Without replacement (Wallenius)	(Aanes, 13 Jan 2026)
Weighting	Stochastic (kernel)	Deterministic (C/CEL-kernel)	(Olsen et al., 2024)
Variance Estimation	Bootstrap (CLT)	Symmetric/doubled half bootstrap	(Aanes, 13 Jan 2026, Covert et al., 2020)
Causality-support	Marginal only	CausalSHAP, extended support	(Remman et al., 2021, Bhattacharjee et al., 29 Mar 2025)
Stability	Stochastic neighbors	Layerwise determinism (ST-SHAP)	(Kelodjou et al., 2023)
Efficient $k$-Selection	Brute force	Bandit/early stopping (SHAP@k)	(Kariyappa et al., 2023)

KernelSHAP is thus established as a theoretically sound, empirically validated, and highly extensible framework for local feature attribution and XAI. Continuing research is focused on optimizing computational efficiency, stability, variance control, and faithfulness to underlying data and causal structure.