---
title: 'SHAPLLM: Shapley Methods for LLMs'
url: https://www.emergentmind.com/topics/shapllm
type: topic
---

# SHAPLLM: Shapley Methods for LLMs

SHAPLLM denotes a collection of recent methodologies and frameworks at the intersection of Shapley value-based explainability and large language models (LLMs). This term encompasses Shapley Value-based Non-Uniform Pruning (SV-NUP) for LLM compression [2505.01731], hybrid SHAP-LLM explainability architectures for human-in-the-loop moderation in ML decision support [2511.20001], and principled extensions of Shapley feature attribution for stochastic LLM inference [2511.01311]. These approaches share the mathematical foundation of the Shapley value from cooperative game theory and are designed to provide either structural model optimization (via pruning and importance quantification) or transparent, mathematically-grounded explanations for LLM predictions. Methods under the SHAPLLM umbrella demonstrate significant performance and interpretability improvements by leveraging theoretical guarantees and scalable approximation schemes.

## 1. Shapley Value Foundations and Adaptation to LLMs

The classical Shapley value assigns importance scores to features or components (e.g., transformer layers, input tokens) based on their marginal contributions to a cooperative “game”—in this context, the model’s prediction or performance functional. For a model with $n$ elements $X = \{x_1,\ldots,x_n\}$ and a function $v:2^{X} \rightarrow \mathbb{R}$, the Shapley value for element $i$ is:
\[
\phi_i(v) = \sum_{S \subseteq X \setminus \{i\}} \frac{|S|! (n - |S| - 1)!}{n!} \left[v(S \cup \{i\}) - v(S)\right]
\]
This allocation uniquely satisfies the axioms of efficiency ($\sum_i \phi_i = v(X)-v(\emptyset)$), symmetry, dummy, and additivity.

For transformer-based LLMs, layers can be treated as “players” ($\mathcal{T} = \{1, \dots, T\}$). The value function for pruning is typically defined as $\nu(S) = 1/\mathrm{PPL}(S)$, with $\mathrm{PPL}(S)$ the perplexity when only layers in $S$ are active [2505.01731]. For explainability, input tokens act as players, and the output function $f_S(x_S)$ denotes the model’s prediction when only tokens in $S$ are “present” [2511.20001, 2511.01311].

Adapting the Shapley value to LLMs necessitates consideration of stochastic model outputs due to sampling-based decoding. Monte Carlo approximations are employed: for each coalition $S$, the empirical payoff $h_n(S)$ is estimated by averaging $n$ model outputs, yielding an expected Shapley value in the limit $n\to\infty$ [2511.01311]. Under deterministic inference (e.g. temperature $=0$), classic Shapley properties hold exactly.

## 2. Efficient Shapley Value-based Non-Uniform Pruning (SV-NUP)

SV-NUP targets pruning of large transformer models by using Shapley values to assign non-uniform sparsity ratios to each layer, in contrast to conventional uniform pruning [2505.01731]. The SV-NUP pipeline comprises:

- **Quantifying Layer Contributions:** For transformer layer $t$, the exact Shapley value $\phi_t$ can only be computed for $T$ layers via $O(T 2^{T-1})$ masked inferences, which is infeasible for large $T$.
- **Sliding Window-based Approximation (SWSV):** Restricts the calculation of each $\phi_t$ to a window of $N \ll T$ consecutive layers around $t$, reducing complexity to $O(T 2^{N-1})$. The approximate value is:
  \[
  \hat\phi_t = \sum_{S \subseteq S_t \setminus \{t\}} \frac{|S|! (N - |S| - 1)!}{N!} [\nu(S \cup T_t \cup \{t\}) - \nu(S \cup T_t)]
  \]
  where $S_t$ is the window and $T_t = \mathcal{T} \setminus S_t$.
- **Pruning Budget Assignment:** Each layer’s pruning ratio $\rho_t$ is inversely proportional to its estimated Shapley value $\hat\phi_t$, with constraints to prevent extreme allocation:
  \[
  a_t = 2\lambda\, \frac{\hat\phi_t - \min_k \hat\phi_k}{\max_k \hat\phi_k - \min_k \hat\phi_k} \qquad
  \rho_t = \rho - a_t + \operatorname{mean}_k a_k
  \]
  The actual pruning is then performed with a one-shot method (e.g. SparseGPT) using these budgets.

**Empirical Results:** On LLaMA-7B and LLaMA-13B at 70% sparsity, SV-NUP achieves $-$18.01% and $-$19.55% relative PPL reductions compared to uniform SparseGPT, respectively. Zero-shot task accuracy is improved or maintained. Optimal sliding window sizes $N=5$ or 7 achieve the best trade-off between approximation and compute [2505.01731].

| Model           | Uniform PPL | SV-NUP PPL | Improvement |
|-----------------|-------------|------------|-------------|
| LLaMA-7B (70%)  | 18.42       | 15.10      | −18.01%     |
| LLaMA-13B (70%) | 13.74       | 11.06      | −19.55%     |

## 3. SHAP-LLM Hybrid Explainability for Human-in-the-Loop Moderation

The SHAP-LLM framework couples local SHAP (kernel-based) feature attributions with LLM-generated free-form rationales to provide interpretable, actionable model predictions for tasks such as mental health and cyberbullying classification [2511.20001]. The pipeline consists of:
- A text classifier (e.g., fine-tuned transformer MentalBERT).
- A SHAP explainer computing per-token attributions $\phi_i(x)$ for input $x$.
- An LLM acting as a narrative rationale generator, conditioned on extracted SHAP attributions and classifier output.

**Algorithmic Overview:**  
Given a post $x$:
1. Predict class probabilities $p = f(x)$, predicted class $\hat y=\mathrm{argmax}(p)$.
2. Use SHAP to compute $(\mathrm{token}_i, \phi_i)$ pairs.
3. Select top-$k$ attributions, assemble a prompt with tokens and their $\phi_i$.
4. Pass to LLM (e.g., GPT-OSS-20B) to obtain a narrative rationale $e$.
5. Render results in a dashboard: highlight top-$k$ tokens, display predicted label/confidence/disclaimer, and provide LLM-generated rationale.

**Mathematical Details:**  
\[
\phi_i(x) = \sum_{S \subseteq \{1, \dots, M\} \setminus \{i\}} \frac{|S|! (M - |S| - 1)!}{M!} \left[ f_{S \cup \{i\}}(x_{S \cup \{i\}}) - f_S(x_S) \right]
\]
Token attributions are normalized to weights $w_i = \phi_i/\sum_j |\phi_j|$. Optional embedding aggregation:
\[
R_{\mathrm{SHAP}} = \sum_{i=1}^M w_i E_i
\]

**Qualitative Effects:**  
The interface provides both quantitative (token-level) and qualitative (natural language) explanations, supporting rapid high-risk content triage and enhanced moderator trust [2511.20001]. Pseudocode is provided in the original work.

## 4. Shapley Attribution for Stochastic LLM Decision Support

llmSHAP extends Shapley feature attribution to stochastic LLM inference, investigating the extent to which classical axiomatic properties (efficiency, symmetry, dummy, additivity) hold under non-deterministic model outputs [2511.01311]. Key insights include:

- **Monte Carlo Adaptation:** Replace $v(S)$ in the Shapley formula by the empirical mean payoff $h_n(S)$ over $n$ LLM outputs from coalition $S$.
- **Implementation Variants:**
  - **shap$_S$:** Exact, $O(n2^n)$ complexity.
  - **shap$_{cS}$:** Cached, $O(2^n)$.
  - **shap$_{SW}$:** Sliding-window, $O((n-w+1)2^w)$ for window size $w$.
  - **shap$_C$:** Counterfactual (leave-one-out), $O(n)$.
- **Axiomatic Satisfaction:**  
  | Method      | Efficiency | Symmetry | Dummy |
  |-------------|------------|----------|-------|
  | shap$_S$    | ✗          | ✓        | ✓     |
  | shap$_{cS}$ | ✓          | ✓        | ✓     |
  | shap$_{SW}$ | ✗          | ✗        | ✓     |
  | shap$_C$    | ✗          | ✓        | ✓     |

  Efficiency can fail under stochastic redraws since intermediate terms (coalition outputs) may not cancel. Symmetry and dummy hold conditionally. Monte Carlo stabilization ($n\approx5-20$) is recommended for reliability [2511.01311].

- **Empirical Evaluation:**  
  Sliding-window variants with $w=3$ balance attribution fidelity ($0.85$–$0.90$ cosine similarity to the gold standard) and tractability ($\sim$1/5th the cost of full-cached Shapley).

## 5. Interpretability and Practical Implications

All SHAPLLM methodologies derive significant interpretability benefits from their Shapley-theoretic grounding:
- **Structured Importance Ranking:** For SV-NUP, $\hat\phi_t$ yields a direct layer ranking, supporting targeted layer ablation, budget assignment for pruning/quantization, or structured removal at finer granularity (e.g., attention heads, submodules) [2505.01731].
- **Post-hoc Model Understanding:** Token- or layer-level attributions can inform model debugging, dynamic inference policies (e.g., early exit), or user-facing explainability modules [2511.20001].
- **Human-in-the-Loop Workflows:** Narrative rationales generated conditionally on SHAP attributions ensure that flagged decisions remain actionable and interpretable for moderators and domain experts [2511.20001].

Scalability is achieved via sliding-window or leave-one-out strategies, trading off axiomatic exactness for practical deployment in large-scale LLMs [2505.01731, 2511.01311]. All frameworks are compatible with arbitrary pruning or classification criteria, encompassing both activation- and gradient-based techniques.

## 6. Future Directions and Research Opportunities

Future extensions proposed in primary SHAPLLM works include:
- **Joint Pruning and Quantization:** Integration of Shapley-based budget allocation with quantization pipelines for end-to-end compression [2505.01731].
- **Fine-grained Attribution:** Application of Shapley analysis at the head, neuron, or even sub-token level; supports more precise structured compression and interpretability [2505.01731, 2511.01311].
- **Ablation and Early-Exit Studies:** Use approximated Shapley values for controlled ablation, dynamic inference, and resource–accuracy trade-offs [2505.01731].
- **Improved Human-AI Collaboration:** Coupling rigorous attribution with LLM-generated rationales, deployed in workflows where interpretability is safety- or efficacy-critical [2511.20001].
- **Theoretically Principled Sampling and Approximation:** Investigation of more efficient or mathematically justified approximations (e.g., stratified sampling, adaptive windowing) to scale attributions to ever-larger LLMs [2511.01311].

SHAPLLM represents a unifying formalism for both pruning and explainability in the LLM domain, balancing the theoretical soundness of the Shapley value with application-driven tractability and transparency.

Source: https://www.emergentmind.com/topics/shapllm