Shapley Value Pruning

Updated 3 March 2026

Shapley value pruning is a method that applies cooperative game theory to determine the marginal utility of components for effective pruning.
It uses various approximation algorithms, such as Monte Carlo sampling and surrogate models, to scale evaluations to large neural networks.
Empirical studies show that Shapley-based pruning improves efficiency–accuracy trade-offs in CNNs, LLMs, and GNNs compared to traditional techniques.

Shapley value pruning is a suite of methods applying the Shapley value from cooperative game theory to identify and remove superfluous, redundant, or even harmful components from machine learning systems. By evaluating the average marginal utility of each component (e.g., neuron, filter, interaction, layer, attention head, or data point) across all possible coalition subsets, Shapley value pruning assigns a theoretically justified importance score that accounts for both synergy and redundancy. Across neural architectures and data modalities, this approach yields principled, data-driven rankings for pruning and denoising; these rankings are supported by multiple sampling and amortized approximation strategies that scale to modern network sizes and data volumes. Empirical results demonstrate that Shapley-based pruning consistently improves efficiency–accuracy trade-offs relative to conventional magnitude- or gradient-based heuristics, especially in low-data and noise-sensitive regimes.

1. Shapley Value Formalism and General Pruning Framework

At the core of Shapley value pruning is the cooperative game formulation. Given a set $N$ of components and a value function $v:2^N\to\mathbb{R}$ quantifying system utility for each subset, the Shapley value $\phi_i$ for component $i\in N$ is defined as:

$\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$

where $v(S\cup\{i\}) - v(S)$ is the marginal contribution of $i$ to $S$ . This definition ensures four key axioms: efficiency ( $\sum_i\phi_i = v(N)-v(\emptyset)$ ), symmetry (identically contributing players are equally rewarded), null-player (zero impact components receive zero value), and additivity (linearity in $v$ ).

In pruning applications, the "players" can represent neural units, model parameters, features, data samples, or other structural submodules. The value function $v:2^N\to\mathbb{R}$ 0 is closely linked to downstream metrics—typically accuracy, negative loss, or some validation objective measured on a (possibly masked/pruned) subsystem. Shapley values thus provide a global, context-aware ranking resilient to redundancy and negative contributions.

2. Algorithms and Approximations for Shapley Value Estimation

For practical deployments, computing exact Shapley values is infeasible for nontrivial $v:2^N\to\mathbb{R}$ 1 due to exponential coalition space. Multiple approximation protocols have been introduced:

Partial Shapley (leave- $v:2^N\to\mathbb{R}$ 2-out): Restricts the coalition sum to subsets of fixed size $v:2^N\to\mathbb{R}$ 3; reduces cost to $v:2^N\to\mathbb{R}$ 4 (Adamczewski et al., 2024).
Monte Carlo permutation sampling: Samples $v:2^N\to\mathbb{R}$ 5 random permutations $v:2^N\to\mathbb{R}$ 6 of $v:2^N\to\mathbb{R}$ 7, estimates $v:2^N\to\mathbb{R}$ 8 as the mean marginal increase when $v:2^N\to\mathbb{R}$ 9 joins the prefix of $\phi_i$ 0 (Ancona et al., 2020, Held et al., 2022, Adamczewski et al., 2024).
Weighted least squares regression (Shapley kernel): Fits a linear model to $\phi_i$ 1 observed for sampled $\phi_i$ 2, using the Shapley kernel as weights (Adamczewski et al., 2024, Akkas et al., 28 Jul 2025).
Sliding-Window/Local Approximations: Restricts Shapley computation to a local window around each component, e.g., neighborhood of adjacent layers in a transformer to capture only meaningful interactions (Sun et al., 3 May 2025).
Surrogate neural networks: Trains a parametric surrogate (e.g., MLP) to amortize $\phi_i$ 3 prediction for rapid Shapley estimation over coalitions (Zhang et al., 28 May 2025, Ding et al., 8 Feb 2026).
Antithetic permutation pairing: Combines permutation orderings and their reversals to reduce variance and bias (Fan et al., 2 Oct 2025).

These methods yield efficient, scalable estimates with quantifiable approximation bias/unbiasedness, enabling end-to-end pruning in networks with hundreds or thousands of prunable units.

3. Instantiations Across Model Classes and Data Modalities

Neural Network Compression

In CNN and MLP channel pruning, each channel/unit is a player, and $\phi_i$ 4 is the network's validation accuracy or negative loss when only $\phi_i$ 5 is active. Shapley pruning demonstrably outperforms magnitude-based and Taylor-gradient heuristics across LeNet, VGG, and ResNet architectures on both MNIST and CIFAR-10, achieving up to $\phi_i$ 6 compression and minimal increase in test error (Adamczewski et al., 2024). Similarly, in structured pruning under no- or low-fine-tuning, Shapley-based rankings result in up to $\phi_i$ 7– $\phi_i$ 8 higher retained top-1 accuracy than alternatives (Ancona et al., 2020).

Recommender Systems and Data Pruning

In the context of recommender systems, Shapley Value-driven Valuation (SVV) treats each user–item interaction as a player, with $\phi_i$ 9 corresponding to the reduction in squared error of a denoising autoencoder reconstructed only from $i\in N$ 0. SVV uses amortized FastSHAP estimation, then prunes the bottom $i\in N$ 1 of interactions in estimated $i\in N$ 2. This approach yields $i\in N$ 3– $i\in N$ 4 gains in Recall@10 and NDCG@10, recovers up to $i\in N$ 5 of injected long-tail noise, and reveals interpretable distinctions between pivotal and negligible behavioral records (Zhang et al., 28 May 2025).

LLMs: Layer and Head Pruning

Layer-wise Shapley value pruning in LLMs assigns value to transformer layers, with $i\in N$ 6. Non-uniform sparsity (SV-NUP) based on estimated $i\in N$ 7 preserves critical layers and prunes redundant ones, yielding $i\in N$ 8– $i\in N$ 9 relative PPL reductions at $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 0 sparsity in LLaMA-7B/13B models over SparseGPT (Sun et al., 3 May 2025). Surrogate-assisted Shapley in LLMs further leverages a lightweight MLP to predict $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 1 for arbitrarily masked layer combinations, aggregating stratified Monte Carlo marginal contributions for efficient large-scale application (Ding et al., 8 Feb 2026).

Head-level Shapley pruning in multilingual transformers isolates and excises attention heads with negative impact on per-language performance, identifying language-specific interference. Pruning with Shapley rankings can produce $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 2 accuracy points in XNLI-Chinese and up to $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 3 relative improvement in UDPOS-Chinese, without retraining (Held et al., 2022).

Non-standard Architectures and Shift-Invariance

In Kolmogorov–Arnold Networks (KANs), where functional weights are spline maps and coordinate-invariant pruning is essential, Shapley value–based "ShapKAN" scores are uniquely shift-invariant, outperforming magnitude scoring on symbolic regression, neuron-rank stability under affine shifts, and post-pruning RMSE (Fan et al., 2 Oct 2025).

Graph Neural Network Sparsification

Shapley-based edge importance in GNNs assigns both positive and negative contributions to edges in the computational graph of each node. Aggregating these edge-wise $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 4 values globally allows pruning of misleading and redundant edges, reducing inference MACs by up to $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 5 while incurring less than $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 6 accuracy loss at high sparsity thresholds, outperforming gradient- and MI-based baselines (Akkas et al., 28 Jul 2025).

4. Theoretical Properties, Guarantees, and Limitations

Shapley value pruning satisfies a set of axiomatic optimality and fairness properties: efficiency (total attribution matches actual performance delta), null-player (units with no marginal value receive zero attribution), symmetry (exchangeable units tie), and additivity. This framework uniquely minimizes expected harm for a given number of pruned components and is robust to redundancy, negative synergy, and shared responsibility among correlated units or features (Ancona et al., 2020, Owen et al., 2016).

However, practical limitations include computational overhead for large $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 7 (addressed by approximation schemes), possible redistribution of importance under strong dependencies among pruned units, and architecture-specific value function choices. In heavily correlated input or model spaces, Shapley ranks reflect shared, not solely marginal, effect; removal of one unit may cause importance redistribution among survivors (Owen et al., 2016).

5. Empirical Evaluation and Comparative Analysis

Experiments consistently demonstrate that Shapley value pruning achieves state-of-the-art accuracy–efficiency trade-offs across tasks and domains. In CNNs, Shapley-based schemes match or surpass oracle ranking within $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 8 of maximum possible overlap with optimal prune sets (Adamczewski et al., 2024). LLM layer- and head-level pruning via Shapley outperforms magnitude, random, and outlier-weighted layerwise allocations by up to $\phi_i = \sum_{S\subseteq N\setminus\{i\}} \frac{|S|! (|N|-|S|-1)!}{|N|!} \bigl[v(S\cup\{i\}) - v(S)\bigr]$ 9 relative in model perplexity and $v(S\cup\{i\}) - v(S)$ 0– $v(S\cup\{i\}) - v(S)$ 1 points in zero-shot accuracy (Sun et al., 3 May 2025, Ding et al., 8 Feb 2026). In recommenders, SVV yields both higher overlap with ground-truth injected noise and, in some cases, test accuracy that even exceeds that of "clean" (uncontaminated) training, reflecting benefits for tail/balance correction (Zhang et al., 28 May 2025). In GNNs, Shapley-pruned graphs retain superior accuracy at fixed sparsity relative to MI-, gradient-, or lottery-ticket-based approaches (Akkas et al., 28 Jul 2025).

6. Interpretability, Robustness, and Practical Guidance

Shapley value pruning offers intrinsic interpretability, with per-unit (or per-data, per-edge) explanations linked directly to marginal system utility. For example, low- $v(S\cup\{i\}) - v(S)$ 2 (but nominally "important") high-rating CDs can be flagged as negligible for collaborative filtering, revealing the difference between frequency and effect (Zhang et al., 28 May 2025). Aggregated attributions enable identification of adversarial, redundant, or negative-synergy components; in ensemble games, adversarial models systematically receive near-zero Shapley value and can be excluded for improved robustness (Rozemberczki et al., 2021).

Recommended best practices involve modest sample budgets ( $v(S\cup\{i\}) - v(S)$ 3–10 for MC-based methods), aggregation of attributions to avoid over-pruning occasional but sometimes critical units (e.g., via $v(S\cup\{i\}) - v(S)$ 4), and Monte Carlo or kernel regression for tractable yet precise approximation. Sliding-window and surrogate strategies are essential for very large LLMs. In low-data regimes where fine-tuning is unfeasible, Shapley methods consistently minimize post-prune accuracy loss.

7. Extensions and Outlook

The Shapley value pruning paradigm has been generalized to feature selection under dependency (Owen et al., 2016), data pruning for denoising (Zhang et al., 28 May 2025), hierarchical and groupwise pruning (Owen value), and even symbolic recoverability in KAN/other structured networks (Fan et al., 2 Oct 2025). Ongoing research focuses on amortized networks for hyper-efficient Shapley estimation, higher-order interaction attributions, and better value functions for nascent architectures. Future directions include extension to configuration games, adaptive sampling of coalitions, and combining Shapley value with alternative cooperative game indices for specialized compression or robustness objectives.

Key references: