---
title: Shapley Value Regression Explained
url: https://www.emergentmind.com/topics/shapley-value-regression
type: topic
---

# Shapley Value Regression Explained

Shapley value regression refers to the family of methodologies that apply Shapley values from cooperative game theory—originally formulated for attributing payoffs to individual players—to regression modeling and model explanation. In this context, Shapley values are used to fairly attribute model fit (e.g., $R^2$, KL-$R^2$) or predictions to individual features, feature groups, training data points, or structural parameters, all in a way that respects feature dependencies and the contribution of each in complex, possibly correlated, and nonlinear regression frameworks.

## 1. Foundations: Shapley Values for Regression Attribution

The canonical Shapley value for a feature $j$ is defined as the average marginal gain in a model's value function (such as out-of-sample $R^2$, log-likelihood, deviance, classification accuracy, or direct prediction) when adding feature $j$ to all possible subsets $S$ of features that do not already contain $j$. For a regression task over $d$ features, this is given by
\[
\varphi_j = \sum_{S \subset \{1, \ldots, d\} \setminus \{j\}} \frac{|S|!\,(d-|S|-1)!}{d!}\,[v(S \cup \{j\}) - v(S)],
\]
where $v(S)$ is the model value associated with the coalition $S$ (such as $R^2(S)$ for the least-squares setting) [2310.19245, 2601.00773, 2505.04775, 2401.05653]. This combinatorial definition satisfies the efficiency, symmetry, and dummy axioms. For regression, $v(S)$ may be the out-of-sample $R^2$ [2310.19245], the KL-divergence–based pseudo-$R^2$ for GLMs [2601.00773], or the prediction itself for instance-level attribution [1903.04209].

## 2. Linear, Generalized Linear, and Structural Models

In least-squares regression and GLMs, Shapley value regression decomposes variance explained or deviance into additive feature components. In the classical OLS case, $R^2(S)$ denotes the out-of-sample $R^2$ of the submodel using features in $S$ only, and the marginal increase in $R^2$ when adding feature $j$ is attributed according to the Shapley formula above [2310.19245]. For GLMs, the recommended value function is the normalized KL-divergence $R^2_{KL}$, interpreted as the fraction of total possible KL-explanatory power explained by the model, ensuring nonnegativity, monotonicity, and normalization [2601.00773].

Structural models and group-level counterfactual decompositions employ group Shapley values, extending classical feature-level attribution to importance decomposition over parameter blocks. These group Shapley values are formulated either by subset-averaging or, equivalently, as the solution to a constrained weighted least squares (WLS) regression, providing closed-form, fair decompositions for arbitrary groupings [2410.06875].

## 3. Efficient Estimation: Algorithms and Computational Innovations

The exponential subset complexity of exact Shapley computation motivates numerous algorithmic strategies:

- **Permutation-based Monte Carlo**: Randomly samples feature orders and averages marginal lifts [2310.19245].
- **Variance-reduction techniques**: Antithetic sampling (including permutations and their reverses) and Quasi-Monte Carlo approaches (e.g., argsort of Sobol’ points for permutation sampling) reduce estimator variance [2310.19245, 2012.01536].
- **Linear-algebraic optimizations for OLS (LS-SPA)**: High-dimensional efficiency is achieved by leveraging a single QR or Cholesky decomposition for the entire training data. Per-permutation update chains are solved in $O(d^3)$ time, and compressed representations are used for test data. This makes full-resolution computation practical for thousands of features and millions of samples, yielding orders-of-magnitude speedup over naïve approach [2310.19245].
- **Weighted least squares regression (KernelSHAP, Leverage SHAP, unified sketch-based methods)**: The Shapley value vector is the solution to a sum-to-constraint weighted least squares regression, with appropriate kernel weights enforcing the Shapley axioms [2012.01536, 2410.01917, 2506.05216]. Advanced row-sampling schemes (e.g., leverage-score sampling, bucketized subsets, or modified weights) offer improved convergence and provable sample complexity, notably $O(d\log d)$ for leverage-based approaches.
- **Regression-adjusted Monte Carlo (RegMSR)**: Combines Monte Carlo estimates for the residual (after subtracting a fitted surrogate model) with closed-form calculation for the surrogate, achieving unbiased, low-variance estimation and supporting any function family with efficiently computable probabilistic values (e.g., trees via path decompositions) [2506.11849].

## 4. Conditional and Local Shapley Value Regression

Conditional Shapley values explain predictions at the instance level by solving for the “local” attributions such that
\[
f(x^*) = \phi_0 + \sum_{j=1}^d \phi_j(x^*).
\]
Estimation requires the conditional expectation $v(S;x^*) = \mathbb{E}[f(X)|X_S = x_S^*]$, which is computationally expensive. Regression-based approaches replace conditional sampling with supervised regression of $v(S;x_S)$ on $x_S$, proceeding via
- **Separate-model regression**: Fitting one regression per coalition [2305.09536].
- **Surrogate-model regression**: A single model with masked inputs encoding coalition membership [2305.09536].
Recent work provides efficient linear-algebraic approaches to simultaneously approximate all $2^p$ conditional submodels for linear (or polynomial/spline) regressors using a block-diagonal “soft constraint” precision matrix and sparse Cholesky factorization, reducing computation from hours to minutes with negligible accuracy loss compared to full submodel enumeration [2504.18167].

## 5. Learning Shapley Prediction Rules

Prediction via Shapley Value Regression (ViaSHAP) is a paradigm in which models are trained to produce their own feature-wise Shapley values as outputs. The prediction is literally the sum over these per-feature contributions, guaranteeing the local accuracy property. This approach leverages universal approximation architectures (MLP or Kolmogorov-Arnold Networks) and is trained with a dual-term loss combining standard prediction error and Shapley value consistency (i.e., efficiency constraint on masked coalitions). ViaSHAP achieves state-of-the-art predictive and explanation fidelity while offering inference-time efficiency, with a single forward evaluation producing both prediction and full Shapley attribution [2505.04775].

## 6. Extensions: Bayesian, Nonparametric, and Robust Treatment

Bayesian models such as Bayesian Additive Regression Trees (BART) allow closed-form calculation of Shapley effects for high-dimensional, potentially nonlinear regression functions. Posterior consistency can be established under standard sparsity and design assumptions, and scalable algorithms exploit piecewise constant conditional means on tree leaves to compute Shapley estimates for $p\sim 500$ dimensions [2304.03809].

Robustness in the presence of concept drift and multicollinearity is addressed by:
- **Local Shapley-based feature selection** robust against concept shift, using groupwise partitioning of error types (over/under/correctly predicted) to identify and eliminate features whose local contributions systematically degrade under shift [2304.14774].
- **Covariance-adjusted (whitened) Shapley values**: Multicollinearity correction is implemented by linearly decorrelating absent features with respect to those in the coalition under consideration, replacing marginal draws with adjusted ones to ensure unbiasedness with respect to feature dependencies [2011.01661].

## 7. Practical Applications and Interpretive Tables

In applied contexts such as marketing mix modeling, Shapley value regression provides a fully interpretable decomposition of aggregate $R^2$ among correlated predictors (e.g., channel partners), supporting non-negative shares and resolving multicollinearity for business-relevant attribution tables. Adjusted regression coefficients proportional to Shapley shares guarantee recapitulation of the total signal and maintain interpretability, outperforming alternative post-hoc coefficient adjustment schemes in terms of attribution clarity [2401.05653].

| Application                    | Value Function      | Computational Approach       |
|------------------------------- |--------------------|-----------------------------|
| OLS / Ridge Regression         | $R^2$, $R^2(S)$    | LS-SPA, QR linear-algebra   |
| GLM (Poisson/Binomial/Etc.)    | KL-based $R^2_{KL}$| Submodel enumerations       |
| Feature Group Attribution      | $v(S)$             | WLS, group Shapley table    |
| Instance Prediction            | $f(x)$             | Surrogate regression, ViaSHAP|
| Large p, nonlinear             | Shapley Effects    | BART, post-hoc treeexact    |

## 8. Theoretical Properties, Consistency, and Convergence

Recent developments provide a unified theoretical framework for the entire regression-based Shapley value estimation landscape [2506.05216, 2410.01917]. Key results include:
- Equivalence between the combinatorial Shapley definition and sum-to-constraint weighted least squares regression.
- Non-asymptotic sample complexity and error bounds for leverage-score–based designs and row sampling.
- Optimal variance reduction via paired (antithetic) sampling, as explained by odd/even decompositions of the set function—the Shapley value depends only on its odd part, and paired sampling orthogonalizes away the even part [2602.01399].
- Modularity to generalize from classical Shapley values to arbitrary probabilistic values (e.g., beta-weighted, Banzhaf) by altering regression weights and constraints [2506.11849].
- Diagnostic tools for uncertainty quantification and convergence stopping based on the explicit CLT of the regression estimator [2012.01536].

## References

- "Efficient Shapley Performance Attribution for Least-Squares Regression" [2310.19245]
- "Prediction via Shapley Value Regression" [2505.04775]
- "Variable Importance in Generalized Linear Models -- A Unifying View Using Shapley Values" [2601.00773]
- "Group Shapley Value and Counterfactual Simulations in a Structural Model" [2410.06875]
- "Fast approximative estimation of conditional Shapley values when using a linear regression model or a polynomial regression model" [2504.18167]
- "A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them" [2305.09536]
- "Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Values" [2506.11849]
- "A Unified Framework for Provably Efficient Algorithms to Estimate Shapley Values" [2506.05216]
- "An Odd Estimator for Shapley Values" [2602.01399]
- "Provably Accurate Shapley Value Estimation via Leverage Score Sampling" [2410.01917]
- "Efficient computation and analysis of distributional Shapley values" [2007.01357]
- "Establishing Shapley Effects in Big-Data Emulation and Regression Settings using Bayesian Additive Regression Trees" [2304.03809]
- "Multicollinearity Correction and Combined Feature Effect in Shapley Values" [2011.01661]
- "Quantifying Marketing Performance at Channel-Partner Level by Using Marketing Mix Modeling (MMM) and Shapley Value Regression" [2401.05653]
- "From interpretability to inference: an estimation framework for universal approximators" [1903.04209]

Source: https://www.emergentmind.com/topics/shapley-value-regression