Provably Adaptive Linear Approximation for the Shapley Value and Beyond

Abstract: The Shapley value, and its broader family of semi-values, has received much attention in various attribution problems. A fundamental and long-standing challenge is their efficient approximation, since exact computation generally requires an exponential number of utility queries in the number of players $n$. To meet the challenges of large-scale applications, we explore the limits of efficiently approximating semi-values under a $Θ(n)$ space constraint. Building upon a vector concentration inequality, we establish a theoretical framework that enables sharper query complexities for existing unbiased randomized algorithms. Within this framework, we systematically develop a linear-space algorithm that requires $O(\frac{n}{ε^{2}}\log\frac{1}δ)$ utility queries to ensure $P(|\hat{\boldsymbolφ}-\boldsymbolφ|_{2}\geqε)\leq δ$ for all commonly used semi-values. In particular, our framework naturally bridges OFA, unbiased kernelSHAP, SHAP-IQ and the regression-adjusted approach, and definitively characterizes when paired sampling is beneficial. Moreover, our algorithm allows explicit minimization of the mean square error for each specific utility function. Accordingly, we introduce the first adaptive, linear-time, linear-space randomized algorithm, Adalina, that theoretically achieves improved mean square error. All of our theoretical findings are experimentally validated.

• The paper introduces Adalina, the first adaptive, linear-time algorithm for efficiently approximating Shapley and Banzhaf values with provable error guarantees.
• It unifies existing randomized methods through novel vector concentration inequalities, achieving sharp query complexity bounds under linear space constraints.
• The work provides definitive criteria for when paired sampling reduces variance, guiding practical improvements in feature attribution across various datasets.

Provably Adaptive Linear Approximation for the Shapley Value and Beyond

Overview

This paper (2604.08438) develops a unified theoretical and algorithmic foundation for efficiently approximating Shapley values and the broader family of semi-values in feature attribution and data valuation. The central challenge addressed is the exponential query complexity required for exact computation, which motivates the quest for scalable, provably adaptive randomized algorithms operating under linear ($\Theta(n)$) space constraints. The work leverages novel vector concentration inequalities to establish sharper query complexity bounds, systematically unifies prior approaches (including OFA, kernelSHAP variants, SHAP-IQ, regression-adjusted estimators), explicates the conditions under which paired sampling is beneficial, and introduces Adalina—the first adaptive, linear-time, linear-space randomized algorithm with improved mean squared error (MSE) guarantees.

Theoretical Foundations and Framework

The semi-value formulation generalizes the Shapley and Banzhaf values through a parameterized weight vector $\mathbf{m}$ and utility function $U$, with the computation requiring a sum over exponentially many subsets. The paper distinguishes two optimization regimes for approximation algorithms: minimization of the MSE and minimizing query complexity with probabilistic guarantees. A vector concentration inequality is introduced, replacing the classical union-bound-based analysis and yielding query complexity of $$O\left(\frac{n}{\epsilon^2 \log \frac{1}{\delta}}\right)$$ for unbiased estimators under linear space.

The framework establishes that, for all commonly used semi-values—including Beta Shapley values and weighted Banzhaf values—the optimal sampling distribution can be uniquely characterized and yields both sharp query complexity and decoupled MSE. The dominant factor in query complexity is a distribution-dependent constant $D^*$, and for these families, $D^* \in O(1)$. Thus, the results guarantee scalability even for large $n$.

Bridging Prior Methods and Characterizing Paired Sampling

Unified analysis demonstrates that several popular randomized algorithms, including OFA, unbiased kernelSHAP (with various sampling distributions), SHAP-IQ, and regression-adjusted estimators, are encompassed as special cases of the framework. The optimal distribution used in OFA and modified kernelSHAP achieves both minimal MSE and query complexity for symmetric semi-values, whereas SHAP-IQ's sampling distribution does not minimize query complexity for arbitrary semi-values.

A significant advancement is the definitive theoretical criterion for when paired sampling improves variance: the MSE is provably reduced if $\mathbb{E}[U(S)\cdot U([n]\setminus S)] > 0$. This characterizes the previously empirically ambiguous effectiveness of paired sampling. The prediction aligns precisely with empirical error curves—when utility functions are strictly positive, paired sampling reduces error, and when not, it may degrade performance.

Figure 1: The relative approximation error of unbiased kernelSHAP in approximating the Shapley value with different sampling distributions; dashed lines use paired sampling.

Adaptive Linear Algorithms and MSE Minimization

The adaptive algorithm Adalina is constructed via control variates, enabling explicit minimization of the MSE for each utility function $U$. The key insight is that the optimal constant offset $\gamma^*$ in the estimator is the mean of $\mathbf{m}$0 with respect to the same optimal sampling distribution. Adalina simultaneously estimates $\mathbf{m}$1 and $\mathbf{m}$2 using only linear space.

Empirical validation shows that Adalina consistently achieves lower relative approximation errors across diverse datasets and semi-value parameters. Furthermore, Adalina is shown to approach the performance of using the optimal fixed offset as sample size increases, demonstrating robustness on both Shapley and Banzhaf values.

Figure 2: The relative approximation error of SHAP-IQ, $\mathbf{m}$3, and Adalina (adaptive variant) on Beta Shapley values and weighted Banzhaf values across multiple datasets.

Comparative Analysis and Empirical Results

A comprehensive empirical comparison against baselines—including MSR-Banzhaf, SHAP-IQ, kernelSHAP variants, AME, ARM, GELS, and GELS-Shapley—substantiates the theoretical claims. The experiments cover multiple datasets, utility functions derived from tree-based models, and a range of semi-value parameters.

Results demonstrate that the proposed adaptive estimator outperforms or matches all baselines except for certain cases (e.g., Shapley value with kernelSHAP sometimes exhibits slightly lower error), indicating the possibility for further improvements. The findings reinforce the point that the new framework not only bridges disparate approximation methods but also enables principled algorithmic enhancements.

Figure 3: The relative approximation error of different randomized algorithms, showing performance across Beta Shapley and weighted Banzhaf values.

Implications and Future Directions

Theoretical implications include a unified perspective for designing scalable, provably efficient randomized algorithms for semi-value estimation. The adaptive construction decouples variance and query complexity optimization, enabling explicit control of MSE for any utility function.

Practically, the work supplies scalable tools for feature attribution and data valuation tasks in large-scale settings (e.g., explainability for complex ensemble models, equitable data pricing). The characterization of when adaptive techniques and paired sampling are beneficial removes prior empirical ambiguity and guides principled algorithm selection.

Looking forward, the paper suggests open questions regarding further improvements for Shapley value estimation (particularly leveraging non-constant offsets), potential extension to interaction values and high-order attribution, and generalization to broader classes of utility functions beyond classical settings.

Conclusion

This work systematically advances the theory and practice of efficient approximation for semi-values, unifies existing methods, and delivers adaptive, scalable algorithms with provable guarantees. It provides both sharper query complexity bounds and definitive guidance on variance reduction techniques, substantiated by empirical results. The proposed framework sets a foundation for future methodological developments in scalable explanation and valuation in machine learning.