An Asymptotic Analysis of the Shapley Value for Dataset Valuation

Abstract: We propose an asymptotic analysis of the Shapley value in a dataset valuation setting in which utilities are modeled as smooth functionals of empirical distributions via reproducing kernel Hilbert space (RKHS) mean embeddings. We prove that, despite its combinatorial definition, the Shapley value of a data source is asymptotically captured by a simple leading term. This term can be interpreted as the first-order contribution of a dataset relative to the surrounding data population. It also identifies the scale of the Shapley value as the number of data sources grows and provides a framework for analyzing existing Shapley value estimators. Moreover, for practitioners working with large numbers of datasets, the leading term becomes a tractable reference against which Shapley value approximations can be benchmarked.

• The paper establishes a rigorous asymptotic expansion for the Shapley value, showing that a data owner’s contribution decays as log(I)/I in collaborative settings.
• It leverages RKHS mean embeddings and Fréchet-differentiable utilities to derive precise error bounds and a benchmark for scalable Shapley estimators.
• The approach informs the design of permutation-based, group-testing, and stratified Monte Carlo methods, enhancing both interpretability and estimator efficiency.

Asymptotic Analysis of the Shapley Value for Dataset Valuation

Overview and Motivation

This paper provides a rigorous asymptotic characterization of the Shapley value in cooperative dataset valuation settings, where utilities are smooth functionals of empirical distributions through RKHS mean embeddings. The motivation is to understand the large-scale behavior of the Shapley value when the number of data owners grows, resolving theoretical and practical questions related to scalability, interpretability, and benchmarking of both the Shapley value itself and efficient proxy estimators.

Formal Setting

The analysis considers a collaborative learning scenario with multiple data owners, each contributing a dataset $D_j$. For each subset $S$ of owners, a utility function $v(D_S)$ encapsulates the gain from pooling their data. The corresponding cooperative game assigns to each owner the Shapley value, calculated as the average marginal improvement their dataset provides over all possible subsets.

Uniquely, the asymptotic regime is realized by fixing one reference data owner $i$ and letting the number of other (surrounding) owners $I\to\infty$, with those other datasets sampled i.i.d. from a population model. To enable analysis, utility functions are structured as $F(\mu(\widehat P_{D_S}))$, where $\mu(\cdot)$ is the RKHS mean embedding, and $F$ is a Fréchet-differentiable, Lipschitz-continuous functional.

Asymptotic Shapley Value Characterization

The paper's primary contribution is to show that, as $I$ grows, the Shapley value $\phi_i^I$ of the fixed data owner $S$0 admits a precise asymptotic expansion. The combinatorial averaging over exponentially many coalitions collapses to a tractable leading order term $S$1: $S$2 where:

• $S$3 is the size of $S$4,
• $S$5 is the mean dataset size in the population,
• $S$6 (harmonic number),
• $S$7 is the first-order utility-relevant deviation of $S$8 from the population reference.

Figure 1: Relative error of each estimator averaged over 30 independent repetitions with standard errors. The dashed curve is a reference guide for the predicted $S$9 relative error decay.

Formally, for smooth utilities and bounded kernel features, the Shapley value satisfies: $v(D_S)$0 where the expectation is over the random draw of surrounding players.

The leading term structurally factorizes:

• Dataset size ($v(D_S)$1): scaling property,
• First-order signal ($v(D_S)$2): RKHS-weighted differential utility,
• Coalition averaging ($v(D_S)$3): combinatorial decay.

Thus, for non-degenerate data owners ($v(D_S)$4), the Shapley value asymptotically scales as $v(D_S)$5. This provides a sharp answer to the order-of-magnitude question of how fast individual incremental value vanishes in collaborative environments as the number of participating owners grows.

Absolute and Relative Error Principle

A direct corollary is that any estimator $v(D_S)$6 of the Shapley value is relatively consistent if its absolute error is $v(D_S)$7, i.e.,

$v(D_S)$8

This reorients the design and analysis of approximate Shapley estimators: for large $v(D_S)$9, tracking the leading term is both necessary and sufficient for consistency in relative error. The approach thus delivers a concrete finite-$i$0 lower bound and sets the error scale that must be beaten by practical algorithms.

Consequences for Efficient Estimation

The paper then systematically applies this asymptotic framework to permutation-based Monte Carlo, group-testing-based, and DU-Shapley/statified MC estimators.

• Permutation Monte Carlo: For $i$1 samples, the estimator achieves vanishing relative error.
• Group Testing: Given a query budget achieving absolute error $i$2, relative consistency follows.
• DU-Shapley/Stratified MC: Both fit a structural class of "leading-order" estimators—those that directly approximate the cardinality-wise first-order signal—with provably bounded error and consistency.

Benchmarking via the Leading Term

The paper establishes, for the first time, a practical, interpretable, and computationally efficient large-scale benchmark for Shapley value approximations: the leading term $i$3. It is readily computed (or consistently estimated under known population parameters) even when exact Shapley computation is infeasible. The presented numerical results confirm that state-of-the-art estimators rapidly converge towards this benchmark as $i$4 increases, with empirically observed relative errors decreasing faster than $i$5 for realistic data distributions and utilities.

Figure 1: The average relative error of permutation Monte Carlo, DU-Shapley, and stratified Monte Carlo estimators, plotted against the leading term, demonstrates the predicted decrease with growing player population.

Theoretical and Practical Implications

Theoretical:

• The analysis provides a foundation for future large-scale, asymptotic theory in cooperative data valuation.
• It clarifies the exact shape of the value decay, settling foundational questions about the limiting regime of Shapley-based incentives in federated and collaborative learning.
• It justifies novel error metrics for estimator evaluation in the growing-player regime.

Practical:

• Practitioners can benchmark and calibrate scalable Shapley proxies by assessing their convergence to the (tractable) leading term.
• The leading term quantities are easily interpretable and allow attribution auditing at scale.
• The structure informs the design of improved sampling and variance reduction schemes in practical Shapley estimation.

Limitations and Directions for Future Work

The analysis presumes i.i.d. sampling in the owner population, smoothness and boundedness of the RKHS embedding and utility functional, and exact knowledge of the population parameters to compute oracle leading terms. Extending the results to dependent owners, more complex (e.g., deep learning-based) utility functionals, and high-dimensional/structured data settings constitutes promising directions. Statistical error from plug-in estimation of population references deserves further study.

Conclusion

This study provides a principled and rigorous framework for understanding Shapley value asymptotics in dataset valuation. It establishes that the value of a fixed data owner is dominated by an explicit, interpretable leading term of order $i$6, with direct implications for the design, analysis, and benchmarking of scalable Shapley estimators. This bridges the gap between cooperative game-theoretic value assignment and the practical realities of collaborative learning with large and growing sets of data owners.

Reference: "An Asymptotic Analysis of the Shapley Value for Dataset Valuation" (2607.03374)