Hierarchical Contrastive Data Valuation (HCDV)

• The paper introduces HCDV, a scalable method that leverages contrastive representation, balanced hierarchical clustering, and Monte Carlo games to approximate Shapley values with controllable error.
• It overcomes the factorial computational bottleneck by reducing complexity from O(n!) to O(K log n) and supports diverse applications such as data filtering, streaming updates, and fair marketplace payouts.
• Empirical benchmarks on datasets ranging from synthetic examples to Criteo-1B validate HCDV’s superior runtime, stability, and predictive improvements with strong theoretical guarantees.

Hierarchical Contrastive Data Valuation (HCDV) is a scalable, Shapley-inspired framework for quantifying the value of individual training examples in large, heterogeneous, and geometrically structured datasets. HCDV is designed to address the factorial computational bottleneck and the geometric insensitivity of classical data-Shapley methods by leveraging a contrastively trained representation, a balanced hierarchical clustering, and local Shapley-style valuation via budget-propagated Monte Carlo games. The method provides theoretical guarantees of approximate Shapley axioms with controllable error, achieves superior runtime and stability, and directly supports diverse applications such as data filtering, streaming updates, and marketplace payouts (Xiao et al., 22 Dec 2025).

1. Foundations and Motivation

Classical Data-Shapley assigns a payoff $\phi_i$ to each data point via the formula

$$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$

where $v(\cdot)$ is typically a model performance metric (e.g., accuracy, AUC). This approach is intractable for large $n$ ($O(n!)$) and treats data points independently, ignoring latent geometric structure. Modern datasets often exceed $n>10^5$, are heterogeneous, and lie on complex manifolds, making classical approaches unsuitable.

HCDV replaces the pointwise "player" assumption with a hierarchy of cluster coalitions, allocating Shapley-style payoffs in a top-down manner. This design aims to:

• Scale to large $n$ by reducing factorial complexity.
• Respect multi-scale geometric structure.
• Regularize outliers through curvature-based constraints.

2. Three-Stage Methodology

HCDV consists of three main stages:

Stage I: Contrastive, Geometry-Preserving Representation

An encoder $f_\theta: \mathcal X \rightarrow \mathbb R^d$ (e.g., ResNet/CNN for images, MLP for tabular data) is trained to both maximize predictive performance and amplify inter-class geometry via augmentations and a contrastive objective:

$$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$

with

• $\mathcal M(S)$: model performance on $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$0.
• $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$1: normalized dispersion over cross-label pairs in $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$2.
• $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$3: curvature-based smoothness regularizer,

$$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$4

where $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$5.

Stage II: Balanced Hierarchical Clustering

The embedded data are hierarchically partitioned via a balanced recursive $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$6-means, producing cluster sets $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$7 at each level $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$8, where each cluster $$\phi_i(\mathcal D) = \sum_{S\subseteq\mathcal D\setminus\{i\}} \frac{|S|!\,(n-|S|-1)!}{n!}\left[v(S\cup\{i\})-v(S)\right]$$9 is subdivided until all leaves have size $v(\cdot)$0. Capacity constraints,

$v(\cdot)$1

prevent imbalanced splits. Total coalitions is $v(\cdot)$2; hierarchy depth is $v(\cdot)$3 for constant $v(\cdot)$4.

Stage III: Local Monte Carlo Shapley-Style Payoffs

At each level $v(\cdot)$5, clusters are treated as players, and a Monte Carlo approximation is used to estimate local Shapley values:

$v(\cdot)$6

where each $v(\cdot)$7 is a random permutation, and $v(\cdot)$8 is the characteristic function incorporating both performance and dispersion:

$v(\cdot)$9

Estimated payoffs are normalized and propagated downward:

$n$0

At the leaf level, payoffs are either computed exactly (if $n$1) or uniformly split among members.

Total computational cost is $n$2, with $n$3 the cost of a forward pass, eliminating the factorial growth of the original method.

3. Theoretical Guarantees

HCDV approximately preserves the four classical Shapley axioms under mild assumptions:

• Approximate Efficiency: The deviation in total value mass satisfies

$n$4

where $n$5 controls MC approximation error.

• Coalition Deviation: With $n$6 permutations,

$n$7

and $n$8 ensures all errors $n$9 with high probability.

• Top-$O(n!)$0 Surrogate Regret: For top-$O(n!)$1 selection, regret is bounded by $O(n!)$2 for $O(n!)$3.
• Approximate Symmetry, Dummy, and Additivity: Monte Carlo Shapley at the coalition level is unbiased and additive, with errors $O(n!)$4; at the point level, symmetry/dummy deviations are at most $O(n!)$5.

4. Empirical Evaluation and Benchmarks

Extensive evaluation across four tasks demonstrates HCDV's accuracy, efficiency, and stability (Xiao et al., 22 Dec 2025):

Dataset	$O(n!)$6	Application	HCDV Gains
Synthetic Two-Gaussians	$O(n!)$7	Toy, geometric sensitivity	Predictive lift up to $O(n!)$8pp
UCI Adult	$O(n!)$9	Tabular/classification	Up to $n>10^5$0 faster valuation
Fashion-MNIST	$n>10^5$1	Image/augmentation	$n>10^5$2--$n>10^5$3\% lower CV (stability)
Criteo-1B$n>10^5$4 CTR	$n>10^5$5M	Click-through prediction	$n>10^5$6--$n>10^5$7pp AUC; sub-$n>10^5$8s latency

Baseline comparisons include Monte Carlo Data-Shapley (MCDS), Group Shapley (GS), Data Banzhaf, Random, DU-Shapley, and KNN-Shapley. OpenDataVal suite evaluation shows HCDV consistently improves predictive utility after training on the top $n>10^5$9 of samples, reduces coefficient of variation, and slashes valuation time.

5. Downstream Applications

HCDV directly enables several data-driven tasks:

• Augmentation Filtering: On Fashion-MNIST with $n$0 augmentations, HCDV selects top-$n$1 samples that yield $n$2pp accuracy, $n$3 cluster overlap, and full class coverage.
• Low-Latency Streaming Updates: For simulated click-streams ($n$4 steps of $n$5 samples), incremental HCDV achieves $n$6 of AUC from full recompute, with $n$7 speedup and sub-$n$8s latency.
• Fair Marketplace Payouts: On UCI Adult with $n$9 sellers, HCDV's seller payoff correlates at $f_\theta: \mathcal X \rightarrow \mathbb R^d$0 with leave-one-out marginals and achieves low Gini index, at $f_\theta: \mathcal X \rightarrow \mathbb R^d$1 min runtime compared to $f_\theta: \mathcal X \rightarrow \mathbb R^d$2 min for MCDS.

6. Hyperparameters, Implementation, and Practical Considerations

Practical instantiation of HCDV requires careful selection of hyperparameters:

• Permutation budget ($f_\theta: \mathcal X \rightarrow \mathbb R^d$3): $f_\theta: \mathcal X \rightarrow \mathbb R^d$4--$f_\theta: \mathcal X \rightarrow \mathbb R^d$5 typically sufficient for $f_\theta: \mathcal X \rightarrow \mathbb R^d$6.
• Cluster counts: $f_\theta: \mathcal X \rightarrow \mathbb R^d$7–$f_\theta: \mathcal X \rightarrow \mathbb R^d$8, $f_\theta: \mathcal X \rightarrow \mathbb R^d$9–$$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$0, leaf size $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$1–$$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$2.
• Contrastive/smoothness weights: $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$3, $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$4.
• Compute: One GPU for embedding; GPU + multi-core CPU for clustering and payoff computation.
• Implementation Pseudocode: (Algorithm 1 in (Xiao et al., 22 Dec 2025))—train contrastive encoder, embed points, build balanced hierarchy, propagate budgets, and estimate local Shapley values at each level. Streaming updates handled via Algorithm 2: assign new data to nearest leaves, update subtrees, and rebalance as needed.

Embedding quality and smoothness critically impact valuation accuracy. Cluster depth $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$5 and leaf size $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$6 control the compute/granularity trade-off. Early stopping on validation metric $$\max_\theta\, E_{S\sim\mathcal P_{\rm batch}} \left[\mathcal M(S)\, +\, \lambda\,\bar\Delta_c(S)\right] - \alpha\,\Omega(\theta)$$7 is advised.

7. Extensions, Limitations, and Future Directions

HCDV's generality supports extensions to federated and active learning environments (Editor's term). Its requirements—balanced trees, bounded characteristic functions, and accessible model/dispersion metrics—are compatible with standard supervised learning pipelines.

A plausible implication is that HCDV can serve as a template for efficient, geometry-aware valuation in privacy-preserving or distributed settings. Modulating hierarchy granularity and representation regularization may further enhance both interpretability and downstream performance.

Current limitations include sensitivity to the clustering quality and the assumed adequacy of the contrastive embedding for all downstream tasks. The uniform split heuristic in large leaves introduces approximation error, though bounded in practice.

HCDV represents an advance in efficient, scalable, and theoretically substantiated data valuation, achieving practical applicability across multiple domains, including streaming and fair exchange, while maintaining approximate compliance with key Shapley principles (Xiao et al., 22 Dec 2025).