Proxy-Based Approximation of Shapley and Banzhaf Interactions

Abstract: Shapley and Banzhaf interactions capture the complex dynamics inherent in modern machine learning applications. However, current estimators for these higher-order interactions trade off between speed and accuracy. To overcome this limitation, we introduce ProxySHAP. ProxySHAP reconciles the high sample efficiency of tree-based proxy models with a principled path to consistency via residual correction. On a theoretical level, we derive a polynomial-time generalization of interventional TreeSHAP to compute exact interaction indices for tree ensembles, successfully bypassing exponential tree-depth dependencies in prior methods. Furthermore, we formally analyze the residual adjustment strategy, characterizing the specific conditions under which Maximum Sample Reuse (MSR) corrects proxy bias without its variance scaling exponentially with interaction size. Extensive benchmarking demonstrates that ProxySHAP sets a new state-of-the-art standard for approximation quality, including in large-scale applications with thousands of features. By achieving the lowest error in both small- and large-budget regimes, ProxySHAP significantly outperforms the prior best estimators ProxySPEX and KernelSHAP-IQ, while also delivering superior performance on downstream explainability tasks.

• The paper introduces ProxySHAP, a framework that efficiently approximates any-order Shapley and Banzhaf interactions via tree-based regression surrogates and theoretically grounded residual adjustment.
• It details a two-phase process that combines exact proxy extraction with Maximum Sample Reuse to balance bias and variance, supported by novel variance scaling bounds.
• Empirical evaluations across 47 diverse datasets show that ProxySHAP outperforms state-of-the-art baselines, enabling scalable, accurate, and actionable model explainability.

Proxy-Based Approximation of Shapley and Banzhaf Interactions: A Detailed Technical Analysis

Motivation and Technical Foundations

Shapley and Banzhaf indices, as classical attribution methods in cooperative game theory, provide a principled framework for quantifying the importance of features, data points, or other entities in complex machine learning models. While single-feature attributions (standard Shapley/Banzhaf values) are well understood and algorithmically mature, higher-order cardinal-probabilistic interactions—which elucidate synergistic, antagonistic, and non-additive dynamics among subsets—remain algorithmically challenging to estimate efficiently and accurately. Historically, estimators for these interactions have exhibited a speed-accuracy dichotomy: sample-efficient methods are typically biased or high-variance for interactions of order $k>1$, while accurate regression-based methods become intractable as $n \to 1000+$ or for high interaction order.

The "Proxy-Based Approximation of Shapley and Banzhaf Interactions" paper introduces ProxySHAP, a framework aimed at reconciling these conflicting objectives by leveraging fast, expressive tree-based surrogate modeling combined with theoretically grounded residual correction. Through a unifying decomposition, ProxySHAP computes cardinal-probabilistic indices (covering both Shapley and Banzhaf families and their higher-order extensions) via an exact proxy extraction stage (using regression surrogates, primarily XGBoost trees) and an efficient, situation-aware bias adjustment phase based on Maximum Sample Reuse (MSR). The work includes new theoretical bounds—most notably, variance scalings for MSR with increasing $n$ and $k$—and a polynomial-time generalization of interventional TreeSHAP for tree ensembles. The algorithm is benchmarked across a diverse suite of domains, showing substantial advantages relative to existing state-of-the-art such as ProxySPEX and KernelSHAP-IQ.

ProxySHAP: Algorithmic Framework

ProxySHAP's core principle is the decomposition

$$\phi^p_S(\nu) = \phi^p_S(\hat{\nu}) + \phi^p_S(\nu - \hat{\nu})$$

where $\nu$ is the true cooperative game (e.g., feature ablation game, data subset game), $\hat{\nu}$ is a proxy model trained on sampled coalitions, $S$ is any subset of interest, and $\phi^p_S$ denotes a cardinal-probabilistic interaction index parameterized by the distribution $p$ over coalition sizes.

This decomposition cleanly separates the estimator into two algorithmic subproblems:

1. Proxy Extraction: Fit a regression surrogate to the observed pairs $n \to 1000+$0, where $n \to 1000+$1 are coalitions, and extract interaction indices of any order for $n \to 1000+$2 exactly and efficiently. For tree-based surrogates, the authors generalize the interventional TreeSHAP framework so that extraction reduces to a single pass over leaves per index, with only polynomial overhead—independent of exponential tree depth.

Figure 1: Left: A ProxySHAP explanation of the SigLIP-2 model using only 2048 model calls. Right: In Phase 1, proxy model fitting on sampled coalitions; in Phase 2, proxy interaction extraction followed optionally by residual adjustment.

2. Residual Adjustment: Correct for bias by estimating $n \to 1000+$3. This uses the generalized MSR approach extended to arbitrary interaction order. The adjustment is theoretically justified for singleton interactions with subpolynomial variance, but as $n \to 1000+$4 or $n \to 1000+$5 increases, the variance can scale as $n \to 1000+$6, making the correction actionable primarily for low-order, small-$n \to 1000+$7 games or very large budgets.

   Figure 2: Comparison of ProxySHAP with and without MSR adjustment, measured by MSE ratio. MSR enhances Shapley value approximation, but for higher-order interactions, the variance scaling can degrade estimates.

The ProxySHAP algorithm is given as a two-phase process: Phase 1 is regression (proxy) fitting; Phase 2 is extraction and (as needed) residual estimation. Hyperparameter optimization (HPO) of the proxy is also advocated to further improve approximation quality, especially for large-scale or higher-budget settings.

Theoretical Results and Variance Analysis

The paper generalizes the variance analysis of MSR to interactions of arbitrary order, showing:

• Singleton Effects ($n \to 1000+$8): Variance of MSR under leverage sampling scales as $n \to 1000+$9.
• Higher-Order Interactions ($n$0): The variance grows as $n$1 (see Theorem and supplement). Therefore, adjustment is only practical (i.e., variance-controlled) when the coalition sample size $n$2 is substantially greater than $n$3.

This analysis informs practical recommendations for applying adjustment: adjust for $n$4 and $n$5 or at extremely high sample budgets; otherwise, rely solely on the proxy term.

Figure 3: Empirical variance scaling with the sampling budget $n$6 for interaction orders $n$7. Black curve matches theory: $n$8 for $n$9, $k$0 for $k$1.

Empirical Evaluation and Results

ProxySHAP is extensively benchmarked across 47 datasets—including tabular, vision, language, graph, and multimodal (CLIP) settings—spanning up to thousands of features.

• Approximation Quality: ProxySHAP achieves minimal relative MSE for both Shapley and Banzhaf indices, outperforming ProxySPEX, KernelSHAP-IQ, and other baselines for both small and large evaluation budgets.

  Figure 4: Relative MSE for Shapley interaction approximations of ProxySHAP versus state-of-the-art baselines, across budgets and interaction orders.

• Decomposition and Adjustment: The benefit of the MSR adjustment for residuals is reinforced empirically for $k$2 and for pairwise interactions with sufficiently large budgets. For large $k$3 or $k$4, adjustment often degrades estimation.
• Scalability: Due to the polynomial-time tree extraction algorithm, ProxySHAP scales efficiently to domains with thousands of features, where prior regression-based proxies become unusable.

  Figure 5: Speedup of interventional proxy extraction compared to Fourier extraction, for interactions of order 1--3 and various datasets.

• Proxy HPO: Careful tuning of the proxy class (e.g., using many shallow trees for high-dimensional games) is essential. HPO-informed XGBoost configurations improve approximation quality in large-scale domains at moderate computational cost.

  Figure 6: Approximation quality (Relative MSE) of ProxySHAP and ProxySPEX under different XGBoost tree depth settings, shown for small, medium, and large player domains.

• CLIP and Multimodal Explanations: ProxySHAP, when used to compute faithful interaction indices in models such as SigLIP-2 and CLIP variants, outperforms previous linear and ProxySPEX-based estimators, both in area under insertion-deletion curves and in $k$5 faithfulness metrics.

  Figure 7: Faithfulness $k$6 for ProxySHAP, ProxySPEX, and baseline on explaining CLIP (ViT-16) over MS COCO.

• Robustness Across Residual Approximators, Sampling, and Disjoint Splits: Experiments show ProxySHAP's performance is stable across different residual approximators and sampling schemes. Using the same coalition set for proxy fitting and residual estimation is preferred over disjoint sets.

Practical and Theoretical Implications

• Practical Impact: ProxySHAP enables efficient, accurate estimation of any-order cardinal-probabilistic interaction indices at scales relevant for modern machine learning, supporting applications in explainability, scientific discovery, and model debugging. Its C++ implementation, integrated with the shapiq toolkit, supports interactive and scalable usage beyond previous methods.
• Theoretical Implications: By establishing the polynomial-time extraction of general indices from tree surrogates and delineating the limitations of MSR adjustment, this work delineates the achievable frontier for interaction estimation in black-box models. It opens the door for further advances in proxy class selection (beyond XGBoost), improved variance reduction for residuals, and the study of generalized values for structured or grouped features.
• Guidance for Practitioners: For pairwise and low-order interactions in moderate $k$7, MSR adjustment should be employed. For large $k$8 or higher $k$9, the proxy term dominates, and attention should be paid to proxy hyperparameter tuning. Regression-based surrogates, especially tree ensembles, are recommended as their interactions of arbitrary order can be computed efficiently using the generalized extraction algorithm.

Conclusion

ProxySHAP constitutes a rigorous, efficient framework for approximating Shapley, Banzhaf, and higher-order interaction indices across domains. By combining expressive regression surrogates and theoretically precise, variance-aware residual correction, it resolves longstanding computational bottlenecks in interaction-based explainability. Its empirical performance consistently dominates alternatives, and its open-source C++ implementation ensures broad accessibility in both research and applied settings. Remaining challenges include proxy class innovation and developing residual estimators with more favorable scaling for high $$\phi^p_S(\nu) = \phi^p_S(\hat{\nu}) + \phi^p_S(\nu - \hat{\nu})$$0/$$\phi^p_S(\nu) = \phi^p_S(\hat{\nu}) + \phi^p_S(\nu - \hat{\nu})$$1 regimes.

Future research may generalize these frameworks to domain-aware proxies, study strategies for non-additive residual correction, and explore applications in scientific modeling, material sciences, and trustworthy AI for sensitive domains.