ProxySHAP: Hybrid Estimator for Feature Interactions

• ProxySHAP is a hybrid estimator that combines decision-tree proxies with an optional residual correction to approximate higher-order feature interactions.
• It generalizes interventional TreeSHAP by enabling exact, polynomial-time extraction of cardinal-probabilistic interaction indices such as Shapley and Banzhaf.
• Empirical evaluations across diverse datasets show ProxySHAP outperforms competing methods in speed and accuracy, guiding its use for singleton and pairwise interactions.

Searching arXiv for the main paper and cited related work to ground the article with current identifiers. arxiv_search(query="Proxy-Based Approximation of Shapley and Banzhaf Interactions", max_results=5) arxiv_search(query="ProxySPEX shapley interactions Butler 2025", max_results=10) arxiv_search(query="SHAP-IQ Fumagalli 2023 interactions", max_results=10) arxiv_search(query="KernelSHAP-IQ Fumagalli 2024", max_results=10) ProxySHAP is a hybrid estimator for higher-order feature interactions that combines a decision-tree proxy model with an optional residual correction term to approximate Shapley and Banzhaf interactions, as well as other cardinal-probabilistic interaction indices, in modern machine learning applications. Its central objective is to reconcile the high sample efficiency of tree-based proxy models with a principled path to consistency via residual correction. The method is introduced in “Proxy-Based Approximation of Shapley and Banzhaf Interactions” (Thies et al., 21 May 2026), which also derives a polynomial-time generalization of interventional TreeSHAP for exact interaction extraction on tree ensembles, analyzes the variance behavior of Maximum Sample Reuse (MSR), and reports extensive benchmarking across tabular, vision, language, graph, and vision–language settings.

1. Formal interaction framework

ProxySHAP is formulated on a cooperative-game representation of feature attribution. Let $N=\{1,\dots,n\}$ be the set of players, identified with features, and let a value function $\nu:2^N\to\mathbb{R}$ assign a real score to each coalition $S\subseteq N$ (Thies et al., 21 May 2026). For any $S\subseteq N$ and $T\subseteq N\setminus S$, the discrete derivative of $\nu$ at $T$ with respect to $S$ is

$$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$

This discrete derivative is the interaction quantity from which the relevant indices are assembled. A cardinal-probabilistic interaction index with weights $p_t^s(n)$ is defined by

$\nu:2^N\to\mathbb{R}$0

Two special cases are emphasized.

Index	Weight specification
Shapley Interaction Index (SII)	$\nu:2^N\to\mathbb{R}$1
Banzhaf Interaction Index (BII)	$\nu:2^N\to\mathbb{R}$2

The same interaction index can be expressed through the Möbius transform. Defining $\nu:2^N\to\mathbb{R}$3, one has

$\nu:2^N\to\mathbb{R}$4

where the $\nu:2^N\to\mathbb{R}$5-weights are obtained from the $\nu:2^N\to\mathbb{R}$6-weights by Möbius inversion (Thies et al., 21 May 2026). This formulation is important because it clarifies that ProxySHAP is not restricted to one attribution semantics: it targets a family of interaction indices, with Shapley and Banzhaf as the principal instances.

2. Proxy model and exact extraction on tree ensembles

The proxy stage fits a decision-tree ensemble, such as XGBoost or LightGBM, to sampled coalitions $\nu:2^N\to\mathbb{R}$7 (Thies et al., 21 May 2026). The fitted tree ensemble $\nu:2^N\to\mathbb{R}$8 then serves as a surrogate value function on which interaction indices can be computed exactly.

For a single tree surrogate, each leaf $\nu:2^N\to\mathbb{R}$9 has prediction $S\subseteq N$0, and the path to that leaf induces two sets: $S\subseteq N$1, the features that must be present, and $S\subseteq N$2, the features that must be absent. The surrogate takes the form

$S\subseteq N$3

By linearity of $S\subseteq N$4, the extraction problem reduces to computing the interaction index on indicator functions of the form $S\subseteq N$5. Proposition 1 gives the closed form

$S\subseteq N$6

with

$S\subseteq N$7

The extraction algorithm iterates over interactions of interest and over leaves, adding the corresponding $S\subseteq N$8 contribution whenever $S\subseteq N$9 (Thies et al., 21 May 2026). The resulting complexity is $S\subseteq N$0 per tree, equivalently $S\subseteq N$1 total for the ensemble, which is polynomial in tree depth rather than exponential $S\subseteq N$2 as in Fourier-based methods. In the terms used by the paper, this is a polynomial-time generalization of interventional TreeSHAP for Shapley, Banzhaf, and other indices.

A common misunderstanding is to view the tree proxy as merely heuristic compression. The closed-form extraction result shows that, once the proxy is fitted, interaction computation on the surrogate is exact for the surrogate itself, not a Monte Carlo approximation of the surrogate (Thies et al., 21 May 2026).

3. Residual correction and the role of MSR

ProxySHAP augments the tree proxy with an optional residual correction based on the linearity decomposition

$S\subseteq N$3

Here, the tree proxy captures the main interaction structure, while the residual term is estimated with Maximum Sample Reuse (MSR), following Fumagalli et al. ’23 as summarized in the ProxySHAP paper (Thies et al., 21 May 2026). Given $S\subseteq N$4 sampled coalitions $S\subseteq N$5, the MSR estimator for the residual $S\subseteq N$6 is

$S\subseteq N$7

Under leverage sampling,

$S\subseteq N$8

Theorem 3.1 yields the variance bound

$S\subseteq N$9

The sketch given in the paper derives a general variance identity, specializes the $T\subseteq N\setminus S$0-weights and $T\subseteq N\setminus S$1, and bounds the relevant double sums in terms of harmonic numbers for $T\subseteq N\setminus S$2 and polynomial factors for $T\subseteq N\setminus S$3 (Thies et al., 21 May 2026).

The practical conclusion is deliberately qualified. MSR is variance-efficient for singleton effects, but its variance blows up exponentially in interaction order $T\subseteq N\setminus S$4 unless $T\subseteq N\setminus S$5; accordingly, the paper recommends using MSR adjustment only when $T\subseteq N\setminus S$6 or $T\subseteq N\setminus S$7. This directly addresses a frequent misconception that residual correction is uniformly beneficial. For higher-order interactions, especially in high-dimensional games, the correction can worsen the estimate because the variance increase can dominate the reduction in proxy bias (Thies et al., 21 May 2026).

4. Computational profile and comparison with prior estimators

The paper places ProxySHAP against several prior estimators, notably ProxySPEX, KernelSHAP-IQ, and SHAP-IQ (Thies et al., 21 May 2026). For the tree-proxy variant of ProxySHAP, the reported complexity components are:

• proxy fitting with XGBoost: $T\subseteq N\setminus S$8;
• exact interaction extraction: $T\subseteq N\setminus S$9;
• MSR adjustment, if used: $\nu$0.

The total complexity is therefore

$\nu$1

The linear-proxy variant is also described. Fitting a linear model with $\nu$2 features requires $\nu$3 via least squares, with $\nu$4 (Thies et al., 21 May 2026). By contrast, the comparison reported in the paper states that KernelSHAP-IQ, attributed there to Fumagalli et al. ’24, has complexity $\nu$5 per explained point; ProxySPEX, attributed to Butler et al. ’25, uses Fourier extraction with worst-case $\nu$6 per tree and then truncation solving least squares in $\nu$7, where $\nu$8 is often large; and SHAP-IQ, attributed to Fumagalli et al. ’23, has complexity $\nu$9 but high variance.

The significance of this comparison is twofold. First, ProxySHAP’s extraction stage avoids the depth-dependent combinatorial blow-up associated with Fourier-based methods on deep trees. Second, the method does not rely solely on asymptotically favorable scaling: its proxy model is intended to reduce the effective sample requirement before residual correction becomes necessary. This suggests that the method is designed to occupy a middle position between pure sampling estimators and exact but structurally constrained extractors.

5. Empirical evaluation across domains

The empirical study covers 47 datasets (Thies et al., 21 May 2026). The tabular component includes 26 TabArena datasets with $T$0 up to approximately $T$1 and UCI tasks with $T$2. Additional evaluations use synthetic “Local XAI” games on TabPFN and LightGBM, a ViT setting with up to 16 patches, DistilBERT on IMDB, two molecular datasets with GNNs having approximately 30–35 nodes, and a vision–language setting explaining CLIP via FIxLIP.

Approximation quality is measured by relative MSE,

$T$3

with lower values better, averaged over 30 explained instances and reported with SEM error bars (Thies et al., 21 May 2026).

The benchmark findings reported in Figures 2 and 3 are specific. ProxySHAP with XGBoost achieves the lowest relative MSE across all budgets and for both second- and third-order Shapley and Banzhaf interactions. In the low-budget regime, $T$4–$T$5, tree-proxy methods dominate linear baselines and KernelSHAP-IQ. In the high-budget regime, $T$6–$T$7, MSR adjustment recovers consistency for singleton and pairwise effects when $T$8, but for $T$9 it often hurts due to variance blow-up unless both $S$0 and $S$1 are small. The paper further reports that ProxySHAP outperforms ProxySPEX by orders of magnitude in speed, with extraction $S$2–$S$3 faster, and in error, especially when deep trees are used (Thies et al., 21 May 2026).

The downstream CLIP experiment evaluates pairwise Banzhaf interactions between image and text tokens for $S$4–$S$5, using AID, defined there as the area between insertion/deletion curves, and $S$6 faithfulness. In this setting, ProxySHAP without adjustment Pareto-dominates both the FIxLIP baseline and ProxySPEX over evaluation budgets from $S$7 to $S$8 CLIP calls (Thies et al., 21 May 2026). This is notable because it moves the discussion from approximation error alone to downstream explainability utility.

6. Contributions, limitations, and practitioner implications

The paper identifies three main contributions (Thies et al., 21 May 2026). The first is ProxySHAP itself: a hybrid estimator combining a tree-based regression proxy with exact polynomial-time extraction of any cardinal-probabilistic interaction index and a situational residual correction via MSR. The second is theoretical: a polynomial-time generalization of interventional TreeSHAP that bypasses the $S$9 blow-up, together with formal MSR variance bounds that characterize when residual adjustment is useful. The third is empirical: state-of-the-art approximation quality on 47 datasets, across budgets and domains, with strong performance in both small- and large-budget regimes.

The practitioner guidance in the paper is correspondingly specific. ProxySHAP requires only $$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$0 model calls and a standard gradient-boosted tree fit, such as XGBoost, to obtain highly accurate interaction estimates. Tree-proxy extraction is recommended for deep ensembles because it avoids combinatorial blow-up. MSR adjustment is recommended for singleton and pairwise interactions when budget exceeds $$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$1, and it is advised to skip it for higher orders or high-dimensional games unless very large $$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$2 is available. The paper also states that the method outperforms ProxySPEX, KernelSHAP-IQ, and SHAP-IQ in both speed and accuracy, making large-scale interaction explanations with $$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$3 practical (Thies et al., 21 May 2026).

Two limitations are integral to the method rather than peripheral caveats. First, the consistency path provided by residual correction is conditional on a variance regime that is favorable only for low-order interactions or comparatively small $$\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).$$4. Second, the performance profile depends on the quality of the fitted tree proxy. A plausible implication is that ProxySHAP is best understood not as a universal replacement for sampling-based estimators, but as a regime-adaptive procedure: exact on the proxy, optionally corrected on the residual, and most effective when the interaction structure is well captured by a tree ensemble while residual variance remains manageable.