---
title: 'ProxySHAP: Hybrid Estimator for Feature Interactions'
url: https://www.emergentmind.com/topics/proxyshap
type: topic
---

# ProxySHAP: Hybrid Estimator for Feature Interactions

Searching arXiv for the main paper and cited related work to ground the article with current identifiers.
arxiv_search(query="Proxy-Based Approximation of Shapley and Banzhaf Interactions", max_results=5)
arxiv_search(query="ProxySPEX shapley interactions Butler 2025", max_results=10)
arxiv_search(query="SHAP-IQ Fumagalli 2023 interactions", max_results=10)
arxiv_search(query="KernelSHAP-IQ Fumagalli 2024", max_results=10)
ProxySHAP is a hybrid estimator for higher-order feature interactions that combines a decision-tree proxy model with an optional residual correction term to approximate Shapley and Banzhaf interactions, as well as other cardinal-probabilistic interaction indices, in modern machine learning applications. Its central objective is to reconcile the high sample efficiency of tree-based proxy models with a principled path to consistency via residual correction. The method is introduced in “Proxy-Based Approximation of Shapley and Banzhaf Interactions” [2605.22738], which also derives a polynomial-time generalization of interventional TreeSHAP for exact interaction extraction on tree ensembles, analyzes the variance behavior of Maximum Sample Reuse (MSR), and reports extensive benchmarking across tabular, vision, language, graph, and vision–language settings.

## 1. Formal interaction framework

ProxySHAP is formulated on a cooperative-game representation of feature attribution. Let $N=\{1,\dots,n\}$ be the set of players, identified with features, and let a value function $\nu:2^N\to\mathbb{R}$ assign a real score to each coalition $S\subseteq N$ [2605.22738]. For any $S\subseteq N$ and $T\subseteq N\setminus S$, the discrete derivative of $\nu$ at $T$ with respect to $S$ is

$$
\Delta_S \nu(T) := \sum_{L\subseteq S} (-1)^{|S|-|L|}\cdot \nu(T\cup L).
$$

This discrete derivative is the interaction quantity from which the relevant indices are assembled. A cardinal-probabilistic interaction index with weights $p_t^s(n)$ is defined by

$$
\phi^p_S(\nu) := \sum_{T\subseteq N\setminus S} p^s_{|T|}(n)\cdot \Delta_S \nu(T).
$$

Two special cases are emphasized.

| Index | Weight specification |
|---|---|
| Shapley Interaction Index (SII) | $p_t^s(n)=1/[(n-s+1)\cdot \binom{n-s}{t}]$ |
| Banzhaf Interaction Index (BII) | $p_t^s(n)=1/2^{n-s}$ |

The same interaction index can be expressed through the Möbius transform. Defining $m_T(\nu):=\Delta_T \nu(\emptyset)$, one has

$$
\phi^p_S(\nu)=\sum_{T\supseteq S} q^s_{|T|}(n)\cdot m_T(\nu),
$$

where the $q$-weights are obtained from the $p$-weights by Möbius inversion [2605.22738]. This formulation is important because it clarifies that ProxySHAP is not restricted to one attribution semantics: it targets a family of interaction indices, with Shapley and Banzhaf as the principal instances.

## 2. Proxy model and exact extraction on tree ensembles

The proxy stage fits a decision-tree ensemble, such as XGBoost or LightGBM, to sampled coalitions $\{(T,\nu(T))\}$ [2605.22738]. The fitted tree ensemble $\hat{\nu}_{\text{tree}}$ then serves as a surrogate value function on which interaction indices can be computed exactly.

For a single tree surrogate, each leaf $j$ has prediction $c_j$, and the path to that leaf induces two sets: $R_j$, the features that must be present, and $L_j$, the features that must be absent. The surrogate takes the form

$$
\hat{\nu}_{\text{tree}}(T)=\sum_{j\in \text{leaves}} c_j \cdot \mathbf{1}[R_j\subseteq T\subseteq N\setminus L_j].
$$

By linearity of $\phi^p$, the extraction problem reduces to computing the interaction index on indicator functions of the form $\mathbf{1}[R_j\subseteq \cdot \subseteq N\setminus L_j]$. Proposition 1 gives the closed form

$$
\phi^p_S(\hat{\nu}_{\text{tree}})
=
\sum_{j: S\subseteq L_j\cup R_j} c_j \cdot \lambda(|L_j|,|R_j|,|S\cap L_j|,|S|),
$$

with

$$
\lambda(\ell,r,u,s)
:=
\sum_{i=0}^{\ell-u} (-1)^{i+u}\cdot \binom{\ell-u}{i}\cdot q^s_{i+u+r}(n).
$$

The extraction algorithm iterates over interactions of interest and over leaves, adding the corresponding $c_j\cdot \lambda(\cdot)$ contribution whenever $S\subseteq L_j\cup R_j$ [2605.22738]. The resulting complexity is $O(\#\text{nodes}\cdot |S|)$ per tree, equivalently $\Theta(n_{\text{nodes}}\cdot |S|)$ total for the ensemble, which is polynomial in tree depth rather than exponential $4^d$ as in Fourier-based methods. In the terms used by the paper, this is a polynomial-time generalization of interventional TreeSHAP for Shapley, Banzhaf, and other indices.

A common misunderstanding is to view the tree proxy as merely heuristic compression. The closed-form extraction result shows that, once the proxy is fitted, interaction computation on the surrogate is exact for the surrogate itself, not a Monte Carlo approximation of the surrogate [2605.22738].

## 3. Residual correction and the role of MSR

ProxySHAP augments the tree proxy with an optional residual correction based on the linearity decomposition

$$
\phi^p_S(\nu)=\phi^p_S(\hat{\nu}_T)+\phi^p_S(\nu-\hat{\nu}_T).
$$

Here, the tree proxy captures the main interaction structure, while the residual term is estimated with Maximum Sample Reuse (MSR), following Fumagalli et al. ’23 as summarized in the ProxySHAP paper [2605.22738]. Given $m$ sampled coalitions $T_i\sim P_{\text{sampling}}$, the MSR estimator for the residual $r=\nu-\hat{\nu}_T$ is

$$
\hat{\phi}^{\text{MSR}}_S(r;T)
=
\frac{1}{m}\sum_{i=1}^m r(T_i)\cdot (-1)^{|S|-|S\cap T_i|} \,
\frac{p^s_{|T_i|-|S\cap T_i|}(n)}{P_{\text{sampling}}(T_i)}.
$$

Under leverage sampling,
$$
P_{\text{sampling}}(T)=\frac{1}{(n+1)\cdot \binom{n}{|T|}},
$$
Theorem 3.1 yields the variance bound

$$
V[\hat{\phi}^{\text{MSR}}_S(\nu;T)] \le
\begin{cases}
O(\|\nu\|_\infty^2\cdot \log n / m) & \text{if } |S|=1, \\
O(\|\nu\|_\infty^2\cdot n^{|S|-1} / m) & \text{if } |S|\ge 2.
\end{cases}
$$

The sketch given in the paper derives a general variance identity, specializes the $p$-weights and $P_{\text{sampling}}$, and bounds the relevant double sums in terms of harmonic numbers for $s=1$ and polynomial factors for $s\ge 2$ [2605.22738].

The practical conclusion is deliberately qualified. MSR is variance-efficient for singleton effects, but its variance blows up exponentially in interaction order $k$ unless $m\gg n^{k-1}$; accordingly, the paper recommends using MSR adjustment only when $n<30$ or $m\gg n^{k-1}$. This directly addresses a frequent misconception that residual correction is uniformly beneficial. For higher-order interactions, especially in high-dimensional games, the correction can worsen the estimate because the variance increase can dominate the reduction in proxy bias [2605.22738].

## 4. Computational profile and comparison with prior estimators

The paper places ProxySHAP against several prior estimators, notably ProxySPEX, KernelSHAP-IQ, and SHAP-IQ [2605.22738]. For the tree-proxy variant of ProxySHAP, the reported complexity components are:

- proxy fitting with XGBoost: $O(m\log m)$;
- exact interaction extraction: $O(|S|\cdot n_{\text{trees}}\cdot \ell_{\max})$;
- MSR adjustment, if used: $O(|S|\cdot m)$.

The total complexity is therefore

$$
O(m\log m + |S|\cdot(n_{\text{trees}}\cdot \ell_{\max}+m)).
$$

The linear-proxy variant is also described. Fitting a linear model with $|S|$ features requires $O(m\cdot |S|^2)$ via least squares, with $|S|\approx \binom{n}{k}$ [2605.22738]. By contrast, the comparison reported in the paper states that KernelSHAP-IQ, attributed there to Fumagalli et al. ’24, has complexity $O(n^2\cdot m)$ per explained point; ProxySPEX, attributed to Butler et al. ’25, uses Fourier extraction with worst-case $O(4^d)$ per tree and then truncation solving least squares in $O(|C^\*|^3)$, where $|C^\*|$ is often large; and SHAP-IQ, attributed to Fumagalli et al. ’23, has complexity $O(m\cdot |S|)$ but high variance.

The significance of this comparison is twofold. First, ProxySHAP’s extraction stage avoids the depth-dependent combinatorial blow-up associated with Fourier-based methods on deep trees. Second, the method does not rely solely on asymptotically favorable scaling: its proxy model is intended to reduce the effective sample requirement before residual correction becomes necessary. This suggests that the method is designed to occupy a middle position between pure sampling estimators and exact but structurally constrained extractors.

## 5. Empirical evaluation across domains

The empirical study covers 47 datasets [2605.22738]. The tabular component includes 26 TabArena datasets with $n$ up to approximately $1{,}776$ and UCI tasks with $n\le 100$. Additional evaluations use synthetic “Local XAI” games on TabPFN and LightGBM, a ViT setting with up to 16 patches, DistilBERT on IMDB, two molecular datasets with GNNs having approximately 30–35 nodes, and a vision–language setting explaining CLIP via FIxLIP.

Approximation quality is measured by relative MSE,

$$
\text{Relative MSE}
=
\frac{\sum_S (\hat{\phi}_S-\phi_S)^2}{\sum_S \phi_S^2},
$$

with lower values better, averaged over 30 explained instances and reported with SEM error bars [2605.22738].

The benchmark findings reported in Figures 2 and 3 are specific. ProxySHAP with XGBoost achieves the lowest relative MSE across all budgets and for both second- and third-order Shapley and Banzhaf interactions. In the low-budget regime, $m\approx 10^2$–$10^3$, tree-proxy methods dominate linear baselines and KernelSHAP-IQ. In the high-budget regime, $m\approx 10^4$–$10^5$, MSR adjustment recovers consistency for singleton and pairwise effects when $n<30$, but for $k\ge 3$ it often hurts due to variance blow-up unless both $n$ and $k$ are small. The paper further reports that ProxySHAP outperforms ProxySPEX by orders of magnitude in speed, with extraction $10\times$–$1{,}000\times$ faster, and in error, especially when deep trees are used [2605.22738].

The downstream CLIP experiment evaluates pairwise Banzhaf interactions between image and text tokens for $n\approx 60$–$72$, using AID, defined there as the area between insertion/deletion curves, and $R^2$ faithfulness. In this setting, ProxySHAP without adjustment Pareto-dominates both the FIxLIP baseline and ProxySPEX over evaluation budgets from $10^3$ to $10^5$ CLIP calls [2605.22738]. This is notable because it moves the discussion from approximation error alone to downstream explainability utility.

## 6. Contributions, limitations, and practitioner implications

The paper identifies three main contributions [2605.22738]. The first is ProxySHAP itself: a hybrid estimator combining a tree-based regression proxy with exact polynomial-time extraction of any cardinal-probabilistic interaction index and a situational residual correction via MSR. The second is theoretical: a polynomial-time generalization of interventional TreeSHAP that bypasses the $4^d$ blow-up, together with formal MSR variance bounds that characterize when residual adjustment is useful. The third is empirical: state-of-the-art approximation quality on 47 datasets, across budgets and domains, with strong performance in both small- and large-budget regimes.

The practitioner guidance in the paper is correspondingly specific. ProxySHAP requires only $m$ model calls and a standard gradient-boosted tree fit, such as XGBoost, to obtain highly accurate interaction estimates. Tree-proxy extraction is recommended for deep ensembles because it avoids combinatorial blow-up. MSR adjustment is recommended for singleton and pairwise interactions when budget exceeds $n^{k-1}$, and it is advised to skip it for higher orders or high-dimensional games unless very large $m$ is available. The paper also states that the method outperforms ProxySPEX, KernelSHAP-IQ, and SHAP-IQ in both speed and accuracy, making large-scale interaction explanations with $n\gtrsim 1{,}000$ practical [2605.22738].

Two limitations are integral to the method rather than peripheral caveats. First, the consistency path provided by residual correction is conditional on a variance regime that is favorable only for low-order interactions or comparatively small $n$. Second, the performance profile depends on the quality of the fitted tree proxy. A plausible implication is that ProxySHAP is best understood not as a universal replacement for sampling-based estimators, but as a regime-adaptive procedure: exact on the proxy, optionally corrected on the residual, and most effective when the interaction structure is well captured by a tree ensemble while residual variance remains manageable.

Source: https://www.emergentmind.com/topics/proxyshap