Generalized Hoeffding Decomposition

• Generalized Hoeffding Decomposition is an extension of the classical technique that decomposes multivariate functions with arbitrarily dependent inputs into hierarchically orthogonal components.
• It employs projection operators and conditional expectations to separate variance into structural and correlative contributions, facilitating accurate global sensitivity analysis.
• This framework enhances model interpretability and feature selection in dependent input scenarios, leveraging iterative methods and nonparametric estimation for practical analysis.

The Generalized Hoeffding Decomposition (GHD) extends the classical orthogonal expansion of square-integrable functions of multiple random inputs to cases in which the input variables exhibit arbitrary dependencies. Traditional Hoeffding decompositions, foundational in probabilistic analysis and sensitivity quantification, rely on mutual independence of inputs to guarantee mutual orthogonality of summands corresponding to different variable coalitions. GHD provides a rigorous framework for decomposing functions of dependent inputs under mild regularity and boundedness conditions on the joint distribution, resulting in hierarchically orthogonal expansions and a variance attribution scheme that incorporates both structural (variance) and correlative (covariance) contributions. This formalism underpins state-of-the-art approaches to global sensitivity analysis and model interpretability when dependencies are present.

1. Classical Versus Generalized Hoeffding Decomposition

In the classical setting, for a function $\eta: \mathbb{R}^p \rightarrow \mathbb{R}$ defined on independent random inputs $X = (X_1, ..., X_p)$, the Hoeffding decomposition takes the form: $$\eta(x) = \eta_0 + \sum_{i=1}^p \eta_i(x_i) + \sum_{1 \leq i < j \leq p} \eta_{ij}(x_i, x_j) + \cdots + \eta_{1,\dots,p}(x_1,\dots,x_p)$$ where each component function is mutually orthogonal with respect to the product measure induced by independence; hence, the variance of the output splits additively: $$\mathrm{Var}(Y) = \sum_{u \subseteq \{1,\dots,p\}} \mathrm{Var}(\eta_u(X_u))$$ Sobol indices $S_u = \mathrm{Var}(\eta_u(X_u))/\mathrm{Var}(Y)$ quantify the sensitivity of each coalition.

When inputs are dependent, the classical mutually orthogonal subspaces do not exist. The generalized framework, under the assumption that $P_X$ is absolutely continuous with respect to a product measure $\nu$ and satisfies a lower-bounded density condition

$$p_X(x) \geq M\cdot p_{X_u}(x_u)p_{X_{u^c}}(x_{u^c}), \qquad 0 < M \leq 1,$$

constructs decomposition into subspaces $H_u^0$ of $L^2$ functions of $X_u$ orthogonal to all lower-order effects, leading to the unique expansion: $$\eta(X) = \sum_{u \subseteq \{1,\dots,p\}} \eta_u(X_u), \quad \eta_u \in H_u^0.$$ Orthogonality is now hierarchical: $\langle \eta_u, \eta_v \rangle = 0$ if $v \subsetneq u$, but not, in general, for unordered pairs.

2. Decomposition and Projection Operators

For the generalized setting, the decomposition is constructed via projectors. For two variables, the system reads: $$Y = \eta(X_1, X_2) = \eta_0 + \eta_1(X_1) + \eta_2(X_2) + \eta_{12}(X_1, X_2)$$ with projections explicitly given by conditional expectation operators:

• $P_{H_\emptyset}(\eta) = \mathbb{E}[\eta(X)]$
• $$P_{H_1^0}(\eta) = \mathbb{E}[\eta(X) | X_1] - \mathbb{E}[\eta(X)]$$
• $$P_{H_2^0}(\eta) = \mathbb{E}[\eta(X) | X_2] - \mathbb{E}[\eta(X)]$$
• $$P_{H_{12}^0}(\eta) = \eta(X_1, X_2) - \mathbb{E}[\eta | X_1] - \mathbb{E}[\eta | X_2] + \mathbb{E}[\eta]$$

Solving for the component functions is operationally mapped to solving a linear system involving these special projectors, which may be numerically addressed via iterative methods such as Gauss–Seidel, particularly when symmetry or sparsity is present.

3. Hierarchical Orthogonality and Sensitivity Indices

Because the decomposed components are not generally mutually orthogonal, additional cross-terms enter the variance decomposition: $$\mathrm{Var}(Y) = \sum_{u \not= \emptyset} \left[ \mathrm{Var}(\eta_u(X_u)) + \sum_{v \not= u, u\cap v \neq \emptyset} \mathrm{Cov}(\eta_u(X_u), \eta_v(X_v)) \right]$$ The generalized sensitivity index for set $u$ is given as

$$S_u = \frac{ \mathrm{Var}(\eta_u(X_u)) + \sum_{v: u\cap v \neq \emptyset} \mathrm{Cov}(\eta_u(X_u), \eta_v(X_v)) }{\mathrm{Var}(Y)}$$

These indices sum to unity even when input variables are dependent: $$\sum_{u \subseteq \{1,...,p\}, u \neq \emptyset} S_u = 1$$ The structure delineates the "structural" (variance) from "correlative" (covariance) contributions of each input coalition to output variability—a necessity for meaningful sensitivity analysis in dependent scenarios.

4. Assumptions, Copula Equivalence, and Estimation Algorithms

The GHD requires:

• Absolute continuity of joint law $P_X$ with respect to $\nu$, typically a product measure.
• Lower-bounded density condition as described above. For two inputs, a related condition on the copula density is equivalent to the required product-form bound.

Estimation in practice entails nonparametric regression for conditional expectations—a kernel-based local polynomial regression with leave-one-out cross-validation is suggested. Matrix inversion in local polynomial estimation is efficiently handled via the Sherman–Morrison formula. For functional equations arising from the projection system, an iterative numerical Gauss–Seidel algorithm is effective.

In the case where inputs are grouped into independent pairs with within-pair dependence (IPDV models), the decomposition applies first at the group level (classical orthogonal decomposition), then at the pair level (independent pairwise generalized decomposition), allowing refined attribution via first and second-order indices.

5. Practical Sensitivity Analysis and Interpretability

The GHD, when combined with new indices, provides interpretable sensitivity analysis of model outputs in non-independent regimes, now standard in realistic scenarios. It quantifies both direct and indirect effects of input variables, capturing how dependency inflates or modulates the contribution of input coalitions.

For model calibration, reliability, or reverse engineering, the separation into variance and covariance contributions immediately identifies structurally influential inputs and disentangles these from purely correlative effects, enhancing the decision-support robustness.

The indices allow resource allocation, feature selection, or uncertainty quantification strategies to correctly account for interdependence, preventing misattribution of importance that can occur when naively assuming independence.

6. Extensions, Limitations, and Future Directions

The framework is robust for distributions admitting densities with respect to product measures with boundedness as required. For higher dimensions, the approach generalizes naturally via recursive projectors onto hierarchically orthogonal subspaces.

For estimation, nonparametric procedures scale with sample size and variable count; improvements in computational linear algebra and sample-efficient regressors further ameliorate resource requirements.

Open problems include relaxing density assumptions, addressing severely degenerate or highly sparse joint supports, and extending the method to discrete or categorical input spaces beyond the binary case, e.g., via combinatorial or algebraic modifications to the projection system.

Finally, the approach bridges classical ANOVA-type decompositions and modern explainable machine learning needs for "black-box" models, incorporating dependence into attribution and certification frameworks. This positions GHD as a core methodology for interpretable statistical analysis in complex, correlated input domains (Chastaing et al., 2011).