Functional ANOVA Decomposition in RKHS

• Functional ANOVA decomposition is a method that represents a multivariate function as an orthogonal sum of contributions from individual variables and their interactions.
• It leverages reproducing kernel Hilbert spaces and centered ANOVA kernels to obtain explicit, non-recursive decompositions and analytical sensitivity index computations.
• This approach underpins surrogate modeling, Gaussian process regression, and global sensitivity analysis, offering clarity and computational efficiency in high-dimensional settings.

Functional ANOVA decomposition is a mathematical formulation that expresses a multivariate function as an orthogonal sum of components associated with individual variables and their interactions. This construction has deep connections to variance-based sensitivity analysis, kernel methods, statistical learning, and interpretability in machine learning. The RKHS-centric framework described in (Durrande et al., 2011) provides foundational advances in the theory and practical computation of functional ANOVA decompositions, especially as they relate to kernel interpolation, Gaussian process regression, and global sensitivity analysis.

1. Functional ANOVA Decomposition: Foundational Concepts

Functional ANOVA (Analysis of Variance) decomposition represents a real-valued function $f: D \subseteq \mathbb{R}^d \to \mathbb{R}$ as a sum of terms reflecting the contribution of individual variables and subsets of variables to the output. More formally, under suitable integrability and measure-theoretic assumptions, the decomposition can be written as: $$f(x) = f_0 + \sum_{i=1}^d f_i(x_i) + \sum_{1 \leq i < j \leq d} f_{ij}(x_i, x_j) + \cdots + f_{1\cdots d}(x_1,...,x_d)$$ where $f_0$ is a constant, $f_i$ are main effects, $f_{ij}$ are second-order interactions, etc. Each component is orthogonal in $L^2(\mu)$ under the product measure $\mu$ and satisfies suitable centering constraints (e.g., main effects have zero mean with respect to the marginal of $x_i$).

The RKHS approach in (Durrande et al., 2011) establishes a rigorous pathway to constructing function spaces and kernels wherein this decomposition is not only possible, but is mirrored algebraically by the kernel itself. This connection is central to ensuring both orthogonality and easy computation of sensitivity indices.

2. RKHS Decomposition: Zero-Mean Functions and Kernel Splitting

Let $\mathcal{H}$ be a one-dimensional RKHS with reproducing kernel $k$ over domain $D$ and measure $\mu$. The decomposition leverages the integral operator $I : h \mapsto \int_D h(s) d\mu(s)$ and the Riesz representation theorem to write: $$\mathcal{H} = \mathcal{H}_0 \oplus^\perp \mathcal{H}_1$$ where:

• $$\mathcal{H}_0 = \left\{ h \in \mathcal{H} : \int_D h(s)d\mu(s) = 0 \right\}$$ is the subspace of zero-mean functions (centered w.r.t. $\mu$)
• $\mathcal{H}_1 = \text{span}\{R\}$, with $R$ the representer of the integral

Correspondingly, the kernel itself splits as: $k(x, y) = k_0(x, y) + k_1(x, y)$ with: $$k_0(x, y) = k(x, y) - \frac{\int_D k(x, s)d\mu(s) \int_D k(y, s)d\mu(s)}{\iint_{D \times D} k(s, t)d\mu(s)d\mu(t)}$$ The function $x \mapsto k_0(x, y)$ is centered for each $y$, i.e., $\int_D k_0(x, y)d\mu(x) = 0$.

In high dimensions, these zero-mean subspaces are combined using tensor products and correspond to subcomponents (main effects, interactions) in the ANOVA decomposition. This approach is the foundation for the construction of “centered” ANOVA (K*_ANOVA) kernels.

3. Construction of (Centered) ANOVA Kernels

Given univariate kernels $k^i$ (with their decomposition into $k^i = k^i_0 + k^i_1$), the multivariate (centered) ANOVA kernel is defined as: $$K^*_{\text{ANOVA}}(x, y) = \prod_{i=1}^d [1 + k^i_0(x_i, y_i)]$$ which expands as: $$K^*_{\text{ANOVA}}(x, y) = 1 + \sum_{I \subseteq \{1, ..., d\}} \prod_{i \in I} k^i_0(x_i, y_i)$$ This kernel's RKHS is composed of functions whose ANOVA terms (main and interactions) are zero-mean and mutually orthogonal in $L^2(\mu)$. This property is critical for the interpretability and computability of the corresponding decomposition and underpins the construction of surrogate models in computer experiments.

4. Analytical Formulas for RKHS Best Predictors and Decomposition

When building the best predictor (either via interpolation or regularization) in the RKHS with K*_ANOVA, the function admits the decomposition: $$m(x) = m_0 + \sum_{i=1}^d m_i(x_i) + \sum_{i < j} m_{ij}(x_i, x_j) + \cdots + m_{1...d}(x)$$ with

$$m_I(x) = \left( \bigodot_{i \in I} k^i_0(x_i) \right)^T K^{-1} F$$

where $\odot$ denotes elementwise product across observed design points, $K$ is the Gram matrix, and $F$ is the vector of observations. This formula is notable for being non-recursive and mirror-symmetric to the functional ANOVA decomposition of the true function.

5. Sensitivity Analysis and Computation of Sobol’ Indices

The decomposition of $m$ yields mutually orthogonal ANOVA submodels, allowing a variance-based breakdown: $$\operatorname{Var}(m(X)) = \sum_{I \subseteq \{1,...,d\}} \operatorname{Var}( m_I( X_I ) )$$ with corresponding Sobol indices: $$S_I = \frac{ \operatorname{Var}(m_I(X_I)) }{ \operatorname{Var}(m(X)) }$$ A distinguishing contribution of (Durrande et al., 2011) is that these sensitivity indices can be computed analytically using explicit formulas involving the kernel integrals: $$S_I = \frac{ F^T K^{-1} \left( \bigodot_{i \in I} \Gamma_i \right) K^{-1} F }{ F^T K^{-1} \left\{ \bigodot_{i=1}^d ( 1_{n \times n} + \Gamma_i ) - 1_{n \times n} \right\} K^{-1} F }$$ with

$$\Gamma_i = \int_{D_i} k^i_0( x ) k^i_0( x )^T d\mu( x )$$

This closed-form computation—avoiding recursive integration—enables efficient and numerically stable estimation of variable importance in high-dimensional models.

6. Practical Applications and Case Studies

The RKHS-based functional ANOVA decomposition is especially suited for:

• Surrogate modeling of expensive simulators in engineering and physical sciences, where only a limited number of runs are feasible, and global interpretation in terms of variable effects is critical.
• Gaussian process regression (kriging) with ANOVA or K*_ANOVA kernels, ensuring each effect is interpretable and orthogonal, with main and interaction terms clearly isolated.
• Analytical and non-recursive computation of global sensitivity indices, critical for model simplification, variable screening, and understanding uncertainty contributions.

The methodology is validated on canonical test functions (e.g., Sobol g-function) and other benchmark problems, where the numerical results confirm the accurate isolation and quantification of main effects and higher-order interactions.

7. Mathematical and Algorithmic Framework

Key mathematical tools:

• Integral operator theory: enabling the identification of zero-mean subspaces and the construction of representers within RKHS.
• Tensor product structure of kernels: facilitating multivariate decomposition as a sum over all subsets, aligned with the functional ANOVA form.
• Closed-form interpolant and regularization solutions: allowing for both exact and regularized fits in the RKHS, with the regularization parameter $\lambda$ controlling noise levels.

Key algorithmic steps involve:

• Construction of centered univariate kernels $k^i_0$ using kernel integral formulas.
• Assembly of multivariate K*_ANOVA kernel.
• Computation of Gram matrices and model predictions using powers of kernel integrals.
• Direct computation of sensitivity indices from fitted model coefficients, circumventing recursion.

8. Impact and Implications

The RKHS-centered approach to functional ANOVA decomposition in (Durrande et al., 2011) achieves:

• Orthogonal, interpretable, and non-recursive decomposition of fitted models.
• Efficient, closed-form variance and sensitivity index computation.
• Applicability to high-stakes domains (scientific, engineering) where model evaluation is costly and variable prioritization is essential.
• Seamless integration into existing kernel-based nonparametric regression frameworks.

This theoretical and practical framework has become foundational in surrogate modeling, global sensitivity analysis, and the interpretable design of kernel-based statistical learning models.