HDMR: High-Dimensional Model Representation

• HDMR is a hierarchical functional decomposition technique that breaks multivariate functions into additive low-order and interaction terms for efficient uncertainty quantification.
• It isolates direct, correlative, and cooperative parameter effects, enabling clearer variance attribution and improved surrogate construction.
• Adaptive schemes and multi-anchor methods in HDMR enhance scalability and robustness for high-dimensional sensitivity analysis and dynamic system simulations.

High-Dimensional Model Representation (HDMR) is a hierarchical functional decomposition that expresses a multivariate function as a sum of component functions involving progressively higher-order interactions among input variables. This framework provides a rigorous mathematical and algorithmic foundation for reducing the effective dimensionality of complex input–output maps, especially when the function exhibits weak or sparse high-order interactions. HDMR underpins modern uncertainty quantification, sensitivity analysis, surrogate modeling, and interpretable machine learning by systematically attributing output variance across direct, correlative, and cooperative modes of parameter influence.

1. Mathematical Formulation and Theoretical Foundation

Let $f: \Gamma \subset \mathbb{R}^d \to \mathbb{R}$ be square integrable with respect to a product measure $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$. The HDMR, also called the ANOVA or Sobol’ expansion, uniquely decomposes $f$ as

$$f(\mathbf{x}) = f_0 + \sum_{i=1}^d f_i(x_i) + \sum_{i<j} f_{ij}(x_i, x_j) + \cdots + f_{1\cdots d}(x_1,\ldots,x_d)$$

subject to hierarchical orthogonality: $$\int f_{u}(\mathbf{x}_u) \, d\mu_k(x_k) = 0 \;\; \forall k \in u \neq \emptyset.$$ Here, each $f_{u}$ is a unique effect associated with the interaction among the variables indexed by $u \subseteq \{1,\ldots,d\}$. The zero-order term $$f_0 = \int_{\Gamma} f(\mathbf{x}) \, d\mu(\mathbf{x})$$ is the global mean. Univariate and bivariate terms are extracted by integrating out complementary variables and subtracting lower-order components: $$f_i(x_i) = \int f(\mathbf{x}) \, d\mu_{\sim i}(\mathbf{x}_{\sim i}) - f_0,$$

$$f_{ij}(x_i, x_j) = \int f(\mathbf{x}) \, d\mu_{\sim\{i,j\}}(\mathbf{x}_{\sim\{i,j\}}) - f_i(x_i) - f_j(x_j) - f_0,$$

and so on recursively.

For many practical cases, truncation at low order is justified: $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$0, with higher-order interactions dropped if their impact is negligible.

2. Practical Constructions: Cut–HDMR, Anchoring, and Multiple Anchors

Direct computation of high-dimensional integrals is infeasible except for low $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$1. A common alternative is the cut–HDMR (“anchored ANOVA”), which replaces $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$2 by a Dirac measure at a reference anchor $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$3: $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$4 and so on. Each low-dimensional “cut” is inexpensive even in large $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$5.

However, the cut–HDMR deteriorates away from the anchor. To mitigate this, clustering-based multi-anchor schemes are employed (Xiong et al., 2023). Given a dataset $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$6, Centroidal Voronoi Tessellation (CVT) partitions the input domain with $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$7 anchors $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$8. Inference at a new $\mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i)$9 uses the expansion centered at the nearest anchor. This approach achieves substantially better locality and accuracy than naive averages or single-anchor expansions.

3. Estimation, Basis Construction, and Algorithmic Implementation

HDMR coefficients are typically parameterized in orthonormal bases, such as (tensorized) polynomials under prescribed measures. For each $f$0, expand

$f$1

where $f$2 are univariate orthonormal bases. The collective coefficients are determined via regression:

• Ordinary Least Squares (OLS): Assemble the design matrix $f$3, fit by pseudoinverse.
• Constrained Regression / D-MORPH: Impose zero-mean (vanishing) constraints by explicit projection of coefficients onto the feasible set (Gao et al., 16 Jun 2025).
• Sparse Basis Selection: Grouped Least Angle Regression (LAR) adds HDMR groups incrementally, leveraging cross-validation for model selection (Mathelin, 2013).
• Kriging and GP-based HDMR: Model each component with a Gaussian process or PC-Kriging surrogate, leveraging adaptive sequential sampling for sample efficiency (Zhang et al., 2023, Ren et al., 2020, Sasaki et al., 2021).

For cut–HDMR, Lagrange polynomial interpolation is often used to approximate component cuts efficiently. Term inclusion is typically governed by error analysis or adaptive energy thresholds, e.g., by pruning components with negligible variance contribution or cross-validated error improvement (Jiang et al., 2012, Eftekhari et al., 14 Mar 2025).

4. Error Analysis, Adaptivity, and Scalability

HDMR reduces the curse of dimensionality by substituting interaction order for ambient dimension. The number of terms up to order $f$4 is $f$5, which for moderate $f$6 is manageable. Truncation error under smoothness assumptions decays with the magnitude of omitted (higher-order) derivatives and the spatial “energy” (e.g., CVT energy in multi-anchor schemes (Xiong et al., 2023)).

Adaptive schemes add or refine component functions or grid points only when their estimated effect (e.g., surplus, variance, or error improvement) exceeds pre-specified tolerances. Hybrid methods combine HDMR with sparse grid or collocation frameworks for high-dimensional uncertainty quantification (Jiang et al., 2012, Eftekhari et al., 2022, Eftekhari et al., 14 Mar 2025). Empirical and theoretical results demonstrate consistent subexponential or even polynomial scaling with dimension when interaction order is low and domain structure is exploited.

5. Sensitivity Analysis and Variance Attribution

HDMR allows for rigorous variance attribution: $f$7 with orthogonality implying the sum reduces to a direct partition in the independent input case. Sobol' indices and their HDMR generalizations (structural, correlative, and total indices) quantify the unique, correlative, and interactive contributions of variables or groups (Gao et al., 16 Jun 2025, Bastian et al., 2018).

HDMR-based approaches exceed classical regression or correlation in separating direct from mediated and cooperative effects, providing sharper diagnostics in, e.g., climate data analysis, global sensitivity analysis, or industrial design optimization (Luo et al., 2018, Gao et al., 16 Jun 2025).

6. Applications and Hybridizations

HDMR is foundational in:

• Uncertainty Quantification (UQ): Establishing robust surrogates for stochastic systems, random PDEs, and multiphysics simulations (Jiang et al., 2012, Mathelin, 2013).
• Surrogate Modeling: Hybrid schemes (PC-Kriging-HDMR, GP-HDMR, Hermite-HDMR) support meshless, mesh-based, or neural expansions, enabling accurate and interpretable metamodels in high dimensions (Zhang et al., 2023, Sasaki et al., 2021, Luo et al., 2019, Nguyen-Thanh et al., 2019).
• Machine Learning Interpretability: HDMR enables “glass-box” characterization of black-box models, supporting variance decomposition of kernel machines, tree ensembles, and theoretical reduced-order analysis (Bastian et al., 2018).
• Dynamic Programming and Control: Quadratic storage and computational requirements in Bellman recursion with HDMR truncation support policy synthesis in large-scale control (Pištěk, 2012).
• Dimension Reduction for Manifold Learning: HDMR-based graph embedding provides explicit, nonlinear, out-of-sample embeddings for hyperspectral and high-dimensional remote sensing data (Taskin et al., 2021).

Benchmarks confirm state-of-the-art efficiency and robustness, including 100+ dimensional models in dynamic macroeconomics (Eftekhari et al., 14 Mar 2025, Eftekhari et al., 2022), meshless PDE solvers at $f$8 (Luo et al., 2019), and high-dimensional molecular potentials (Sasaki et al., 2021, Ren et al., 2020). HDMR further enables variable selection, model order selection, and kernel parameter optimization in sparse-data regimes (Manzhos et al., 2021).

7. Limitations and Ongoing Developments

The accuracy of low-order HDMR truncations depends critically on the sparsity of intrinsic interactions in the target function. If high-order couplings are significant, higher truncation order or alternative representations may be required, increasing complexity. Component-order selection, anchor choice, and adaptive refinement strategies are areas of active methodological research.

Extensions incorporate:

• Adaptive dimension reduction (via preliminary Sobol’ screening or variance-based partitioning (Jiang et al., 2012)),
• Hybrid and hierarchical collocation with multiscale solvers,
• Deep-learning–based sub-surrogate representations,
• Global optimization strategies that exploit the hierarchical decoupling paradigm (Luo et al., 2018),
• Robustification to non-Gaussian noise and heteroskedastic error structures.

HDMR thus synthesizes classical ANOVA, modern regression methodology, and advanced computational strategies into a unified analysis and approximation paradigm for high-dimensional, weakly coupled systems (Xiong et al., 2023, Gao et al., 16 Jun 2025, Ren et al., 2020, Eftekhari et al., 14 Mar 2025, Bastian et al., 2018).