---
title: 'HDMR: High-Dimensional Model Representation'
url: https://www.emergentmind.com/topics/high-dimensional-model-representation-hdmr
type: topic
---

# HDMR: High-Dimensional Model Representation

High-Dimensional Model Representation (HDMR) is a hierarchical functional decomposition that expresses a multivariate function as a sum of component functions involving progressively higher-order interactions among input variables. This framework provides a rigorous mathematical and algorithmic foundation for reducing the effective dimensionality of complex input–output maps, especially when the function exhibits weak or sparse high-order interactions. HDMR underpins modern uncertainty quantification, sensitivity analysis, surrogate modeling, and interpretable machine learning by systematically attributing output variance across direct, correlative, and cooperative modes of parameter influence.

## 1. Mathematical Formulation and Theoretical Foundation

Let \( f: \Gamma \subset \mathbb{R}^d \to \mathbb{R} \) be square integrable with respect to a product measure \( \mu(\mathbf{x}) = \prod_{i=1}^d \mu_i(x_i) \). The HDMR, also called the ANOVA or Sobol’ expansion, uniquely decomposes \( f \) as
\[
f(\mathbf{x}) = f_0 + \sum_{i=1}^d f_i(x_i) + \sum_{i<j} f_{ij}(x_i, x_j) + \cdots + f_{1\cdots d}(x_1,\ldots,x_d)
\]
subject to hierarchical orthogonality:
\[
\int f_{u}(\mathbf{x}_u) \, d\mu_k(x_k) = 0 \;\; \forall k \in u \neq \emptyset.
\]
Here, each \( f_{u} \) is a unique effect associated with the interaction among the variables indexed by \( u \subseteq \{1,\ldots,d\} \). The zero-order term \( f_0 = \int_{\Gamma} f(\mathbf{x}) \, d\mu(\mathbf{x}) \) is the global mean. Univariate and bivariate terms are extracted by integrating out complementary variables and subtracting lower-order components:
\[
f_i(x_i) = \int f(\mathbf{x}) \, d\mu_{\sim i}(\mathbf{x}_{\sim i}) - f_0, 
\]
\[
f_{ij}(x_i, x_j) = \int f(\mathbf{x}) \, d\mu_{\sim\{i,j\}}(\mathbf{x}_{\sim\{i,j\}}) - f_i(x_i) - f_j(x_j) - f_0,
\]
and so on recursively.

For many practical cases, truncation at low order is justified: \( f(\mathbf{x}) \simeq f_0 + \sum_i f_i(x_i) + \sum_{i<j} f_{ij}(x_i, x_j) \), with higher-order interactions dropped if their impact is negligible.

## 2. Practical Constructions: Cut–HDMR, Anchoring, and Multiple Anchors

Direct computation of high-dimensional integrals is infeasible except for low \( d \). A common alternative is the cut–HDMR (“anchored ANOVA”), which replaces \( \mu \) by a Dirac measure at a reference anchor \( \bar{\mathbf{x}} \):
\[
\begin{aligned}
f_0 &= f(\bar{\mathbf{x}}), \\
f_i(x_i) &= f(\bar{x}_1,\ldots,\bar{x}_{i-1},x_i,\bar{x}_{i+1},\ldots,\bar{x}_d) - f_0, \\
f_{ij}(x_i,x_j) &= f(\ldots, x_i, \ldots, x_j, \ldots) - f_i(x_i) - f_j(x_j) - f_0, 
\end{aligned}
\]
and so on. Each low-dimensional “cut” is inexpensive even in large \( d \).

However, the cut–HDMR deteriorates away from the anchor. To mitigate this, clustering-based multi-anchor schemes are employed [2310.19277]. Given a dataset \( X \), Centroidal Voronoi Tessellation (CVT) partitions the input domain with \( L \) anchors \( \{\bar{\mathbf{x}}^{(l)}\} \). Inference at a new \( \mathbf{x} \) uses the expansion centered at the nearest anchor. This approach achieves substantially better locality and accuracy than naive averages or single-anchor expansions.

## 3. Estimation, Basis Construction, and Algorithmic Implementation

HDMR coefficients are typically parameterized in orthonormal bases, such as (tensorized) polynomials under prescribed measures. For each \( f_u \), expand
\[
f_u(\mathbf{x}_u) = \sum_{\beta} c_{u,\beta} \prod_{k \in u} \phi^{(k)}_{\beta_k}(x_k),
\]
where \( \{\phi^{(k)}_j\} \) are univariate orthonormal bases. The collective coefficients are determined via regression:

- **Ordinary Least Squares (OLS):** Assemble the design matrix \( \Phi \), fit by pseudoinverse.
- **Constrained Regression / D-MORPH:** Impose zero-mean (vanishing) constraints by explicit projection of coefficients onto the feasible set [2506.13939].
- **Sparse Basis Selection:** Grouped Least Angle Regression (LAR) adds HDMR groups incrementally, leveraging cross-validation for model selection [1302.7083].
- **Kriging and GP-based HDMR:** Model each component with a Gaussian process or PC-Kriging surrogate, leveraging adaptive sequential sampling for sample efficiency [2307.01436, 2012.02704, 2111.11001].

For cut–HDMR, Lagrange polynomial interpolation is often used to approximate component cuts efficiently. Term inclusion is typically governed by error analysis or adaptive energy thresholds, e.g., by pruning components with negligible variance contribution or cross-validated error improvement [1211.6510, 2503.11464].

## 4. Error Analysis, Adaptivity, and Scalability

HDMR reduces the curse of dimensionality by substituting interaction order for ambient dimension. The number of terms up to order \( r \) is \( \sum_{k=0}^r \binom{d}{k} \), which for moderate \( r \ll d \) is manageable. Truncation error under smoothness assumptions decays with the magnitude of omitted (higher-order) derivatives and the spatial “energy” (e.g., CVT energy in multi-anchor schemes [2310.19277]).

Adaptive schemes add or refine component functions or grid points only when their estimated effect (e.g., surplus, variance, or error improvement) exceeds pre-specified tolerances. Hybrid methods combine HDMR with sparse grid or collocation frameworks for high-dimensional uncertainty quantification [1211.6510, 2202.06555, 2503.11464]. Empirical and theoretical results demonstrate consistent subexponential or even polynomial scaling with dimension when interaction order is low and domain structure is exploited.

## 5. Sensitivity Analysis and Variance Attribution

HDMR allows for rigorous variance attribution:
\[
\mathrm{Var}[f] = \sum_{u} \mathrm{Var}[f_u] + 2\sum_{u<v} \mathrm{Cov}(f_u, f_v),
\]
with orthogonality implying the sum reduces to a direct partition in the independent input case. Sobol' indices and their HDMR generalizations (structural, correlative, and total indices) quantify the unique, correlative, and interactive contributions of variables or groups [2506.13939, 1807.10320].

HDMR-based approaches exceed classical regression or correlation in separating direct from mediated and cooperative effects, providing sharper diagnostics in, e.g., climate data analysis, global sensitivity analysis, or industrial design optimization [1811.03800, 2506.13939].

## 6. Applications and Hybridizations

HDMR is foundational in:

- **Uncertainty Quantification (UQ)**: Establishing robust surrogates for stochastic systems, random PDEs, and multiphysics simulations [1211.6510, 1302.7083].
- **Surrogate Modeling**: Hybrid schemes (PC-Kriging-HDMR, GP-HDMR, Hermite-HDMR) support meshless, mesh-based, or neural expansions, enabling accurate and interpretable metamodels in high dimensions [2307.01436, 2111.11001, 1905.04715, 1906.02005].
- **Machine Learning Interpretability**: HDMR enables “glass-box” characterization of black-box models, supporting variance decomposition of kernel machines, tree ensembles, and theoretical reduced-order analysis [1807.10320].
- **Dynamic Programming and Control**: Quadratic storage and computational requirements in Bellman recursion with HDMR truncation support policy synthesis in large-scale control [1210.0706].
- **Dimension Reduction for Manifold Learning**: HDMR-based graph embedding provides explicit, nonlinear, out-of-sample embeddings for hyperspectral and high-dimensional remote sensing data [2111.14680].

Benchmarks confirm state-of-the-art efficiency and robustness, including 100+ dimensional models in dynamic macroeconomics [2503.11464, 2202.06555], meshless PDE solvers at \(d=30\) [1905.04715], and high-dimensional molecular potentials [2111.11001, 2012.02704]. HDMR further enables variable selection, model order selection, and kernel parameter optimization in sparse-data regimes [2112.01374].

## 7. Limitations and Ongoing Developments

The accuracy of low-order HDMR truncations depends critically on the sparsity of intrinsic interactions in the target function. If high-order couplings are significant, higher truncation order or alternative representations may be required, increasing complexity. Component-order selection, anchor choice, and adaptive refinement strategies are areas of active methodological research.

Extensions incorporate:
- Adaptive dimension reduction (via preliminary Sobol’ screening or variance-based partitioning [1211.6510]),
- Hybrid and hierarchical collocation with multiscale solvers,
- Deep-learning–based sub-surrogate representations,
- Global optimization strategies that exploit the hierarchical decoupling paradigm [1811.03800],
- Robustification to non-Gaussian noise and heteroskedastic error structures.

HDMR thus synthesizes classical ANOVA, modern regression methodology, and advanced computational strategies into a unified analysis and approximation paradigm for high-dimensional, weakly coupled systems [2310.19277, 2506.13939, 2012.02704, 2503.11464, 1807.10320].

Source: https://www.emergentmind.com/topics/high-dimensional-model-representation-hdmr