---
title: Gaussian Process Distance Correlation (GPDC)
url: https://www.emergentmind.com/topics/gaussian-process-distance-correlation-gpdc
type: topic
---

# Gaussian Process Distance Correlation (GPDC)

Searching arXiv for the specified paper and closely related work to ground the article.
{"queries":[{"query":"ti:\"Using Distance Correlation for Efficient Bayesian Optimization\"","limit":5},{"query":"Gaussian process distance correlation Bayesian optimization arXiv","limit":10}]}
{"queries":[{"query":"2102.08993","limit":5}]}
Gaussian Process Distance Correlation, commonly abbreviated **GPDC**, denotes a Gaussian process–based Bayesian optimization and sequential design scheme in which **distance correlation** or **distance covariance** is used as the acquisition principle. In the primary arXiv source on the method, the overall scheme is introduced as **BDC** in the abstract and explicitly described in the text as **GP-DC**, i.e., a Gaussian Process–Distance Correlation approach [2102.08993]. The method addresses expensive black-box objectives and related experimental-design problems by drawing posterior samples from a Gaussian process surrogate and selecting the next query according to the statistical dependence between candidate observations and either the unknown maximum of the function or the global function configuration itself. Its defining claim is that this dependence-based criterion balances exploration and exploitation automatically and requires no explicit acquisition hyperparameter [2102.08993].

## 1. Conceptual scope and problem setting

GPDC is situated within standard **Gaussian process–based Bayesian optimization (GP-BO)**, but it is formulated for two distinct tasks. The first is **function estimation / experimental design**, where the goal is to reconstruct an unknown function globally with as few measurements as possible. The second is **black-box maximization**, where the goal is to identify $\max_x f(x)$ using as few evaluations as possible [2102.08993].

A distinguishing feature of the framework is that the function-estimation setting admits multiple observation modalities. The source describes **point observations** $f(x)$, **integral observations** with adjustable width in one and two dimensions, and **smoothed derivative observations** or coarse-grained gradients. The algorithm therefore chooses not only where to observe, but, in the multi-modality setting, also which observation width or modality to use at each step [2102.08993].

The proposed acquisition principle is motivated by the limitations of standard BO rules. The source contrasts GPDC with **EI**, **PI**, **GP-UCB**, **GP-MI**, **ES**, **PES**, and **MES/OPES**, emphasizing that many of these methods require explicit exploration parameters, entropy approximations, nested sampling, or numerical integration. GPDC instead uses distance-based dependence between posterior-derived random quantities as the basis for query selection. This gives the method an **information-theoretic in flavor** interpretation while remaining nonparametric and computationally simpler than mutual-information-based alternatives, according to the source description [2102.08993].

## 2. Statistical basis: distance covariance and distance correlation

The acquisition mechanism is built on **distance covariance** and **distance correlation** as introduced by Székely and collaborators, with the paper also considering a generalized form with exponent $0<\alpha<2$ in the function-estimation experiments [2102.08993]. At the population level, for random vectors $X \in \mathbb{R}^p$ and $Y \in \mathbb{R}^q$, distance covariance is defined through a weighted $L^2$ discrepancy between the joint characteristic function and the product of marginals:
\[
\mathcal{V}^2(X,Y) =
\int_{\mathbb{R}^{p+q}}
\frac{\left|\phi_{X,Y}(t,s)-\phi_X(t)\phi_Y(s)\right|^2}
{c_p c_q \,\|t\|_p^{1+p}\|s\|_q^{1+q}}
\, dt\, ds.
\]
Distance variance is obtained by setting the two arguments equal, and distance correlation is the normalized quantity
\[
\mathcal{R}(X,Y)=
\frac{\mathcal{V}(X,Y)}
{\sqrt{\mathcal{V}(X,X)\mathcal{V}(Y,Y)}},
\]
with the convention that it is zero when the denominator vanishes [2102.08993].

The properties used directly by GPDC are the ones emphasized in the source: $\mathcal{R}(X,Y)\in[0,1]$; $\mathcal{R}(X,Y)=0$ **if and only if** $X$ and $Y$ are statistically independent; the measure detects **any kind of dependence**, not only linear association; it applies in arbitrary dimensions; and it is nonparametric. The source also states that, in practice, the quantity can be estimated reliably with relatively few posterior samples, with experiments using approximately $M=200$ to $300$ samples [2102.08993].

For empirical computation, the method uses i.i.d. samples $\{(X_i,Y_i)\}_{i=1}^M$, pairwise distance matrices
\[
a_{ij}=\|X_i-X_j\|,\qquad b_{ij}=\|Y_i-Y_j\|,
\]
double-centering,
\[
A_{ij}=a_{ij}-\bar a_{i\cdot}-\bar a_{\cdot j}+\bar a_{\cdot\cdot},\qquad
B_{ij}=b_{ij}-\bar b_{i\cdot}-\bar b_{\cdot j}+\bar b_{\cdot\cdot},
\]
and then
\[
\widehat{\mathcal{V}^2(X,Y)}=\frac{1}{M^2}\sum_{i,j=1}^M A_{ij}B_{ij}.
\]
The corresponding empirical distance variances and distance correlation are formed analogously. The source specifies that computations were carried out numerically with the Python library `dcor`, and that **distance correlation**, rather than distance covariance, is used in the core selection rules for function estimation because covariance is scale-dependent across modalities [2102.08993].

## 3. Gaussian process surrogate and posterior sampling

GPDC inherits the surrogate model of standard GP regression. The unknown objective is modeled as
\[
f \sim \mathcal{GP}(m(x),k(x,x')),
\]
on a domain $\chi \subset \mathbb{R}^d$, with the source noting that a **Matérn kernel**, especially **Matérn 5/2**, is used consistently in the experiments, following Snoek et al. (2012) [2102.08993]. Given data
\[
D_t=\{(x_i,y_i)\}_{i=0}^t,\qquad y_i=f(x_i)+\epsilon_i,
\]
with Gaussian noise, the GP posterior at $x_*$ has predictive mean and variance
\[
\mu_t(x_*)=k_*^\top (K+\sigma_n^2 I)^{-1}\mathbf{y},
\]
\[
\sigma_t^2(x_*)=k(x_*,x_*)-k_*^\top (K+\sigma_n^2 I)^{-1}k_*,
\]
where $K$ is the Gram matrix, $k_*$ is the covariance vector to the test point, and $\sigma_n^2$ is the observation-noise variance [2102.08993].

Posterior sampling is central to GPDC. At each iteration, the method evaluates the GP posterior over a finite set of representative points $\{x_n\}_{n=1}^N$ and draws $M$ posterior function samples
\[
f_m=[f_m(x_1),\dots,f_m(x_N)]^\top,\qquad m=1,\dots,M.
\]
These samples induce empirical distributions for quantities such as the maximum value
\[
f_m^{\max}=\max_n f_m(x_n)
\]
and the maximizing location
\[
x_m^{\max}=\arg\max_n f_m(x_n),
\]
which then enter the distance-correlation acquisition [2102.08993].

The same GP formalism also supports non-point observation models. Because the GP is closed under linear operations, the source uses integrated kernels for line integrals, area integrals, and convolution-type smoothed derivatives. For integral observations, kernels of the form
\[
k(x,q)=\int k(x,x')G(x',q)\,dx',\qquad
k(q,q')=\iint k(x,x')G(x,q)G(x',q')\,dx\,dx'
\]
are employed, although the source notes that these integrations can be computationally expensive [2102.08993].

## 4. Acquisition constructions and algorithmic variants

The source distinguishes between a **function-estimation** version and a **maximization** version of GPDC. In both cases, the common principle is to choose the next query that is most strongly distance-correlated with a posterior-derived random object of primary interest.

For **function estimation**, the algorithm first draws posterior function samples $\{f_m\}_{m=1}^M$, each discretized on the representative grid. For every candidate width or modality $w_j$, it computes the predictive variance $\sigma_j^2(x)$, chooses the most uncertain location
\[
x_j=\arg\max_{x\in\{x_n\}}\sigma_j^2(x),
\]
and then evaluates, for each posterior sample, the hypothetical observation at $(x_j,w_j)$, denoted $f_m^{(j)}$. The acquisition score is the distance correlation between the collection of global function samples and the hypothetical observation values,
\[
\mathrm{DC}_j=\mathcal{R}\big(\{f_m\}_{m=1}^M,\{f_m^{(j)}\}_{m=1}^M\big),
\]
after which the selected modality is
\[
j^*=\arg\max_j \mathrm{DC}_j.
\]
The source interprets this as selecting the candidate observation that is most informative, in a dependence sense, about the entire function. It further notes that this induces an automatic coarse-to-fine behavior: wider integrals are often selected early, and narrower widths or point observations become favored later [2102.08993].

For **black-box maximization**, the basic variants are **GP-dCor** and **GP-dCov**. For each candidate point $x_n$, the method forms the vector of posterior sample values at that point,
\[
f_m^{(n)}=f_m(x_n),\qquad m=1,\dots,M,
\]
and computes either
\[
\mathrm{DC}(n)=\mathcal{R}\big(\{f_m^{\max}\},\{f_m^{(n)}\}\big)
\]
for GP-dCor or
\[
\mathrm{DC}(n)=\mathcal{V}\big(\{f_m^{\max}\},\{f_m^{(n)}\}\big)
\]
for GP-dCov. The next evaluation is then
\[
n^*=\arg\max_n \mathrm{DC}(n).
\]
This is explicitly described as selecting the point whose posterior value is most strongly dependent on the eventual maximum value of the function [2102.08993].

A second family of variants, **GP-dCor-X** and **GP-dCov-X**, replaces the sampled maximum values $\{f_m^{\max}\}$ with sampled argmax locations $\{x_m^{\max}\}$. Their corresponding acquisition scores are
\[
\mathcal{R}\big(\{x_m^{\max}\},\{f_m^{(n)}\}\big)
\]
or
\[
\mathcal{V}\big(\{x_m^{\max}\},\{f_m^{(n)}\}\big).
\]
These versions target dependence with the **location** of the maximizer rather than its value [2102.08993].

| Variant | Dependence target | Dependence measure |
|---|---|---|
| GP-dCor | $\{f_m^{\max}\}$ vs. $\{f_m(x_n)\}$ | Distance correlation |
| GP-dCov | $\{f_m^{\max}\}$ vs. $\{f_m(x_n)\}$ | Distance covariance |
| GP-dCor-X | $\{x_m^{\max}\}$ vs. $\{f_m(x_n)\}$ | Distance correlation |
| GP-dCov-X | $\{x_m^{\max}\}$ vs. $\{f_m(x_n)\}$ | Distance covariance |

The source repeatedly emphasizes that this acquisition rule contains **no tunable exploration parameter** analogous to EI’s $\xi$, UCB’s $\kappa$, or GP-MI’s $\alpha$. Exploration and exploitation are instead said to emerge from the dependence structure itself: highly uncertain regions can score well because they co-vary strongly with the posterior maximum, while already-explained or redundant regions score poorly because their values are weakly tied to changes in the maximum [2102.08993].

## 5. Computational profile and implementation considerations

GPDC relies on repeated posterior sampling over a representative grid and repeated evaluation of pairwise-distance statistics. For function estimation, the source identifies three main costs: GP posterior sampling over $N$ points, which requires a Cholesky decomposition of an $N\times N$ covariance matrix and therefore scales as $\mathcal{O}(N^3)$; integrated-kernel computations for non-point observations, which may involve multidimensional numerical integration; and distance-correlation computation, which costs roughly $\mathcal{O}(M^2)$ per evaluation because it forms $M\times M$ distance matrices [2102.08993].

For maximization, the same sampling cost appears, together with a stated cost of approximately $\mathcal{O}(NM^2)$ for scoring all $N$ candidate points once $M$ posterior samples have been drawn. The experiments use moderate values of $M$, typically about $200$ to $300$, and moderate candidate grids, which the source describes as feasible in practice [2102.08993].

The paper also provides specific practical recommendations. It recommends **Matérn 5/2** as a robust kernel choice and repeatedly updates kernel length scale and noise level using **leave-one-out cross-validation** as described in Rasmussen and Williams, Section 5.4.2. For implementation, posterior samples may be generated by drawing $z_m \sim \mathcal{N}(0,I_N)$ and setting
\[
f_m=\mu+Lz_m,
\]
where $L$ is the Cholesky factor of the posterior covariance. For large $N$ or higher-dimensional settings, the source recommends **random Fourier feature approximations** and spectral sampling methods, citing **SSGP** and Hernandez-Lobato (2014), in order to mitigate the cubic scaling of full-grid GP sampling [2102.08993].

A notable applied constraint appears in the two-dimensional terrain example. Because of the high cost of computing four-dimensional integrated kernels, the source states that each iteration and each candidate width used a $30\times 30$ mesh from which **100 points** were randomly subsampled as the representative set. This illustrates that the computational bottleneck of GPDC in multi-modality settings is often the kernel integration rather than the dependence statistic itself [2102.08993].

## 6. Empirical behavior, comparative position, and open questions

The empirical study in the source covers both sequential function estimation and conventional black-box maximization. For **synthetic 1D function estimation** with integral observations, the experiments use **64 random 1D functions** on $[0,1]$, **35 observations sequentially**, **120 equidistant grid points**, **$M=200$ posterior samples**, and widths
\[
w\in\{0,0.0875,0.175,\dots,0.7\}.
\]
The reported metric is the coefficient of determination $R^2$ between the GP posterior mean and the true function. The paper reports that GP-DC significantly outperforms both random choice and a pure point-observation max-variance policy until approximately step $32$, and that it selects broad widths early before shifting to narrower widths and eventually to point observations [2102.08993].

For **coarse-grained gradient observations**, the study uses **48 random 1D functions**, starts with observations at $f(0)$ and $f(1)$, and then performs **33 sequential observations** with widths
\[
w \in \{0.02,0.06,0.12,0.2,0.3,0.4,0.5,0.6,0.7\},
\]
using **$M=300$** and $\alpha=1$. The paper reports higher and more stable $R^2$ than random selection and a narrow-gradient baseline, again with a gradual shift toward smaller widths over time [2102.08993].

For the **Grand Canyon** terrain experiment, the data are a digital elevation model of size **$2800\times 2800$ pixels**, scaled to $[0,1]^2$, with disk integrals
\[
f_r(q)=\int_{\|x-q\|\le r} f(x)\,dx,\qquad
r\in\{0,0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4\}.
\]
The paper reports that a point-only policy performs worst, that GP-DC initially trails random selection up to about step $14$, and that GP-DC subsequently overtakes it. In the reported comparison, random selection needs **45 steps** to reach $R^2\ge 0.68$, whereas GP-DC needs approximately **30 steps**, described as about **30\% fewer** steps [2102.08993].

For **maximization on random 1D functions**, the study uses **256 random 1D functions**, **50 observations per function**, and compares the proposed GP-dCor, GP-dCov, GP-dCor-X, GP-dCov-X, and GP-MIS against **Random**, **VarMax**, **PI**, **EI**, **GP-UCB**, **GP-MI**, **MES**, and **Brent**. Regret is defined as
\[
\mathrm{Regret}_T=\max_x f(x)-\max_{1\le t\le T} y_t.
\]
The paper reports that all GP-DC variants outperform the baselines across the full horizon in average regret, with **GP-UCB** as the strongest baseline and with the GP-DC family exhibiting shorter error bars and lower accumulated regret [2102.08993].

For **standard 2D benchmark functions**—Himmelblau, Eggholder, Branin, and Goldstein–Price, optimized via their negatives—the source reports normalized median cumulative regret over steps $20$ to $50$, with the random policy fixed at $1.0$. The average normalized scores reported for the proposed methods are **0.49** for GP-dCor, **0.59** for GP-dCov, **0.39** for GP-dCor-X, **0.52** for GP-dCov-X, and **0.43** for GP-MIS, with **GP-dCor-X** identified as best overall across the four functions [2102.08993].

The paper’s principal caution is theoretical rather than empirical. It explicitly states that it does **not** provide formal regret bounds, convergence-rate analysis, or consistency proofs for the distance-correlation acquisitions. The justification is instead conceptual and empirical: if a candidate evaluation is nearly independent of the maximum under the current posterior, its distance correlation with that maximum will be near zero, whereas any functional dependence, linear or nonlinear, will raise the score. This suggests a close relationship to information-seeking acquisition design, but a formal theory comparable to GP-UCB’s regret analysis remains open [2102.08993].

Source: https://www.emergentmind.com/topics/gaussian-process-distance-correlation-gpdc