Distance Covariance Regularisation Framework

• The Distance Covariance Regularisation Framework is a set of statistical methods that utilizes distance covariance as an objective or penalty to gauge and control dependencies between random vectors.
• It employs both U-statistic and V-statistic estimators, with a convex combination approach to achieve low mean squared error and robust performance across varied dependence structures.
• Applications span fairness-aware machine learning, high-dimensional network inference, and dimensionality reduction, leveraging efficient algorithms for scalable implementation.

Distance Covariance Regularisation Framework refers to a broad class of statistical and machine learning methodologies wherein distance covariance or its generalizations serve as objective functions, regularisation penalties, or core test statistics for measuring and controlling dependence between random vectors or functions. These frameworks leverage the unique ability of distance covariance to detect arbitrary (linear and nonlinear) dependencies, thus enabling rigorous decorrelation and independence criteria in a variety of advanced settings such as model selection, multivariate analysis, fairness enforcement, network inference, and inverse problems.

1. Mathematical Foundation and Generalizations

Distance covariance was introduced by Székely, Rizzo, and Bakirov as a measure of dependence between random elements $X \in \mathbb{R}^m$ and $Y \in \mathbb{R}^n$ based on the weighted $L^2$-distance between their joint and marginal characteristic functions. For populations:

$$V^2(X, Y) = \int_{\mathbb{R}^m} \int_{\mathbb{R}^n} |f_{X,Y}(s, t) - f_X(s)f_Y(t)|^2\, \mu(ds)\nu(dt)$$

where $\mu$ and $\nu$ are symmetric Lévy measures (generalizing the classical power-law weights in the Euclidean case), each associated with a continuous negative definite function (cndf) via the Lévy–Khintchine formula. For example, $$\Phi(x) = \int(1 - \cos\langle x, s \rangle)\, \mu(ds)$$. The measure is:

• Zero if and only if $X$ and $Y$ are independent.
• Well-defined under minimal integrability conditions, and generalizable to non-Euclidean metrics (e.g. Minkowski distance for $p \in [1,2]$).
• Admits equivalent matrix (double-centered) and inner-product (Hilbert space) formulations, allowing efficient computation and extension to functional or infinite-dimensional settings.

Generalizations further permit multivariate (distance multivariance), partial (partial distance correlation), and metric/Hilbert space-valued inputs by replacing Euclidean distances with arbitrary cndfs (Böttcher et al., 2017, Janson, 2019).

2. Estimation: U- and V-Statistics and Their Regularisation

There exist two principal estimators:

• The V-statistic estimator (Monroy-Castillo et al., 3 May 2024), based on double-centered distance matrices, is always computable and asymptotically unbiased, but can be positively biased under independence:

$$V_n^2(X, Y) = \frac{1}{n^2} \sum_{k=1}^n \sum_{l=1}^n A_{kl}B_{kl}$$

• The U-statistic estimator (Huo & Székely, 2016) is unbiased, constructed via U-centering, but admits negative values for the squared estimate under independence or small sample sizes. It is given by:

$$\Omega_n = \frac{1}{n(n-3)} \sum_{i \neq j} \widetilde{A}_{ij}\widetilde{B}_{ij},$$

where $\widetilde{A}$ and $\widetilde{B}$ are U-centered versions (Huo et al., 2014).

A convex linear combination of the two estimators has been shown to yield low mean squared error (MSE) regardless of the dependence structure:

$$\operatorname{dCor}_\lambda = \lambda \operatorname{dCorU} + (1-\lambda)\operatorname{dCorV}$$

with the optimal $\lambda$ estimated by bootstrap to balance the MSE (Monroy-Castillo et al., 3 May 2024).

Efficient algorithms (using sorting and dyadic updates or AVL trees) reduce complexity from $O(n^2)$ to $O(n\log n)$ (Huo et al., 2014).

3. Frameworks and Algorithmic Incorporation

Regularisation in Statistical Learning

Distance covariance is suitable as a penalty in regression and classification loss functions to enforce (near-)independence between outputs and specified variables (e.g., protected attributes in fairness applications or confounders in causal inference):

$$\mathcal{L}_{\text{reg}} = \mathcal{L}_{\text{data}} + \lambda\, \operatorname{dCov}^2(\hat{y}, S)$$

where $\mathcal{L}_{\text{data}}$ is the accuracy loss, $S$ represents protected or nuisance attributes, and $\lambda$ calibrates the trade-off (Lee et al., 9 Sep 2025). The penalty immediately extends to partial distance covariance when conditioning is required (Szekely et al., 2013).

Multivariate and Intersectional Extensions

To address dependence across multiple attributes (e.g., fairness gerrymandering), multivariate formulations are used:

• Joint distance covariance (JdCov) penalizes all marginal and joint dependencies:

$$\psi(\hat{y}, S_1, \ldots, S_d) = \widehat{\operatorname{JdCov}}^2(\hat{y}, S_1, \ldots, S_d)$$

but may introduce instability if protected attributes are correlated.

• Concatenated distance covariance (CCdCov) aggregates attributes into a single joint vector, providing stable regularization that captures joint dependence with predictions but not among protected attributes:

$$\psi(\hat{y}, S_1, \ldots, S_d) = \operatorname{CCdCov}(\hat{y}, (S_1, \ldots, S_d))$$

(Lee et al., 9 Sep 2025).

Calibration and Model Selection

Choosing the regularisation strength $\lambda$ is typically achieved via validation–set–level metrics which quantify group-level distributional similarity (e.g., Jensen–Shannon divergence between predicted distributions across groups) and prediction error (e.g., Ranked Probability Score, Poisson deviance). Models are trained for a grid of $\lambda$, and the fairness–accuracy trade-off curve guides selection.

4. Applications and Inference

Fairness in Machine Learning

Distance covariance penalties are adapted to ensure demographic parity across regression and classification models, including continuous, categorical, and intersectional protected attributes—addressing deficiencies of earlier approaches restricted to binary or single-type attributes (Lee et al., 9 Sep 2025). Empirical studies on real datasets (COMPAS recidivism, motor insurance claims) demonstrate mitigation of subgroup disparities without severe accuracy loss.

Network Inference and High-Dimensional Structure Learning

In high-dimensional settings (e.g., gene regulatory networks), distance covariance can be used to estimate conditional independence through the inversion of the distance correlation matrix, yielding robust graph recovery even when $p > n$ (Khoshgnauz, 2012, Ghanbari et al., 2016). Variants include the Distance Precision Matrix, which provides a direct nonlinear analogue of the Gaussian precision matrix.

Dimensionality Reduction and ICA

Distance covariance-based criteria are used for both supervised (maximize dependence between representation and response, regularize dependence between features and covariates) (Vepakomma et al., 2016) and unsupervised (independent component analysis) settings (Matteson et al., 2013), leveraging U-statistics estimation and optimization under minimal regularity conditions.

Functional and Manifold Data

Regularized representations of covariance operators in inverse problems and functional data analysis use distance covariance for penalizing deviations from spatial smoothness or for independence testing among stochastic processes (after careful discretization), with bootstrap procedures for inference (Lila et al., 2018, Dehling et al., 2018).

5. Extensions and Theoretical Insights

• Generalized kernels: By varying the underlying symmetric Lévy measures, the metric, and the cndf, the framework is applicable to spaces far beyond Euclidean, including Hilbert and other separable metric spaces (Böttcher et al., 2017, Janson, 2019). All core properties (independence characterization, positive semidefiniteness) are preserved.
• Partial and conditional dependence: Frameworks for partial distance correlation enable conditioning out confounding effects, with rigorous Hilbert space projections and sample estimators implemented via matrix inner products (Szekely et al., 2013).
• Optimization perspectives: Efficient ADMM or MM algorithms underpin scalable application in large and complex optimization problems, such as sparse covariance or regularized geodesic distance estimation (Xu et al., 2021, Edelstein et al., 2023).

6. Practical Implementation and Computational Issues

Fast algorithms and unbiased estimation allow use in large-scale and high-dimensional data contexts. The ability to design adaptive and robust estimators via convex combinations of U- and V-statistics, and to incorporate regularization into standard convex and MM optimization routines, enables practical application in machine learning systems and statistical toolkits. Multivariate and empirical extensions are implemented in open-source software such as the R packages “energy,” “pdcor,” and “multivariance.”

---

In summary, the Distance Covariance Regularisation Framework provides a mathematically rigorous, flexible, and computationally practical foundation for quantifying, penalizing, and inferring independence and decorrelation in complex modern statistical models. Its core properties—characterization of independence, multivariate and nonlinear capacity, adaptable estimator schemes, and algorithmic scalability—position it as a crucial tool for dependence control in diverse domains such as fairness-aware learning, high-dimensional inference, and functional/structural data analysis.