Dimensionality-Invariant Metrics

• Dimensionality-invariant metrics are measures for distance and similarity that depend solely on intrinsic features such as geometry and stable dimension, not on the ambient space.
• They employ constructions like metric space magnitude, compressive Mahalanobis learning, and Hassanat's similarity measure to mitigate the curse of dimensionality.
• These metrics are applied in high-dimensional analysis, metric learning, and generative model evaluation, ensuring robust performance across varied data modalities.

A dimensionality-invariant metric is a class of quantitatively-defined measures of distance, dissimilarity, or diversity whose values or statistical properties do not depend on the ambient (extrinsic) dimension of the input space, but rather on intrinsic features such as the geometry, stable dimension, or the metric structure of the data. Such metrics are relevant in high-dimensional and geometric data analysis, metric learning, and diversity measurement for latent representations. A dimensionality-invariant metric may refer to a strictly isometry-invariant function (depending solely on pairwise distances), a similarity function whose statistical error or distortion bounds depend only on the intrinsic or numeric dimension of the data, or a metric for which large discrepancies in feature vectors are compressed to mitigate the curse of dimensionality.

1. Mathematical Definitions and Core Constructs

A central example is metric space magnitude, which defines the effective size or diversity of a finite metric space $X = \{x_1, \dots, x_n\}$ via the function

$$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$

where $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$ is the similarity matrix at scale $t>0$ and $d(x_i, x_j)$ is the metric (Limbeck et al., 2023). This construction is invariant under isometries and embedding into different Euclidean spaces of any dimension.

Another approach is compressive Mahalanobis metric learning, where a metric is defined in a compressed space obtained by a random projection $R \in \mathbb{R}^{k \times d}$ with $k \ll d$: $\widetilde{x} = R x \in \mathbb{R}^k$ A full-rank Mahalanobis matrix $M \in \mathbb{R}^{k \times k}$ is then learned in this low-dimensional space. The critical property is that the generalization error can be bounded in terms of the stable dimension $s(\mathcal{X})$, defined by

$$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$0

where $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$1 denotes the Gaussian width (Palias et al., 2023). The key consequence is that neither the ambient dimension $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$2 nor the number of extraneous features appear in these bounds.

A distinct yet related construction is the dimensionality-invariant similarity measure of Hassanat, defined for vectors $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$3 as

$$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$4

with

$$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$5

which bounds the maximum per-dimension distance, preventing domination by a single divergent coordinate (Hassanat, 2014).

2. Invariance Properties and Theoretical Guarantees

The most principled sense of dimensionality invariance is isometry invariance: if a metric depends only on pairwise distances $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$6, then for any isometric embedding $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$7, one obtains $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$8 for all $$\operatorname{Mag}_X(t) = \mathbf{1}^\top\,\zeta_{tX}^{-1}\,\mathbf{1}$$9 (Limbeck et al., 2023). Consequently, the associated distance

$\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$0

is also dimension-invariant.

For Mahalanobis metrics learned after compression, the generalization error admits high-probability upper bounds independent of $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$1: $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$2 This reflects statistical complexity governed by intrinsic geometry ($\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$3), not ambient dimension (Palias et al., 2023). When $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$4, the resulting metrics are operationally dimension-free.

The self-bounding structure of Hassanat's metric ($\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$5) ensures that the effect of large discrepancies in any coordinate saturates at 1, and the full-vector distance remains in $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$6 regardless of the raw scale or outlier dimensions (Hassanat, 2014).

3. Algorithms and Computational Complexity

For metric space magnitude, matrix inversion is $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$7 per scale. Practical implementations employ Cholesky factorization or conjugate-gradient solvers to leverage positive definiteness of the kernel matrix and attain higher stability and efficiency (Limbeck et al., 2023).

The compressive Mahalanobis metric pipeline consists of: (1) generating a random Gaussian projection $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$8, (2) compressing all input vectors to $\zeta_{tX}(i, j) = \exp(- t \, d(x_i, x_j))$9, and (3) learning $t>0$0 with a constraint $t>0$1 by minimizing a loss (e.g., LMNN or ITML). The main computational benefit is reducing the cost of metric learning from $t>0$2 complexity to $t>0$3 (Palias et al., 2023).

The dimensionality-invariant similarity measure is $t>0$4 to compute between two $t>0$5-dimensional vectors, involving $t>0$6 comparisons, a handful of additions, and at most $t>0$7 divisions (Hassanat, 2014). There are no tunable parameters beyond a shift constant (default $t>0$8).

4. Statistical and Empirical Advantages

A defining property of dimensionality-invariant metrics is that their generalization, stability, or distortion error is governed by intrinsic structure, not by the curse of dimensionality. For compressive Mahalanobis learning, the error-variance is governed by $t>0$9; error curves plateau as the projected dimension $d(x_i, x_j)$0 exceeds this intrinsic value, regardless of $d(x_i, x_j)$1 (Palias et al., 2023). Experiments on UCI datasets and synthetic ellipsoidal supports confirmed this effect.

Metric space magnitude enables rigorous, scale-dependent comparisons across datasets with heterogeneous or unknown ambient dimensions. Empirical results on text, image, and graph modality embeddings show that summary statistics such as $d(x_i, x_j)$2 distinguish diversity, mode collapse, and generative model coverage, with superior robustness and dimension-independence compared to $d(x_i, x_j)$3-NN-graph methods (Limbeck et al., 2023).

On benchmarks, the Hassanat metric outperformed Euclidean and Manhattan distances in $d(x_i, x_j)$4-NN classification accuracy for datasets with a mix of positive, negative, and varying-scale features, directly mitigating the issue of single-feature domination (Hassanat, 2014).

5. Applications and Broader Impact

Dimension-invariant metrics underpin robust algorithms in metric learning, unsupervised embedding evaluation, and geometric data analysis. Metric space magnitude offers an interpretable and rigorous diversity statistic across representation learning scenarios (including text, vision, and graphs), supporting both automated diversity estimation and evaluation of generative models (Limbeck et al., 2023).

Compressive Mahalanobis learning provides an explicit, practical route to low-sample-complexity supervised metric learning in high-dimensional spaces where the data support is concentrated near a lower-dimensional manifold or subspace (Palias et al., 2023). Dimensionality-invariant metrics such as $d(x_i, x_j)$5 are directly relevant to content-based retrieval, time-series classification, and clustering in mixed-domain or noisy-feature settings (Hassanat, 2014).

A plausible implication is that in large-scale machine learning pipelines subject to feature explosion or adversarial perturbations, adopting metrics or learning schemes with proven intrinsic-dimension dependence may yield enhanced robustness and interpretability. Dimensionality-invariant metrics also enable direct comparison and evaluation across models or modalities without the confound of differing latent dimensions, crucial in modern embedding-based representations.

6. Limitations and Future Directions

While dimensionality-invariant metrics resolve the dependence on extrinsic features, they may require careful parameter selection (e.g., projection dimension $d(x_i, x_j)$6 in compressive learning, or scale parameter $d(x_i, x_j)$7 in magnitude) to balance computation and accuracy. For the Hassanat metric, advantages are less pronounced when features are well-normalized and of similar scale.

Ongoing research avenues include kernelized generalizations, data-driven tuning of metric constants, adaptation to non-Euclidean or structured domains (e.g., graphs or hyperbolic spaces), and scaling deterministic implementations for very large $d(x_i, x_j)$8 in magnitude computation. Empirical studies suggest further integration of such metrics with local sensitivity hashing, coresets, and compressed data structures remains promising for high-dimensional learning contexts (Palias et al., 2023, Limbeck et al., 2023, Hassanat, 2014).