Scale-Invariant Distance Measures

• Scale-Invariant Distance Measure is a function that quantifies dissimilarity and remains unchanged under uniform scaling, ensuring comparisons are robust across varying units.
• It encompasses methods like Mahalanobis distance, RDAD, and tree-based techniques that adjust for heterogeneous scales in high-dimensional and noisy data.
• Applications include clustering, object detection, and anomaly detection, where maintaining invariant metrics leads to more reliable and interpretable results.

A scale-invariant distance measure is a function that quantifies separation or dissimilarity between mathematical objects—typically vectors, probability distributions, or structured data—in a manner unaltered under uniform scaling transformations. This property is fundamental for ensuring that distance-based comparisons or analyses remain valid when units, magnitudes, or reference frames vary, which is often critical in high-dimensional statistics, machine learning, signal processing, and theoretical physics. Various mathematical constructions yielding scale-invariant (or in some contexts, affinely invariant or duality-invariant) distances have been developed, each with distinctive technical underpinnings and applications.

1. Core Definitions and Theoretical Foundations

Scale-invariant distance measures are rigorously defined by their invariance under scaling: for appropriate objects (e.g., vectors $x, y$ in a normed space, or random variables with non-degenerate covariance), a distance $d$ is scale-invariant if for all $\alpha > 0$, $d(\alpha x, \alpha y) = d(x, y)$. Several formalizations extend this to affine or even broader transformation groups.

A canonical construction is the family on inner-product spaces proposed in (Galvan, 2014): $$d_p(x, y) = \frac{\|x - y\|}{(\|x\|^p + \|y\|^p)^{1/p}}$$ with limiting case

$$d_\infty(x, y) = \frac{\|x - y\|}{\max\{\|x\|, \|y\|\}}$$

which are all strictly scale-invariant for $p \in [1, \infty]$.

In the context of statistical dependence between random vectors $(X, Y)$, the affinely invariant distance correlation $\widetilde{\mathcal{R}}(X, Y)$ (Dueck et al., 2012) involves Mahalanobis-type whitening to ensure both scale and affine invariance. For high-dimensional geometric/combinatorial contexts, robust density-weighted filtrations such as RDAD (Siu et al., 2022) also realize scale invariance via conformal reweighting.

2. Principal Construction Paradigms

Prominent scale-invariant distances arise from several paradigms:

• Mahalanobis-based Measures: Whitening by the empirical or population covariance matrix yields distances with both rotational and scale invariance. For instance, in rotated object detection, the Mahalanobis Distance Loss (MDL) (Wen et al., 2022) is defined for bounding box vectors by

$d_M(x, y) = \sqrt{(x-y)^\top \Sigma^{-1} (x-y)}$

where $\Sigma$ captures covariance structure and is itself scale-covariant, so $d_M$ is unchanged under $x \mapsto \alpha x$.

• Distance Correlation (Affinely Invariant): The scale-invariant version (Dueck et al., 2012) of Székely–Rizzo–Bakirov distance covariance is

$$\widetilde{V}^2(X, Y) = V^2(\Sigma_X^{-1/2} X, \Sigma_Y^{-1/2} Y)$$

yielding a dependence measure that is not only scale-invariant, but truly affinely invariant.

• Tree-based and Combinatorial Methods: The expected separation depth in an Isolation Forest (Cortes, 2019) results in a metric

$d(x, y) = 2^{-(\bar h(x, y) - 1)/2}$

which is scale-invariant since the random split procedure respects the relative ordering, not the absolute magnitude, of features.

• Density-aware Topological Distances: The RDAD filtration (Siu et al., 2022) replaces the Euclidean distance with $f(y)^{1/D} \|x-y\|$, which remains invariant under $x \mapsto a x + b$ once the underlying density $f$ transforms accordingly. Quantile-based averages over the density-weighted metric then retain scale-invariance.
• Coordinate-wise and Mixed Measures: In the dimensionality-invariant metric (Hassanat, 2014), per-coordinate distances are compressed via min/max ratios bounded in $[0,1)$ to ensure no dimension can dominate due to scaling, yielding robustness in high-dimensional or heterogeneously scaled data.

3. Characteristic Properties and Metric Axioms

Most scale-invariant distances discussed are shown to satisfy key metric properties:

• Non-negativity: $d(x, y) \geq 0$
• Identity of Indiscernibles: $d(x, y) = 0$ iff $x = y$ (or, for statistical measures, iff variables are independent)
• Symmetry: $d(x, y) = d(y, x)$
• Triangle Inequality: Proven directly or via geometric embedding; in some conjectural families (Galvan, 2014), validity for all parameter regimes remains unproven but is supported by special cases and empirical evidence.

In measures such as affinely invariant distance correlation (Dueck et al., 2012), scale invariance is accompanied by invariance under translations and general linear transformations. For instance, for invertible $A, B$, the transformation $(X, Y) \mapsto (A X + a, B Y + b)$ leaves $\widetilde{\mathcal{R}}$ invariant.

4. Comparison of Notable Scale-Invariant Distances

Measure Type	Core Formula / Principle	Application Context
$d_p$-family (Galvan, 2014)	$\|x - y\|/(\|x\|^p + \|y\|^p)^{1/p}$	Machine learning, geometry
Mahalanobis	$\sqrt{(x-y)^\top \Sigma^{-1} (x-y)}$	Pattern recognition, detection
Affine Distance Corr.	$V^2(\Sigma_X^{-1/2}X, \Sigma_Y^{-1/2}Y)$	Dependence, statistics
RDAD (Siu et al., 2022)	$f(y)^{1/D} \|x-y\|$, quantile-averaged	TDA, topological inference
Isolation Forest Metric	$2^{-(\bar h(x,y) -1)/2}$	Nonlinear data, anomaly/cluster
Dim.-invariant (Hassanat, 2014)	Bounded min/max coordinate compression	Robust k-NN, hetero-scale data

Each measure is tailored to preserve scale-invariance while aligning with domain-specific requirements—rotation, density-adaptivity, or invariance to affine group action.

5. Exact Expressions and Statistical Asymptotics

For multivariate normal variables, the affinely invariant distance covariance $\widetilde{V}^2(X, Y)$ is an explicit function of canonical correlations between $X$ and $Y$, expressible via zonal polynomials and generalized hypergeometric functions with matrix arguments (Dueck et al., 2012). Asymptotic analysis reveals:

• In high dimensions with weak dependence, $$\widetilde{\mathcal{R}}^2(X, Y) \sim \frac{\operatorname{tr}(\Lambda)}{4 p q \sqrt{A(p) A(q)}}$$
• In large-$p,q$ regimes with $\operatorname{tr}(\Lambda_p) = O(p)$, $$\frac{p}{\operatorname{tr}(\Lambda_p)} \widetilde{\mathcal{R}}^2(X,Y) \to 1$$

Such results ensure that the scale-invariant properties persist under classical and high-dimensional statistical limits.

6. Computational Algorithms and Practicalities

Computational cost depends on distance type:

• Coordinate-wise measures (e.g., (Hassanat, 2014)) have $O(m)$ complexity per vector pair.
• Tree-based scale-invariant distances (Isolation Forest (Cortes, 2019)) require tree construction and bulk traversal ($O(n \log n)$ per tree, though practical implementations exploit sparsity).
• Affinely-invariant distances in statistics (Dueck et al., 2012) involve covariance matrix estimation and centering, with $O(n^2)$ for pairwise computation in $n$-sample settings.

Empirical methods such as RDAD (Siu et al., 2022) additionally rely on $k$-NN graph construction and density estimation, but remain algorithmically tractable for modern data set sizes.

7. Applications and Implications

Scale-invariant distances are integral in applications where magnitude should not confound geometric or statistical inference:

• High-dimensional clustering: Prevents dominant feature scaling from overwhelming clustering structure (Galvan, 2014).
• Object detection: Improves loss surface consistency across scales in rotated object detection (Wen et al., 2022).
• Dependence detection: Enables non-parametric, non-linear dependence assessment between multivariate time series (Dueck et al., 2012).
• Manifold and topological data analysis: Enhances sensitivity to small, high-density features robustly in the presence of noise (Siu et al., 2022).
• Anomaly detection and similarity search: Tree-based scale-invariant distances adapt to both numeric and categorical attributes, improving detection and retrieval efficacy (Cortes, 2019).

A common thread is the mitigation of distortion from heterogeneous scaling, outlier magnitudes, or affine transformation—essential for robust, interpretable, and transferable analysis in multivariate data environments.

---

Scale-invariant distances occupy a central role in modern data science, geometry, and mathematical statistics, providing foundational tools for robust, unit-insensitive, and transformation-invariant analysis. Continued advances refine the metric properties, computational schemes, and domain adaptivity of such measures in increasingly high-dimensional and heterogeneous data landscapes.