Distance-Based Approaches

• Distance-based approaches are methodologies that represent objects using specialized metrics or dissimilarities to enable optimization, learning, and inference across diverse applications.
• They employ a range of metrics such as Mahalanobis, Wasserstein, and domain-specific distances to effectively manage anisotropic noise, topological structure, and outlier resistance.
• Applications span from supervised learning and manifold embedding to data cleaning and ontology-based inference, demonstrating both scalability challenges and robust performance.

Distance-based approaches encompass a broad set of methodologies in which distances (or generalized dissimilarities) serve as primary mathematical, algorithmic, or statistical objects—informing optimization, learning, clustering, data quality, and inference. These approaches appear across a diversity of subfields: optimization and embedding, supervised and semi-supervised learning, clustering, time series analysis, data repair, and statistical hypothesis testing, among others. This article surveys the foundational concepts, principal methodologies, and significant empirical findings characterizing distance-based approaches in contemporary computational and statistical sciences.

1. Foundations and Core Principles

At the core of distance-based approaches lies the representation of objects—whether they are vectors, probability distributions, programs, structured terms, or sensor measurements—via distances defined by application-appropriate metrics or dissimilarities. These distance functions must often be mathematically structured (metric, positive-definite, invariant under group actions) or statistically robust (e.g., affine-invariant, outlier-resistant, density-sensitive) to yield effective algorithms.

Crucially, distances need not coincide with Euclidean or ℓₚ norms. Mahalanobis distance is employed to account for anisotropic noise and dependency structures (Lin et al., 2021). Wasserstein and sliced Wasserstein distances capture support geometry and are widely leveraged in probability-measure learning and generative modeling (Rakotomamonjy et al., 2018, Tran et al., 14 Mar 2025). Distance covariance, as in S²D²R, is used for model-free dimension reduction sensitive to nonlinear association (Ni et al., 2019). For hierarchical or structured data, term-based distances (e.g., Nienhuys–Cheng's, Estruch et al.'s) operate over XML trees or logic terms (Bedoya-Puerta et al., 2011). Emerging methods design specialized distances for program semantics in genetic programming (Galván et al., 2020) or for trajectory evaluation along lane graphs (Schmidt et al., 2023).

Distance-centric reasoning provides robustness (insensitivity to feature-scale/rotation), interpretability, and—when properly integrated—efficiency and scalability.

2. Distance-based Optimization and Embedding

Optimization over distances is fundamental to problems such as localization, sensor network calibration, and manifold embedding. The classical energy minimization

$$\min_{x_1,\dots,x_N}\sum_{i<j}w_{ij}(\|x_i - x_j\|^2 - d_{ij}^2)^2$$

is ubiquitous in geometric embedding. Recent research exploits this structure via physical or photonic analogues. Notably, the Canonical Transformation (CT) and Gain-Based Bifurcation (GBB) methods provide dual-primal and dynamical-system approaches for mapping distance-based energy functions onto optical hardware, enabling parallel, near-optimal solutions for high-dimensional embedding and network localization tasks (Li et al., 15 Jul 2025). CT leverages dual variables for a complementary function whose critical points coincide with the original s-stress landscape, while GBB capitalizes on natural bifurcation in gain-dissipative oscillators to converge to low-energy configurations. Both techniques exploit the equivalence between squared Euclidean geometry and complex-valued amplitude/phase representations.

In metric measure space, Wasserstein (OT) distances are rendered computationally tractable via one-dimensional projections (“slicing”), but projection alone may destroy topology. Tree-Sliced Wasserstein (TSW) and its distance-based generalization Db-TSW introduce tree-structured systems of lines and E(d)-invariant splitting maps; these effect finer preservation of positional/topological information while maintaining nearly linear scaling in data size and dimension (Tran et al., 14 Mar 2025).

3. Distance-based Learning and Classification

Supervised learning with distances comprises a broad array of paradigms:

• Template and Prototype Embeddings: The Distance Measure Machine constructs explicit m-dimensional embeddings of distributions by recording dissimilarities to reference (template) distributions, allowing low-error linear decision boundaries for empirical distributions, subject to population generalization guarantees. The Wasserstein distance, in particular, yields superior margins and computational efficiency compared to MMD or KL-based embeddings (Rakotomamonjy et al., 2018).
• Distance-based Neural Architectures: Recent geometric analyses demonstrate that neural networks naturally support both intensity-based (“activation magnitude”) and distance-based representations (proximity to learned prototypes). The OffsetL2 architecture directly parameterizes Mahalanobis distance responses to class prototypes, yielding stable, interpretable, and robust representations with superior avoidance of “dead nodes” (Oursland, 4 Feb 2025).
• Term/Tree-based Distances: For data described by structured representations (XML, logic terms), term distances such as those of Nienhuys–Cheng and Estruch et al. capture context, depth, and repetition, greatly improving k-NN classification when paired with strategic hierarchy construction (Bedoya-Puerta et al., 2011). Analogously, in multivariate and functional data, the bagdistance (based on halfspace depth) and projection-adjusted outlyingness underpin robust, affine-invariant transformations into “distance space”, leaving subsequent kNN classification robust to outliers and nonconvexity (Hubert et al., 2015).
• Time Series Classification (TSC): Elastic distances (DTW, TWED, ERP, LCSS) power 1-NN, give rise to distance feature embeddings (global, local/shapelet, MDS-based), and—through distance substitution or alignment-sum schemes—enable SVMs and kernel machines adapted for temporal structure. Notably, careful design is required to ensure positive-definiteness and efficiency, particularly in kernel-based methods (Abanda et al., 2018).
• Mixture-based Sample Distributions: For count data (e.g., microbiome OTU tables), modeling samples as mixtures (Poisson-Gamma, with explicit zero inflation) and then comparing entire per-sample distributions via L² PDF/CDF norms yields substantial gains in classification calibration and performance, especially under high sparsity (Shestopaloff et al., 2020).

4. Distance-based Clustering, Inference, and Data Quality

Distance-based clustering and inference decouple model specification from strict parametric assumptions:

• Bayesian Distance Clustering (BDC): BDC models the partial likelihood of pairwise within-cluster distances (not the complete data likelihood), enforcing Gamma-structured distance densities for nonparametric robustness. This approach mitigates sensitivity to kernel misspecification and performs strongly relative to both Gaussian and subspace clustering models in skewed or structurally mismatched settings (Duan et al., 2018).
• Consensus Distance-clustering with Feature Weights: Sharp implements consensus clustering with optimally calibrated attribute weights (via sparse or COSA weighting), using novel consensus stability scores derived directly from subsample co-membership matrices. This enhances cluster detection and feature selection robustness in high-dimensional genomics (Bodinier et al., 2023).
• Distance-based Data Cleaning: Data cleaning via distances generalizes beyond equality, incorporating metrics such as Euclidean, edit, and Jaccard, into rule discovery (metric FDs, matching dependencies), error detection (DBSCAN, LOF), repair, and imputation. Distance thresholds enable higher precision/recall, tolerating data heterogeneity at the cost of increased computational and parameter-tuning complexity (Sun et al., 2020).
• Repair Semantics in Ontology-based Access: The Distance-Based Inference Framework (DIF) defines syntactic metrics over repairs (maximal consistent subsets of facts), clusters repairs using spectral or metric-space clustering, and enables user-personalized inference, reducing combinatorial explosion and facilitating visualization (Prouté et al., 2019).
• Density-based Distances and Manifolds: Semi-supervised approaches replace ambient Euclidean distance with graph-based density-sensitive shortest-path distances, computed via specialized Dijkstra* algorithms. These methods penalize paths through sparse regions and support manifold-adaptive, density-aware learning (Bijral et al., 2012).

5. Distance-based Statistical Inference and Testing

Modern statistical inference unifies distance-based and score-based goodness-of-fit via integral probability metrics (IPMs):

• Any GoF test employing an IPM of the form

$$\sup_{f\in\mathcal F} |\mathbb{E}_{\text{data}} f - \mathbb{E}_{\text{model}} f|$$

(e.g., Kolmogorov–Smirnov, Wasserstein, MMD) can be recast as a maximized score from an exponentially tilted likelihood. This insight leads to unified nonparametric testing methodologies with well-characterized asymptotics and computationally tractable estimators (Huang et al., 23 Dec 2025).

• The semiparametric Kernelized Stein Discrepancy (SKSD) test, induced by kernelized Stein function classes, offers universally consistent, efficient GoF tests for ordinary or intractable models, including kernel exponential families and conditional Gaussians. Empirically, it achieves power on par with task-specific normality tests (Huang et al., 23 Dec 2025).
• For forensic science, indirect distance-based methods (e.g., vector dissimilarities plus logistic regression) outperform direct likelihood-ratio approaches, retaining discrimination without high-dimensional density estimation, and providing robust likelihood ratios for same-source/different-source testing (Rivals et al., 2023).

6. Distance-based Approaches in Program Evolution and Semantics

In genetic programming, semantic-based distances—quantified as the difference in behavioral output vectors between individuals—can be elevated from mere diversity control at the operator level to explicit multi-objective criteria. In multi-objective genetic programming (MOGP), embedding a semantic distance to a dynamically selected “pivot” solution as a third optimization objective (alongside classical metrics) consistently yields superior Pareto fronts, greater behavioral diversity, and improved performance, as compared to crossovers that inject semantic similarity (Galván et al., 2020).

7. Limitations, Open Problems, and Prospects

Distance-based approaches universally contend with trade-offs: computational cost (O(n²) pairwise distances), the need for efficient implementations (e.g., GPU-fused kernels, randomized embeddings), and parameter tuning (distance thresholds, metric selection, weighting). Ensuring mathematical properties (metricity, positive-definiteness, invariance) is often problem-specific and may require customized design or regularization.

Open problems include scalable and automatically calibrated distance selection, integration of multiple heterogeneous distance functions (e.g., deep-learned features for complex domains), interpretability in black-box models (e.g., kernel methods for time series), downstream implications for end-to-end learning (e.g., in differentiable pipelines), and transposability to novel data modalities (e.g., graphs, temporal events).

Nonetheless, distance-based frameworks remain central, versatile, and evolving components in data science, machine learning, statistics, signal processing, and domain-specific computational modeling. Recent advances in hardware-aware algorithms, statistical-theoretic unification, and robust high-dimensional distance learning continue to extend their reach and practical effectiveness.