Distance-Based Significance Analysis

• Distance-Based Significance Analysis is a methodological template that transforms geometric distances into inferential statistics through calibrated simulation and asymptotic modeling.
• The framework is applied across diverse domains such as hallucination detection, gene-expression clustering, and manifold comparisons, highlighting its adaptability.
• By employing techniques like Gaussian KDE estimation, self-normalization, and extreme-value theory, the approach provides formal significance testing beyond ad hoc thresholds.

A distance-based significance analysis framework, as the term is used across several recent methodological lines, can be understood as a class of inferential schemes in which distances, distance profiles, or distance-derived summaries are the primary statistical object, and significance is assessed by comparing their observed behavior to class-conditional, null, or scale-indexed reference laws. In these formulations, distances are not merely descriptive; they are treated as random variables, quadratic forms, spectral signatures, or scale-dependent fields from which one derives likelihoods, p-values, confidence intervals, or decision rules (Ricco et al., 10 Feb 2025, Dette et al., 2024, Liu et al., 30 Oct 2025, Shnitzer et al., 2022).

1. Conceptual scope

A common structural pattern appears across otherwise disparate problems. In hallucination detection, distances between response embeddings are analyzed as empirical distributions that differ between hallucinated and non-hallucinated content (Ricco et al., 10 Feb 2025). In high-dimensional, low-sample size partially labeled data, significance is assessed through a 2-means Cluster Index built from squared Euclidean distances and calibrated under a Gaussian null (Lu et al., 2015). In infinite-dimensional dependence testing, the object of interest is the distance correlation, and the null hypothesis is formulated as a thresholded relevance statement, $H_0^{\rm rel}: \mathrm{dcor}(X,Y) \le \Delta$ versus $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$ (Dette et al., 2024). In two-dimensional scale-space inference, significance is attached to maxima of standardized derivative fields indexed by location and smoothing scale, with extreme-value limits used for valid testing (Liu et al., 30 Oct 2025). In PDDA, pairwise distances along cumulative trajectories define scale laws for Hurst analysis (Soriano et al., 19 May 2026). In LES, distances between datasets are induced by regularized log-eigenvalue descriptors of diffusion operators and interpreted as lower bounds of a log-Euclidean metric on SPD matrices (Shnitzer et al., 2022).

Instantiation	Distance object	Significance mechanism
Hallucination detection (Ricco et al., 10 Feb 2025)	Minkowski distances in embedding space	Wilcoxon rank-sum, KL divergence, KDE likelihood aggregation
SigPal (Lu et al., 2015)	2-means Cluster Index from squared Euclidean distances	Gaussian null simulation and empirical p-value
Relevant dependence testing (Dette et al., 2024)	Distance correlation in separable metric spaces of negative type	Self-normalized pivotal test for $H_0^{\rm rel}$
Advanced SSS (Liu et al., 30 Oct 2025)	Maxima of standardized slope or curvature fields	Extreme-value theory and controlled Type I error
PDDA (Soriano et al., 19 May 2026)	Pairwise distances of cumulative trajectories	Scaling exponents from $R_D(n)$, $M_2(\tau)$, and recurrence
LES (Shnitzer et al., 2022)	Log-eigenvalue distances between diffusion operators	Lower bound of the log-Euclidean metric

Taken together, these formulations suggest that a distance-based significance analysis framework is not a single algorithm but a methodological template: choose a representation, define a distance-derived statistic, estimate its reference behavior, and then decide whether the observed configuration is significant relative to that reference.

2. Statistical architecture

The most explicit abstract formulation is given in the hallucination-detection work, where the method is summarized as a general pattern: define a distance-based statistic, estimate class-conditional distributions of that statistic, use distances from a test point to a reference set as evidence, and make a significance or likelihood-based decision by comparing class scores (Ricco et al., 10 Feb 2025). There, embeddings of keyword sets extracted from LLaMA responses are treated as points in $\mathbb{R}^{768}$, and intra-class Minkowski distances

$$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$

are elevated to the status of random samples whose distributions are tested, summarized, and modeled. For $q=64$ questions, $r \in \{4,6,8,10,12,14,16\}$ responses per question, $n \in \{1,\dots,10\}$ keywords, and $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$0, the framework computes all $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$1 intra-class distances per class, then analyzes medians, quartiles, outliers, KL divergence, and Wilcoxon p-values. The Wilcoxon rank-sum test yields $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$2-values $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$3 for all tested configurations, and the median difference

$H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$4

is consistently positive, indicating that hallucinated embeddings are more spread out than non-hallucinated embeddings (Ricco et al., 10 Feb 2025).

A different but structurally analogous architecture appears in SigPal. There the statistic is the 2-means Cluster Index

$H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$5

which is a normalized within-class over total-distance quantity. Labels for unlabeled points are completed by a semi-supervised method, then the observed $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$6 is calibrated against a simulated null obtained from a single Gaussian model with eigenvalue-based covariance estimation. The empirical p-value is

$H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$7

with rejection for small observed cluster index (Lu et al., 2015).

In the dependence-testing setting, the architecture is threshold-based rather than class-based. The null is not exact independence but bounded dependence: $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$8 This replaces a point null by a relevance null. The test statistic

$H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$9

uses a self-normalizer $H_0^{\rm rel}$0 constructed from the sequential distance-correlation process, yielding a pivotal limit law driven by a Brownian functional rather than by an estimated long-run variance (Dette et al., 2024).

These examples differ in calibration machinery—nonparametric likelihood modeling, parametric bootstrap, and self-normalized asymptotics—but share the same architecture: distances are transformed from geometry into inference.

3. Distance objects and representational spaces

The framework has been instantiated in several representational regimes, each with its own geometry.

In embedding space, the hallucination work uses BERT embeddings of keyword sets, with KeyBERT extracting $H_0^{\rm rel}$1 keywords and the resulting set mapped to a single 768-dimensional vector. The core statistic is then a Minkowski distance with $H_0^{\rm rel}$2, $H_0^{\rm rel}$3, or $H_0^{\rm rel}$4, chosen partly because fractional $H_0^{\rm rel}$5 can better differentiate points in high-dimensional settings (Ricco et al., 10 Feb 2025).

In Euclidean sample space, SigPal uses squared Euclidean distances implicitly through the cluster index. The within-class sum of squares can be rewritten via pairwise distances,

$H_0^{\rm rel}$6

so the test is fundamentally distance-based even though it is expressed as a variance ratio (Lu et al., 2015).

In general metric spaces of negative type, the dependence-testing framework uses the distance covariance kernel

$H_0^{\rm rel}$7

from which $H_0^{\rm rel}$8 and the normalized $H_0^{\rm rel}$9 are defined. This broadens the admissible data types to Euclidean and functional data, provided the spaces are separable metric spaces of negative type (Dette et al., 2024).

In scale-space image analysis, the primary object is not a scalar distance but a field of distance- or scale-indexed derivative statistics. Local quadratic regression with Gaussian kernel bandwidth $R_D(n)$0 yields directional slope and curvature estimators such as

$R_D(n)$1

and

$R_D(n)$2

Here the bandwidth is itself a spatial distance parameter controlling which neighborhoods contribute to the statistic (Liu et al., 30 Oct 2025).

In PDDA, the foundational object is the distance matrix of the cumulative trajectory. For a zero-mean stationary process with cumulative deviate path $R_D(n)$3, distances are

$R_D(n)$4

or, in the multivariate case,

$R_D(n)$5

From this single object one derives the diameter $R_D(n)$6, the mean-squared displacement

$R_D(n)$7

and recurrence probabilities $R_D(n)$8 (Soriano et al., 19 May 2026).

In manifold comparison, LES represents each dataset by a diffusion operator $R_D(n)$9, regularizes it by $M_2(\tau)$0, retains the top $M_2(\tau)$1 eigenvalues, and defines the descriptor

$M_2(\tau)$2

The inter-dataset distance is then

$M_2(\tau)$3

a truncated lower bound of the log-Euclidean metric on SPD matrices (Shnitzer et al., 2022).

A plausible implication is that the framework is most naturally specified by first choosing a representation space—embedding, Euclidean sample space, metric space, scale space, trajectory space, or operator space—and only then choosing the distance-derived statistic.

4. Inference, calibration, and decision rules

The decision layer is where the framework becomes a significance procedure rather than a geometric summary.

The hallucination detector is a class-conditional generative model over one-dimensional distance distributions. For each configuration $M_2(\tau)$4, two Gaussian KDEs are fitted: $M_2(\tau)$5 Given a test embedding $M_2(\tau)$6, distances to all hallucinated and non-hallucinated reference embeddings are computed, converted into log-likelihoods, and aggregated into

$M_2(\tau)$7

The response is classified as hallucinated if $M_2(\tau)$8. This is explicitly described as a distance-based likelihood-ratio-like rule (Ricco et al., 10 Feb 2025).

SigPal uses simulation under a Gaussian null. After semi-supervised label completion, the observed cluster index is compared with $M_2(\tau)$9 values computed on null datasets with randomly revealed labels and the same semi-supervised assignment mechanism. The framework is therefore distance-based and model-based simultaneously: the statistic is a distance ratio, while significance is obtained by parametric bootstrap (Lu et al., 2015).

The relevant-dependence framework replaces variance estimation by self-normalization. Its crucial object is

$\mathbb{R}^{768}$0

which leads to the pivotal ratio

$\mathbb{R}^{768}$1

where

$\mathbb{R}^{768}$2

Rejection occurs when $\mathbb{R}^{768}$3, with $\mathbb{R}^{768}$4 the corresponding Brownian-functional quantile. The same construction yields

$\mathbb{R}^{768}$5

interpreted as the minimum practically significant dependency supported by the data at level $\mathbb{R}^{768}$6 (Dette et al., 2024).

Advanced SSS uses random-field extremes rather than per-location tests. Standardized derivative statistics $\mathbb{R}^{768}$7 are treated as Gaussian random fields on a lattice, and maxima are calibrated through a Gumbel limit of the form

$\mathbb{R}^{768}$8

For slope and curvature detection, this yields simultaneous thresholds over all image locations at a fixed scale, with Bonferroni used only over a small number of directions (Liu et al., 30 Oct 2025).

PDDA uses a different inferential logic. There the decisive quantity is not a p-value but a slope in log-log coordinates. For MSD-PDDA,

$\mathbb{R}^{768}$9

while for R/S-PDDA,

$$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$0

The simulations provide bias, SD, and RMSE rather than closed-form test thresholds, but the paper explicitly notes that the estimators can serve as test statistics for $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$1 by pairing them with simulation-based standard errors or bootstrap intervals (Soriano et al., 19 May 2026).

5. Empirical instantiations

The hallucination framework is evaluated on 64 time-sensitive questions about events between September 2022 and September 2023, with LLaMA2-7B treated as hallucinating and LLaMA3-8B treated as non-hallucinating. Across $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$2, $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$3, and $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$4, the best accuracy is $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$5, obtained at $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$6, $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$7, and $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$8. The paper states that this is comparable with the best results in the field and reports that the method outperforms most baselines listed from TruthfulQA except HaloScope (Ricco et al., 10 Feb 2025).

SigPal is illustrated on TCGA BRCA gene-expression data with 4000 genes and 348 samples from LumA, LumB, Her2, and Basal subtypes. For LumA versus LumB, SigClust gives $$d_p(\mathbf{x}, \mathbf{y}) = \left( \sum_{i=1}^d |x_i - y_i|^p \right)^{1/p}, \quad p>0,$$9, SigPal gives $q=64$0, and DiProPerm gives $q=64$1, showing that partial labels can recover strong separation that unsupervised cluster significance misses (Lu et al., 2015).

The relevant-dependence test is developed for strictly stationary and absolutely regular processes with components in separable metric spaces of negative type, and is illustrated on both vector autoregressive data and functional data generated from a 10-dimensional VAR process. Its asymptotic rejection probabilities behave as required: they converge to $q=64$2 when $q=64$3, to $q=64$4 on the boundary $q=64$5, and to $q=64$6 when $q=64$7 (Dette et al., 2024).

Advanced SSS is validated on pure-noise $q=64$8 images and on signal-bearing images. In pure noise, the proposed method controls Type I error, whereas the original SSS identifies spurious features across scales. On structured synthetic surfaces and gamma camera images, the advanced method recovers major slopes and curvatures while remaining more conservative than the original procedure (Liu et al., 30 Oct 2025).

PDDA is evaluated by Monte Carlo on ARFIMA processes. For $q=64$9 and $r \in \{4,6,8,10,12,14,16\}$0, R/S-PDDA has Bias $r \in \{4,6,8,10,12,14,16\}$1, SD $r \in \{4,6,8,10,12,14,16\}$2, and RMSE $r \in \{4,6,8,10,12,14,16\}$3, while MSD-PDDA has Bias $r \in \{4,6,8,10,12,14,16\}$4, SD $r \in \{4,6,8,10,12,14,16\}$5, and RMSE $r \in \{4,6,8,10,12,14,16\}$6. For $r \in \{4,6,8,10,12,14,16\}$7, R/S-PDDA has RMSE $r \in \{4,6,8,10,12,14,16\}$8 and MSD-PDDA has RMSE $r \in \{4,6,8,10,12,14,16\}$9. For $n \in \{1,\dots,10\}$0, both are almost unbiased, with MSD-PDDA tending to lower variance (Soriano et al., 19 May 2026).

LES is tested on synthetic tori, scRNA-seq time-series data, few-shot meta-learning tasks, and neural network layer matching. On the scRNA-seq developmental trajectory, the correlation between the embedded LES representation and actual time is $n \in \{1,\dots,10\}$1, versus $n \in \{1,\dots,10\}$2 for IMD, $n \in \{1,\dots,10\}$3 for OT, $n \in \{1,\dots,10\}$4 for GS, and $n \in \{1,\dots,10\}$5 for MTD; when time points use different subsets of genes, LES remains applicable and yields correlation about $n \in \{1,\dots,10\}$6 (Shnitzer et al., 2022).

A broader extension of the same distance-centric logic appears in Distance Measure Machines, where a distribution $n \in \{1,\dots,10\}$7 is embedded by its dissimilarities to template distributions,

$n \in \{1,\dots,10\}$8

and large-margin linear separation in this distance space is guaranteed when the dissimilarity is $n \in \{1,\dots,10\}$9-good (Rakotomamonjy et al., 2018). A related modeling extension is GP regression on distributions, where kernels are defined directly from distances between input distributions, for example

$H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$00

so that significance questions about effects of distributional changes can be posed through posterior means and variances in distribution space (Dolgov et al., 2018).

6. Robustness, assumptions, and limitations

One recurrent claim is robustness to nuisance choices, but the meaning of robustness varies by instantiation. In the hallucination study, the central empirical claim is “scale-free” behavior: qualitative separation of hallucinated and non-hallucinated distance distributions persists across $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$01, across $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$02, and across $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$03, with KL divergence and $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$04 showing similar qualitative trends regardless of norm (Ricco et al., 10 Feb 2025). In PDDA, the same distance matrix supports both R/S- and MSD-based routes to $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$05, and in multivariate anisotropic settings different functionals of the same distance object isolate different aspects of scaling, such as $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$06 versus $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$07 (Soriano et al., 19 May 2026).

The assumptions, however, are substantial. The hallucination detector uses an artificial labeling scheme in which all LLaMA2 responses to post-cutoff questions are treated as hallucinated and all LLaMA3 responses as non-hallucinated, with explicit acknowledgment of label noise. Its training complexity is $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$08 because of all pairwise intra-class distances, and the authors stop at $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$09 for computational reasons (Ricco et al., 10 Feb 2025). SigPal relies on a single-Gaussian null and can become anti-conservative under non-diagonal covariance unless the data are rotated to diagonalize covariance approximately; the detailed theory is provided mainly for $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$10LDA (Lu et al., 2015). The relevant-dependence test requires strictly stationary and absolutely regular processes, finite $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$11 moments, and separable metric spaces of negative type, so it is not distribution-free in the broadest sense (Dette et al., 2024). Advanced SSS assumes Gaussian i.i.d. noise and derives validity through a specific lattice random-field asymptotic regime (Liu et al., 30 Oct 2025). LES is alignment-free precisely because it discards eigenvectors, but this also means it is only a lower bound to the log-Euclidean metric and is not a strict metric, since identity of indiscernibles can fail for non-isometric isospectral manifolds (Shnitzer et al., 2022).

A common misconception is that “significance” in these frameworks always means a point-null test of zero effect. The thresholded dependence formulation explicitly rejects that view by testing practical relevance, $H_1^{\rm rel}: \mathrm{dcor}(X,Y) > \Delta$12, rather than exact independence (Dette et al., 2024). Another misconception is that distance-based inference is necessarily heuristic. Several of the methods are explicitly framed as mathematically sound: the hallucination detector uses nonparametric hypothesis testing and KDE-based likelihoods; the relevant-dependence test is pivotal; advanced SSS uses an advanced distribution theory for maxima of Gaussian random fields; LES rests on an eigenvalue lower bound of a Riemannian metric (Ricco et al., 10 Feb 2025, Liu et al., 30 Oct 2025, Shnitzer et al., 2022).

These works collectively suggest a durable research direction: use distances as first-order inferential primitives, but attach them to explicit probabilistic calibration rather than to ad hoc thresholds. Where that calibration is available—through null simulation, asymptotic process theory, extreme-value limits, or class-conditional density estimation—the framework yields not just geometric comparison but formal significance analysis.