Distance-Aware Uncertainty Quantification

• Distance-aware uncertainty quantification is a technique where prediction confidence decreases as inputs stray further from the training data manifold, ensuring reliable risk assessment.
• It employs methods such as nearest-neighbor distances, density estimation, and adaptive Bayesian priors to translate distance metrics into calibrated uncertainty scores.
• By enhancing out-of-distribution detection and calibration, these methods improve model robustness in applications from image recognition to scientific simulations and graph-based learning.

Distance-aware uncertainty quantification refers to a family of methods in machine learning that explicitly link predictive uncertainty to the geometric or statistical distance between a test input and the empirical support of the training data. The key principle is that uncertainty should increase as an input moves farther from the manifold or distribution of seen/training examples—an essential property for robust modeling, calibrated risk assessment, and reliable out-of-distribution (OOD) detection.

1. Theoretical Foundations and Motivation

Distance-aware uncertainty quantification is motivated by foundational limitations in standard neural architectures, which often yield overconfident predictions even for inputs far from the data manifold. In contrast, classical models such as Gaussian Processes (GPs) or Bayesian inference provide predictive uncertainty that grows with input distance due to their kernel or prior structure. The goal is to endow modern deep models with analogous properties by making prediction confidence explicitly sensitive to “distance from training data,” enabling reliable uncertainty quantification under covariate shift, OOD detection, and safety-critical decision making (Chouraqui et al., 2023, Apostolopoulou et al., 2024, Detommaso et al., 2022).

Distance-awareness can be considered a necessary condition for high-quality uncertainty: it ensures that the credible intervals, predictive variances, or misclassification probabilities grow as the model's evidence shrinks with data scarcity in the input space (Bui et al., 2024, Desai et al., 8 Jul 2025, Rajput et al., 2023).

2. Core Methodologies

Distance-aware UQ methods are implemented at the level of model architecture, feature space geometry, and calibration routines. Canonical approaches include:

• Geometric Separation and Nearest-Neighbor Distance: Compute distances in the input or learned feature space from a test point to the closest training points (or class-conditional regions), then use these metrics as raw uncertainty signals. For example, the “fast-separation” score is

$\text{stab}(x) = \frac{D(x,O(x)) - D(x,F(x))}{2},$

with $D(x,A) = \min_{a\in A} \|x - a\|_2$, where $F(x)$ is the set of training points with same predicted label as $x$, and $O(x)$ is the complement (Chouraqui et al., 2023).

• Density in Feature/Latent Space: Model the probability density $p(z)$ at the location of the encoded sample; uncertainty is then an explicit function of density (e.g., variance proportional to $1/p(z)$) (Bui et al., 2024). Normalizing flows or kernel density estimators are typically used; low feature-space density yields high predictive variance.
• Statistical Depth and Generalized Distances: Use data depth measures such as Lens Depth (LD) and distances along learned or sample manifolds (e.g., Fermat distance) to quantify “centrality” and “outlyingness” in the distribution of seen features (Nguyen et al., 2024).
• Graph and Prototype-Based Regularization: In node classification or prototype-based uncertainty (e.g., DDAR), regularization terms enforce that OOD nodes or remote samples remain distinct/unclustered in latent space, or distance to learned prototypes or codebook centers directly informs uncertainty (Hart et al., 2023, Zhang et al., 2024, Apostolopoulou et al., 2024).
• Distance-aware Priors in Bayesian Models: Make prior variances in Bayesian neural networks input-dependent, expanding away from the training data to force epistemic uncertainty to increase with distance from empirical support (Detommaso et al., 2022).
• Kernel/RFF GP Layers with Distance Preservation: Attach GP approximations (RBF-kernel) to the output of a spectral-normalized or SVD-projected feature extractor, so that representation-space distances preserve input distances, and distance from training features correlates with posterior variance (Goldenberg et al., 2023, Rajput et al., 2023, Razzaq et al., 9 Dec 2025).
• Entropy, Temperature Scaling, and Morse Networks: Use learned or parametric “temperature” scaling to adjust logit or class-probability sharpness according to distance-based signals (e.g., Morse networks, where temperature is an explicit inverse density or kernel evaluation) (Dherin et al., 2023).

3. Calibration and Post-hoc Mapping

Distance signals are not themselves calibrated probabilities or direct uncertainty measures. They must be mapped into calibrated confidence or uncertainty scores via statistical calibration strategies:

• Isotonic Regression: Non-parametric, monotonic calibration on a hold-out set, often found superior to Platt/sigmoid fitting near boundaries (Chouraqui et al., 2023).
• Empirical Depth-to-Accuracy Mapping: Binning distance-based or depth scores and regressing against empirical accuracy yields monotonic mappings for classification confidence (Chouraqui et al., 2023, Nguyen et al., 2024).
• Distributional/Second-Order Measures: In the context of second-order predictive distributions (e.g., Dirichlet models), minimal distance to reference sets (degenerate, pure aleatoric, or pure epistemic distributions) under Wasserstein or other divergences yields decomposed uncertainty metrics satisfying desirable theoretical properties (Sale et al., 2023).

4. Architectures and Algorithms

Representative model frameworks employing distance-aware UQ include:

• Geometric Fast-Separation Pipeline: Partition training data by predicted label, efficiently compute set-distances (nearest-neighbor queries), and calibrate the scores as confidences (Chouraqui et al., 2023).
• Density-Regression: Three-phase training (initial regressor, feature density estimation via normalizing flows, fine-tune variance) providing closed-form uncertainty inversely proportional to local feature density, which is strictly monotonic in feature-space distance (Bui et al., 2024).
• PG-SNGP and Related GP-Approximation Heads: Spectral normalization on hidden layers, RFF kernel head, and variance estimation controlled by distance from training embeddings (Razzaq et al., 9 Dec 2025, Goldenberg et al., 2023, Rajput et al., 2023).
• DDAR and Deterministic Deep Methods: Prototypes in latent space, distinction maximization, RBF-based kernels, ensemble-like uncertainty from single-pass inference (Zhang et al., 2024).

A table summarizing key classes of distance-aware UQ methods is presented below:

Approach	Mechanism	Output/Interpretation
Geometric separation	Nearest-neighbor distances, class regions	Calibrated classwise confidence
Density-modulated regression	$p(z)$ via normalizing flow; variance $\sim 1/p(z)$	Predictive variance
GP/RFF layers	Kernel distance in feature space	Posterior variance
Statistical depth + Fermat	Depth wrt. sample manifold FD	Outlyingness score
Distance-aware prior	Input-dependent prior variance	Epistemic uncertainty
Prototype/Codebook	KL/Bregman/dist. to code distributions	Uncertainty from assignment

5. Empirical Properties and Performance

Distance-aware UQ methods consistently improve misclassification detection, OOD detection, and calibration, especially under distribution shift:

• Calibration Error: Fast-separation + isotonic consistently achieves lowest Expected Calibration Error (ECE) across >80% of experiments, often surpassing deep ensembles and other post-hoc methods by wide margins (Chouraqui et al., 2023).
• OOD Detection: Statistical depth + Fermat distance (LD–FD) achieves AUROC values >0.97 on FashionMNIST (ID) vs MNIST (OOD), outperforming Mahalanobis and deep ensemble baselines (Nguyen et al., 2024).
• Regression and OOD Behavior: Density-Regression, PG-SNGP, and Spectral-Normalized GP layers produce predictive variances that track distance from training support, yielding better-quantified uncertainty on real-world and adversarial samples, as quantified by calibration error, NLL, and DAC metrics (Bui et al., 2024, Razzaq et al., 9 Dec 2025).
• Node Classification: Distance-based regularization in GPNs prevents collapse of in-distribution clusters and OOD nodes, improving both OOD AUROC and misclassification AUPR over vanilla uncertainty cross-entropy methods (Hart et al., 2023).

6. Theoretical Guarantees and Limitations

Distance-aware UQ frameworks often provide stronger theoretical guarantees than vanilla deep or Bayesian models:

• Axiomatic Separation of Uncertainty Types: Second-order Wasserstein-distance-based metrics provide decompositions that provably separate aleatoric and epistemic uncertainty, satisfying monotonicity, invariance, and additivity requirements (Sale et al., 2023).
• Distance-Awareness Proofs: For methods based on feature-space density, predictive entropy or variance is proven monotonic in the distance to the empirical data manifold (Bui et al., 2024).
• Bi-Lipschitz Constraints: Enforcing (or approximating) input-to-latent distance preservation ensures that distance-based uncertainty signals are not rendered uninformative by non-isometric neural maps (Goldenberg et al., 2023, Rajput et al., 2023).
• Regularization-based Methods: Distance-based regularizers prevent latent collapse and maintain OOD separability, even in graphical or structured data settings, alleviating theoretical failure modes of cross-entropy minimization (Hart et al., 2023).

Limitations and failure modes include dependence on the geometry learned by the feature extractor, potential inefficiency for very large datasets (if pairwise distances or quadratic kernel statistics are not approximated/subsampled), and the need for careful calibration or regularization tuning. Some approaches rely on the quality of a density estimator (e.g., normalizing flow), which may underperform if trained poorly or on complex manifolds (Bui et al., 2024). In node and graph domains, adversarial graph structures or high homophily may limit the effectiveness of distance-aware regularizers (Hart et al., 2023).

7. Application Domains and Extensions

Distance-aware uncertainty quantification has seen adoption and empirical validation across a range of domains:

• Vision and Pattern Recognition: Pixel-space geometric calibration for image classification, OOD detection, and semantic segmentation (e.g., MNIST, CIFAR, FashionMNIST) (Chouraqui et al., 2023, Nguyen et al., 2024, Bui et al., 2024).
• Scientific Modeling: Quantification of geometric ambiguity in 3D physical simulation via SDF-based UQ, where local surface uncertainty is made explicit for subsequent physical modeling or robotic navigation (Desai et al., 8 Jul 2025).
• Physics-Guided Maintenance and Fault Prediction: Bearing health estimation and degradation prediction under distribution shift, with explicit robustness and distance calibration (PG-SNGP, PG-SNER) (Razzaq et al., 9 Dec 2025).
• Graph-based Semi-supervised Learning: Detecting defective nodes or anomalies via distance-sensitive latent clustering (Hart et al., 2023).
• NLP and Intent Detection: OOD detection and calibration via prototype-based and codebook-based assignments (Zhang et al., 2024, Apostolopoulou et al., 2024).

Extensions include the development of faster approximations for pairwise depth/distance computation, adaptive local metric tuning, principled calibration metrics (DAC), GPU-accelerated nearest-neighbor or graph-search implementations, and the application of these methods to regression, medical diagnostics, and time-series forecasting (Nguyen et al., 2024, Bui et al., 2024).

---

References:

“Uncertainty Estimation based on Geometric Separation” (Chouraqui et al., 2023) “Combining Statistical Depth and Fermat Distance for Uncertainty Quantification” (Nguyen et al., 2024) “Second-Order Uncertainty Quantification: A Distance-Based Approach” (Sale et al., 2023) “Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts” (Bui et al., 2024) “Distance Preserving Machine Learning for Uncertainty Aware Accelerator Capacitance Predictions” (Goldenberg et al., 2023) “Uncertainty Aware Deep Learning for Particle Accelerators” (Rajput et al., 2023) “Discriminant Distance-Aware Representation on Deterministic Uncertainty Quantification Methods” (Zhang et al., 2024) “Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priors” (Detommaso et al., 2022) “Morse Neural Networks for Uncertainty Quantification” (Dherin et al., 2023) “A Probabilistic Approach to Uncertainty Quantification Leveraging 3D Geometry” (Desai et al., 8 Jul 2025) “A Rate-Distortion View of Uncertainty Quantification” (Apostolopoulou et al., 2024) “Improvements on Uncertainty Quantification for Node Classification via Distance-Based Regularization” (Hart et al., 2023) “Developing Distance-Aware Uncertainty Quantification Methods in Physics-Guided Neural Networks for Reliable Bearing Health Prediction” (Razzaq et al., 9 Dec 2025)