---
title: Mahalanobis Distance Scores in ML & OOD Detection
url: https://www.emergentmind.com/topics/mahalanobis-distance-scores-mds
type: topic
---

# Mahalanobis Distance Scores in ML & OOD Detection

Mahalanobis Distance Scores (MDS) are covariance-aware distance-derived statistics used to quantify how far an observation lies from a reference distribution, prototype, class centroid, or representational state. In the classical form, the reference is a mean vector and covariance matrix; in contemporary machine learning, MDS most often denotes post-hoc scores computed in feature space, especially for out-of-distribution (OOD) detection, anomaly detection, and related ranking or classification tasks. The acronym is overloaded in the broader literature, where “MDS” can also mean multidimensional scaling; in the present sense, it refers specifically to scores computed from Mahalanobis distances [1708.03914]. In neural OOD detection, a standard convention is to use the negative of the smallest class-wise squared Mahalanobis distance, so that larger scores indicate stronger in-distribution (ID) evidence [2605.14413].

## 1. Mathematical definition and geometric interpretation

The canonical Mahalanobis distance between a vector \(x\), a mean \(\mu\), and a positive-definite covariance \(\Sigma\) is
\[
d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.
\]
Many applied papers work directly with the squared form,
\[
d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),
\]
and use that quantity itself as the score [2410.19352]. Geometrically, the distance measures deviation after whitening: if \(\Sigma = LL^\top\), then \(y=L^{-1}(x-\mu)\) satisfies \(d_M(x)=\|y\|_2\). The level sets of the squared score are ellipsoids aligned with the eigenvectors of \(\Sigma\), with high-variance directions downweighted and low-variance directions amplified [2410.19352, 1306.2004].

This whitening view underlies both classical statistics and modern deep learning uses. In a linear layer \(z=Wx+b\), the parameterization \(W \approx \Sigma^{-1/2}\) and \(b \approx -\Sigma^{-1/2}\mu\) yields \(\|z\|_2 = d_M(x)\), so the Euclidean norm of the transformed activation is exactly a Mahalanobis distance when the layer implements whitening [2410.19352]. Under class-conditional Gaussian models with shared covariance, the same quadratic form appears in the linear discriminant function, which is why Mahalanobis scores are closely connected to LDA-like generative interpretations [2003.00402].

In neural OOD detection, the most common instantiation is class-conditional and feature-based. Given a penultimate-layer feature \(\phi(x)\), class means \(\hat\mu_c\), and a pooled covariance \(\hat\Sigma\), the class-wise squared Mahalanobis distance is
\[
d_{\mathrm{Maha},c}(x)=\bigl(\phi(x)-\hat{\mu}_c\bigr)^\top \hat{\Sigma}_\lambda^{-1}\bigl(\phi(x)-\hat{\mu}_c\bigr),
\]
with \(\hat\Sigma_\lambda=\hat\Sigma+10^{-3}I\) for numerical stability in one recent formulation [2605.14413]. The corresponding score is often sign-flipped,
\[
S_{\mathrm{MDS}}(x)=-\min_c d_{\mathrm{Maha},c}(x),
\]
so that higher values correspond to stronger ID evidence [2605.14413].

## 2. Standard post-hoc construction in neural networks

The standard pipeline in CNN and Transformer OOD detection is post-hoc and does not retrain the backbone. Features \(f_\ell(x)\) are extracted at one or more layers \(\ell\); per-class means \(\mu_{c,\ell}\) and a covariance \(\Sigma_\ell\) are estimated from ID training data; class-wise Mahalanobis distances are computed; and a confidence score such as
\[
S_\ell(x)=-\min_c d_M(f_\ell(x);\mu_{c,\ell},\Sigma_\ell)
\]
is thresholded to reject OOD inputs [2110.07043, 2003.00402]. In many implementations, a pooled covariance is used at each layer “primarily for numerical stability and to avoid overfitting class-specific covariances when samples per class are limited” [2110.07043].

A recent formulation makes the feature-space construction explicit. With ID training data \(\{(x_i,y_i)\}_{i=1}^N\), class centroids and pooled covariance are estimated as
\[
\hat{\mu}_c=\frac{1}{N_c}\sum_{i:\,y_i=c}\phi(x_i), \qquad
\hat{\Sigma}=\frac{1}{N}\sum_{c=1}^C\sum_{i:\,y_i=c}(\phi(x_i)-\hat{\mu}_c)(\phi(x_i)-\hat{\mu}_c)^\top,
\]
after which \(\hat\Sigma_\lambda^{-1}\) is precomputed and reused at inference [2605.14413]. Mahalanobis++ replaces \(\phi(x)\) by the normalized feature \(\hat\phi(x)=\phi(x)/\|\phi(x)\|_2\) and estimates the same quantities in normalized feature space [2605.14413].

Layer aggregation is another standard variant. Rather than using only the penultimate representation, one computes \(S_\ell(x)\) at several depths and combines them linearly, for example as \(S(x)=\sum_\ell \alpha_\ell S_\ell(x)+\beta\), with weights fitted on validation data [2003.00402]. This multi-layer construction was adopted in CNN OOD work and later contrasted with single-layer alternatives and nonparametric replacements [2110.07043].

For Transformer-based intent detection, fine-tuning on ID labels reshapes the representation geometry in a manner especially favorable to MDS. Fine-tuned Transformer encoders produce dense class clusters, near-orthogonal centroids, and low-dimensional ID structure, after which Mahalanobis distance “outperforms other methods by a wide margin and establishes new state-of-the-art results for all datasets” in that setting [2101.03778]. A related speech setting models each wav2vec 2.0 layer as a global Gaussian, computes a per-layer squared Mahalanobis score, concatenates the layer scores, and feeds the resulting vector to a KNN outlier detector [2308.04886].

## 3. Mechanisms of effectiveness

One influential explanation is that MDS works not because the tied-covariance Gaussian model is literally correct, but because \(\Sigma^{-1}\) amplifies deviations along low-variance directions. In PCA coordinates, the squared Mahalanobis distance is
\[
\Delta^2=\sum_{i=1}^{N}\frac{y_i^2}{\lambda_i},
\]
so directions with small \(\lambda_i\) contribute disproportionately. Empirically, OOD and adversarial inputs inflate variance in these low-variance components, even when classification confidence remains high; correspondingly, partial Mahalanobis distances using only low-variance components can recover nearly the full detection performance of the original score [2003.00402]. The same paper further reports that a marginal variant using a single Gaussian and no label information often matches the class-conditional score, indicating that anomaly-sensitive structure can be largely orthogonal to discriminative structure [2003.00402].

A second mechanism, formalized in MahaVar, is class-wise dispersion. On \(L_2\)-normalized features, ID samples exhibit a “sharp minimum” in their class-wise distances:
\[
d_{(1)}(x)\ll d_{(2)}(x)\approx d_{(3)}(x)\approx \cdots \approx d_{(C)}(x),
\]
where \(d_{(1)}(x)\) is the nearest-class distance after sorting [2605.14413]. OOD samples, by contrast, are relatively far from all class means and show a flatter distance profile. This motivates the variance term
\[
\mathrm{Var}_c[d_{\mathrm{Maha},c}(x)]
=
\frac{1}{C}\sum_{c=1}^C
\left(
d_{\mathrm{Maha},c}(x)-\frac{1}{C}\sum_{k=1}^C d_{\mathrm{Maha},k}(x)
\right)^2,
\]
and the augmented score
\[
S_{\mathrm{MahaVar}}(x)
=
-\min_c d_{\mathrm{Maha},c}(\hat\phi(x))
+
\alpha\,\mathrm{Var}_c[d_{\mathrm{Maha},c}(\hat\phi(x))].
\]
Under relaxed Neural Collapse assumptions—Weak NC1 for within-class compactness and NC2 for inter-class ETF geometry—ID samples are shown to have structurally high class-wise distance variance, and MahaVar achieves state-of-the-art performance on CIFAR-100 and ImageNet under the OpenOOD v1.5 protocol [2605.14413].

The Neural Collapse analysis also exposes a limitation. Complete ETF geometry requires \(d\ge C-1\); the paper notes that Transformer backbones on ImageNet violate this condition, which weakens variance separation [2605.14413]. This suggests that the theoretical basis of a particular MDS variant can be highly representation-dependent even when the scoring formula itself is simple.

## 4. Failure modes, criticisms, and corrective variants

The principal technical criticism of MDS in deep networks is sample complexity. CNN penultimate features are frequently \(1024\)–\(2048\)-dimensional, while per-class training sample sizes can be only on the order of \(1{,}000\)–\(2{,}000\). In that regime, estimating \(\Sigma\) is unstable; if \(n\le p\), the sample covariance is singular; and even when \(n>p\), inversion can be ill-conditioned and highly variable [2110.07043]. A simulation study in that work reports that at \(d=1000\), Mahalanobis AUCROC \(\approx 54.6\%\) and TNR@TPR95 \(\approx 7.1\%\), while LOF remains substantially more robust [2110.07043]. The same paper argues that CNN feature distributions are often multimodal and non-Gaussian within classes, so global Gaussianity and tied covariance may be materially misspecified [2110.07043].

A separate critique concerns near-OOD behavior. Relative Mahalanobis Distance (RMD) introduces a background Gaussian fitted to all training features, defines
\[
MD_0(x)=(z-\mu_0)^\top \Sigma_0^{-1}(z-\mu_0),
\qquad
RMD_c(x)=MD_c(x)-MD_0(x),
\]
and uses
\[
s_{\mathrm{RMD}}(x)=-\min_c RMD_c(x)
\]
as the confidence score [2106.09022]. The likelihood-ratio interpretation is explicit:
\[
s_{\mathrm{RMD}}(x)=\max_c[\log p_c(z)-\log p_0(z)].
\]
The stated rationale is that non-discriminative background directions where \(p_c\approx p_0\) cancel out, which improves near-OOD detection and reduces hyperparameter sensitivity; the paper reports improvements of up to \(15\%\) AUROC on genomics OOD [2106.09022].

Layer dependence is another recurring controversy. In medical imaging, one study explicitly rejects the idea of a universally optimal Mahalanobis layer: the best depth varies with the OOD pattern, early layers can be best for local artefacts, deeper layers can be best for semantic or device-related shifts, and the last hidden layer can be sub-optimal or degrading [2309.01488]. The same work proposes Multi-Branch Mahalanobis (MBM), in which per-layer scores are standardized and summed within branches separated by downsampling operations, with an OR rule across branches for detection [2309.01488]. The broader implication is that “Mahalanobis distance” is not a single detector but a family whose behavior depends strongly on representation depth, covariance estimation, and the relevant OOD mechanism.

## 5. Generalizations across domains

Outside post-hoc neural OOD detection, MDS appears in several mathematically distinct but structurally related forms.

In time-series classification, a univariate series of equal length is treated as a vector, and class-specific Mahalanobis metrics are used inside 1-NN classification. Because time-series covariance matrices are often low rank, the paper studies three remedies—pseudoinverse, covariance shrinkage, and diagonal restriction—and recommends learning one distance measure per class using either covariance shrinking or the diagonal approach [1010.1526]. The diagonal construction reduces to a standardized Euclidean distance with determinant-normalized weights, while shrinkage keeps off-diagonal structure but avoids singularity [1010.1526].

In functional data analysis, the finite-dimensional inverse covariance is replaced by a regularized square-root inverse covariance operator in Hilbert space. The functional Mahalanobis semi-distance is
\[
d_{FM}^{K}(\chi,\mu_{\chi})
=
\left\langle
\Gamma_K^{-1/2}(\chi-\mu_{\chi}),
\Gamma_K^{-1/2}(\chi-\mu_{\chi})
\right\rangle^{1/2}
=
\left(\sum_{k=1}^{K}\omega_k^2\right)^{1/2},
\]
where \(\omega_k\) are standardized FPCA scores and \(K\) is a truncation parameter [1304.4786]. It is a semi-distance rather than a full metric because equality of the first \(K\) standardized FPCA scores does not imply equality of functions [1304.4786].

In semiparametric classification, global class-wise Mahalanobis distances \(d_k^2(x)=(x-\mu_k)^\top\Sigma_k^{-1}(x-\mu_k)\) are used as covariates in a generalized additive model with logistic link, and a localized version replaces the global score by a kernel-weighted local Mahalanobis feature designed for non-elliptic and multimodal class structure [2402.08283]. The global construction is motivated by elliptically contoured densities, for which log-posteriors are additive functions of the class-wise Mahalanobis distances [2402.08283].

In representational similarity analysis, the relevant object is the cross-validated Mahalanobis distance, or crossnobis estimator,
\[
\hat d_j
=
\frac{1}{MP}\sum_{m=1}^{M}
\hat\delta_{j,m}\hat\delta_{j,\sim m}^{\top},
\]
which is unbiased for the true squared Mahalanobis dissimilarity between conditions and admits an approximate multivariate normal sampling distribution with analytically tractable covariance [1607.01371]. This use is conceptually distinct from anomaly scoring but retains the same core idea: a covariance-whitened quadratic form, now cross-validated to remove positive bias [1607.01371].

Other extensions are even more specialized. Clustering-informed Mahalanobis distance replaces ordinary principal directions by cluster-averaged directions to stabilize covariance estimation in high-dimensional, low-sample regimes and can recover hidden Euclidean distances under a linear observation model [1708.03914]. Online metric learning with KISSME yields a learned PSD matrix \(M\) for stream classification, after which neighbor ranking is performed with \(d_M(x,y)=\sqrt{(x-y)^\top M(x-y)}\) [1604.04879]. Approximate distance estimation with Mahalanobis metrics can be accelerated by factorizing \(A=U^\top U\) and sketching \(Ux\) via Johnson–Lindenstrauss embeddings, supporting adaptive queries and online updates to both the metric and the dataset [2309.01030]. In recommender systems, Mahalanobis distances are used as learned relation-specific scores between users, playlists, and songs, and as metric-based attention weights over playlist members [1906.03450].

## 6. Estimation, computation, and interpretive outlook

Across these formulations, the dominant technical issue is covariance estimation. Full covariances are informative but expensive and unstable in high dimension; pooled covariances improve stability but impose a shared shape across classes; class-specific covariances increase flexibility but raise variance; and singularity forces either pseudoinverses, shrinkage, ridge loading, diagonal approximations, or low-rank truncation [1010.1526, 2003.00402, 2110.07043]. In practice, several papers recommend Cholesky or eigendecomposition-based computation rather than explicit matrix inversion, and recent OOD formulations commonly add a small ridge term such as \(10^{-3}I\) before inversion [2605.14413].

Computational scaling is likewise formulation-dependent. In static post-hoc OOD detection, the main costs are feature extraction, covariance estimation, and quadratic forms against class means. In online or large-scale settings, sketching can reduce Mahalanobis queries to approximate Euclidean distance estimation in a lower-dimensional sketched space, with update operations on both \(U\) and the database points [2309.01030]. This suggests that the historical computational burden of Mahalanobis geometry is not intrinsic, but heavily dependent on whether the metric is fixed, learned, local, or adaptive.

The interpretive status of MDS has also broadened. One theoretical line shows that a linear layer can be understood as a whitening transform whose output norm equals a Mahalanobis distance, making prototype proximity and covariance-aware deviation explicit in network internals [2410.19352]. Another line, centered on OOD detection, argues that the empirical success of MDS often stems from anomaly-sensitive geometry that the classifier itself does not directly use for prediction [2003.00402]. Taken together, these results suggest that MDS is best regarded not as a single probabilistic model, but as a broad family of covariance-normalized geometric statistics whose effectiveness depends on representation geometry, covariance regularization, and the match between the scoring convention and the target phenomenon.

A recurring misconception is that there exists one definitive Mahalanobis score. The literature instead supports a stratified picture: global scores, class-conditional scores, marginal scores, relative scores, variance-augmented scores, local scores, functional scores, cross-validated scores, and learned metric scores all preserve the same quadratic core while targeting different failure modes and different statistical regimes [2106.09022, 2605.14413, 2402.08283]. This suggests that future work will continue to treat MDS less as a fixed recipe than as a design space for covariance-aware inference.

Source: https://www.emergentmind.com/topics/mahalanobis-distance-scores-mds