Mahalanobis Distance Scores in ML & OOD Detection

• Mahalanobis Distance Scores are covariance-aware measurements that quantify how far an observation deviates from a reference distribution using the inverse covariance structure.
• They are applied in out-of-distribution and anomaly detection by transforming feature spaces through whitening, enabling effective ranking and classification.
• Practical challenges include stable covariance estimation in high dimensions, leading to corrective variants like MahaVar and relative Mahalanobis distance.

Mahalanobis Distance Scores (MDS) are covariance-aware distance-derived statistics used to quantify how far an observation lies from a reference distribution, prototype, class centroid, or representational state. In the classical form, the reference is a mean vector and covariance matrix; in contemporary machine learning, MDS most often denotes post-hoc scores computed in feature space, especially for out-of-distribution (OOD) detection, anomaly detection, and related ranking or classification tasks. The acronym is overloaded in the broader literature, where “MDS” can also mean multidimensional scaling; in the present sense, it refers specifically to scores computed from Mahalanobis distances (Lahav et al., 2017). In neural OOD detection, a standard convention is to use the negative of the smallest class-wise squared Mahalanobis distance, so that larger scores indicate stronger in-distribution (ID) evidence (Kim et al., 14 May 2026).

1. Mathematical definition and geometric interpretation

The canonical Mahalanobis distance between a vector $x$, a mean $\mu$, and a positive-definite covariance $\Sigma$ is

$$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$

Many applied papers work directly with the squared form,

$$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$

and use that quantity itself as the score (Oursland, 2024). Geometrically, the distance measures deviation after whitening: if $\Sigma = LL^\top$, then $y=L^{-1}(x-\mu)$ satisfies $d_M(x)=\|y\|_2$. The level sets of the squared score are ellipsoids aligned with the eigenvectors of $\Sigma$, with high-variance directions downweighted and low-variance directions amplified (Oursland, 2024, Spurek et al., 2013).

This whitening view underlies both classical statistics and modern deep learning uses. In a linear layer $z=Wx+b$, the parameterization $\mu$0 and $\mu$1 yields $\mu$2, so the Euclidean norm of the transformed activation is exactly a Mahalanobis distance when the layer implements whitening (Oursland, 2024). Under class-conditional Gaussian models with shared covariance, the same quadratic form appears in the linear discriminant function, which is why Mahalanobis scores are closely connected to LDA-like generative interpretations (Kamoi et al., 2020).

In neural OOD detection, the most common instantiation is class-conditional and feature-based. Given a penultimate-layer feature $\mu$3, class means $\mu$4, and a pooled covariance $\mu$5, the class-wise squared Mahalanobis distance is

$\mu$6

with $\mu$7 for numerical stability in one recent formulation (Kim et al., 14 May 2026). The corresponding score is often sign-flipped,

$\mu$8

so that higher values correspond to stronger ID evidence (Kim et al., 14 May 2026).

2. Standard post-hoc construction in neural networks

The standard pipeline in CNN and Transformer OOD detection is post-hoc and does not retrain the backbone. Features $\mu$9 are extracted at one or more layers $\Sigma$0; per-class means $\Sigma$1 and a covariance $\Sigma$2 are estimated from ID training data; class-wise Mahalanobis distances are computed; and a confidence score such as

$\Sigma$3

is thresholded to reject OOD inputs (Szyc et al., 2021, Kamoi et al., 2020). In many implementations, a pooled covariance is used at each layer “primarily for numerical stability and to avoid overfitting class-specific covariances when samples per class are limited” (Szyc et al., 2021).

A recent formulation makes the feature-space construction explicit. With ID training data $\Sigma$4, class centroids and pooled covariance are estimated as

$\Sigma$5

after which $\Sigma$6 is precomputed and reused at inference (Kim et al., 14 May 2026). Mahalanobis++ replaces $\Sigma$7 by the normalized feature $\Sigma$8 and estimates the same quantities in normalized feature space (Kim et al., 14 May 2026).

Layer aggregation is another standard variant. Rather than using only the penultimate representation, one computes $\Sigma$9 at several depths and combines them linearly, for example as $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$0, with weights fitted on validation data (Kamoi et al., 2020). This multi-layer construction was adopted in CNN OOD work and later contrasted with single-layer alternatives and nonparametric replacements (Szyc et al., 2021).

For Transformer-based intent detection, fine-tuning on ID labels reshapes the representation geometry in a manner especially favorable to MDS. Fine-tuned Transformer encoders produce dense class clusters, near-orthogonal centroids, and low-dimensional ID structure, after which Mahalanobis distance “outperforms other methods by a wide margin and establishes new state-of-the-art results for all datasets” in that setting (Podolskiy et al., 2021). A related speech setting models each wav2vec 2.0 layer as a global Gaussian, computes a per-layer squared Mahalanobis score, concatenates the layer scores, and feeds the resulting vector to a KNN outlier detector (Das et al., 2023).

3. Mechanisms of effectiveness

One influential explanation is that MDS works not because the tied-covariance Gaussian model is literally correct, but because $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$1 amplifies deviations along low-variance directions. In PCA coordinates, the squared Mahalanobis distance is

$$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$2

so directions with small $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$3 contribute disproportionately. Empirically, OOD and adversarial inputs inflate variance in these low-variance components, even when classification confidence remains high; correspondingly, partial Mahalanobis distances using only low-variance components can recover nearly the full detection performance of the original score (Kamoi et al., 2020). The same paper further reports that a marginal variant using a single Gaussian and no label information often matches the class-conditional score, indicating that anomaly-sensitive structure can be largely orthogonal to discriminative structure (Kamoi et al., 2020).

A second mechanism, formalized in MahaVar, is class-wise dispersion. On $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$4-normalized features, ID samples exhibit a “sharp minimum” in their class-wise distances: $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$5 where $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$6 is the nearest-class distance after sorting (Kim et al., 14 May 2026). OOD samples, by contrast, are relatively far from all class means and show a flatter distance profile. This motivates the variance term

$$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$7

and the augmented score

$$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$8

Under relaxed Neural Collapse assumptions—Weak NC1 for within-class compactness and NC2 for inter-class ETF geometry—ID samples are shown to have structurally high class-wise distance variance, and MahaVar achieves state-of-the-art performance on CIFAR-100 and ImageNet under the OpenOOD v1.5 protocol (Kim et al., 14 May 2026).

The Neural Collapse analysis also exposes a limitation. Complete ETF geometry requires $$d_M(x;\mu,\Sigma) = \sqrt{(x-\mu)^\top \Sigma^{-1}(x-\mu)}.$$9; the paper notes that Transformer backbones on ImageNet violate this condition, which weakens variance separation (Kim et al., 14 May 2026). This suggests that the theoretical basis of a particular MDS variant can be highly representation-dependent even when the scoring formula itself is simple.

4. Failure modes, criticisms, and corrective variants

The principal technical criticism of MDS in deep networks is sample complexity. CNN penultimate features are frequently $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$0–$$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$1-dimensional, while per-class training sample sizes can be only on the order of $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$2–$$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$3. In that regime, estimating $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$4 is unstable; if $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$5, the sample covariance is singular; and even when $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$6, inversion can be ill-conditioned and highly variable (Szyc et al., 2021). A simulation study in that work reports that at $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$7, Mahalanobis AUCROC $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$8 and TNR@TPR95 $$d_M^2(x;\mu,\Sigma) = (x-\mu)^\top \Sigma^{-1}(x-\mu),$$9, while LOF remains substantially more robust (Szyc et al., 2021). The same paper argues that CNN feature distributions are often multimodal and non-Gaussian within classes, so global Gaussianity and tied covariance may be materially misspecified (Szyc et al., 2021).

A separate critique concerns near-OOD behavior. Relative Mahalanobis Distance (RMD) introduces a background Gaussian fitted to all training features, defines

$\Sigma = LL^\top$0

and uses

$\Sigma = LL^\top$1

as the confidence score (Ren et al., 2021). The likelihood-ratio interpretation is explicit: $\Sigma = LL^\top$2 The stated rationale is that non-discriminative background directions where $\Sigma = LL^\top$3 cancel out, which improves near-OOD detection and reduces hyperparameter sensitivity; the paper reports improvements of up to $\Sigma = LL^\top$4 AUROC on genomics OOD (Ren et al., 2021).

Layer dependence is another recurring controversy. In medical imaging, one study explicitly rejects the idea of a universally optimal Mahalanobis layer: the best depth varies with the OOD pattern, early layers can be best for local artefacts, deeper layers can be best for semantic or device-related shifts, and the last hidden layer can be sub-optimal or degrading (Anthony et al., 2023). The same work proposes Multi-Branch Mahalanobis (MBM), in which per-layer scores are standardized and summed within branches separated by downsampling operations, with an OR rule across branches for detection (Anthony et al., 2023). The broader implication is that “Mahalanobis distance” is not a single detector but a family whose behavior depends strongly on representation depth, covariance estimation, and the relevant OOD mechanism.

5. Generalizations across domains

Outside post-hoc neural OOD detection, MDS appears in several mathematically distinct but structurally related forms.

In time-series classification, a univariate series of equal length is treated as a vector, and class-specific Mahalanobis metrics are used inside 1-NN classification. Because time-series covariance matrices are often low rank, the paper studies three remedies—pseudoinverse, covariance shrinkage, and diagonal restriction—and recommends learning one distance measure per class using either covariance shrinking or the diagonal approach (Prekopcsák et al., 2010). The diagonal construction reduces to a standardized Euclidean distance with determinant-normalized weights, while shrinkage keeps off-diagonal structure but avoids singularity (Prekopcsák et al., 2010).

In functional data analysis, the finite-dimensional inverse covariance is replaced by a regularized square-root inverse covariance operator in Hilbert space. The functional Mahalanobis semi-distance is

$\Sigma = LL^\top$5

where $\Sigma = LL^\top$6 are standardized FPCA scores and $\Sigma = LL^\top$7 is a truncation parameter (Joseph et al., 2013). It is a semi-distance rather than a full metric because equality of the first $\Sigma = LL^\top$8 standardized FPCA scores does not imply equality of functions (Joseph et al., 2013).

In semiparametric classification, global class-wise Mahalanobis distances $\Sigma = LL^\top$9 are used as covariates in a generalized additive model with logistic link, and a localized version replaces the global score by a kernel-weighted local Mahalanobis feature designed for non-elliptic and multimodal class structure (Ghosh et al., 2024). The global construction is motivated by elliptically contoured densities, for which log-posteriors are additive functions of the class-wise Mahalanobis distances (Ghosh et al., 2024).

In representational similarity analysis, the relevant object is the cross-validated Mahalanobis distance, or crossnobis estimator,

$y=L^{-1}(x-\mu)$0

which is unbiased for the true squared Mahalanobis dissimilarity between conditions and admits an approximate multivariate normal sampling distribution with analytically tractable covariance (Diedrichsen et al., 2016). This use is conceptually distinct from anomaly scoring but retains the same core idea: a covariance-whitened quadratic form, now cross-validated to remove positive bias (Diedrichsen et al., 2016).

Other extensions are even more specialized. Clustering-informed Mahalanobis distance replaces ordinary principal directions by cluster-averaged directions to stabilize covariance estimation in high-dimensional, low-sample regimes and can recover hidden Euclidean distances under a linear observation model (Lahav et al., 2017). Online metric learning with KISSME yields a learned PSD matrix $y=L^{-1}(x-\mu)$1 for stream classification, after which neighbor ranking is performed with $y=L^{-1}(x-\mu)$2 (Perez et al., 2016). Approximate distance estimation with Mahalanobis metrics can be accelerated by factorizing $y=L^{-1}(x-\mu)$3 and sketching $y=L^{-1}(x-\mu)$4 via Johnson–Lindenstrauss embeddings, supporting adaptive queries and online updates to both the metric and the dataset (Qin et al., 2023). In recommender systems, Mahalanobis distances are used as learned relation-specific scores between users, playlists, and songs, and as metric-based attention weights over playlist members (Tran et al., 2019).

6. Estimation, computation, and interpretive outlook

Across these formulations, the dominant technical issue is covariance estimation. Full covariances are informative but expensive and unstable in high dimension; pooled covariances improve stability but impose a shared shape across classes; class-specific covariances increase flexibility but raise variance; and singularity forces either pseudoinverses, shrinkage, ridge loading, diagonal approximations, or low-rank truncation (Prekopcsák et al., 2010, Kamoi et al., 2020, Szyc et al., 2021). In practice, several papers recommend Cholesky or eigendecomposition-based computation rather than explicit matrix inversion, and recent OOD formulations commonly add a small ridge term such as $y=L^{-1}(x-\mu)$5 before inversion (Kim et al., 14 May 2026).

Computational scaling is likewise formulation-dependent. In static post-hoc OOD detection, the main costs are feature extraction, covariance estimation, and quadratic forms against class means. In online or large-scale settings, sketching can reduce Mahalanobis queries to approximate Euclidean distance estimation in a lower-dimensional sketched space, with update operations on both $y=L^{-1}(x-\mu)$6 and the database points (Qin et al., 2023). This suggests that the historical computational burden of Mahalanobis geometry is not intrinsic, but heavily dependent on whether the metric is fixed, learned, local, or adaptive.

The interpretive status of MDS has also broadened. One theoretical line shows that a linear layer can be understood as a whitening transform whose output norm equals a Mahalanobis distance, making prototype proximity and covariance-aware deviation explicit in network internals (Oursland, 2024). Another line, centered on OOD detection, argues that the empirical success of MDS often stems from anomaly-sensitive geometry that the classifier itself does not directly use for prediction (Kamoi et al., 2020). Taken together, these results suggest that MDS is best regarded not as a single probabilistic model, but as a broad family of covariance-normalized geometric statistics whose effectiveness depends on representation geometry, covariance regularization, and the match between the scoring convention and the target phenomenon.

A recurring misconception is that there exists one definitive Mahalanobis score. The literature instead supports a stratified picture: global scores, class-conditional scores, marginal scores, relative scores, variance-augmented scores, local scores, functional scores, cross-validated scores, and learned metric scores all preserve the same quadratic core while targeting different failure modes and different statistical regimes (Ren et al., 2021, Kim et al., 14 May 2026, Ghosh et al., 2024). This suggests that future work will continue to treat MDS less as a fixed recipe than as a design space for covariance-aware inference.