A Bayesian Lens on Mahalanobis Distance for OOD Detection
This presentation explores how the widely-used Relative Mahalanobis Distance Score for detecting out-of-distribution inputs can be understood as approximate Bayesian inference in a Dirichlet Process Mixture Model. The authors show that this probabilistic interpretation naturally extends to hierarchical models that allow different classes to have different covariance structures, improving detection performance when classes vary in shape or when training data are limited, particularly for near-distribution outliers.Script
Modern AI models often produce high-dimensional feature embeddings, and a critical challenge emerges: how do we detect when a new input falls outside everything the model was trained on?
The popular Relative Mahalanobis Distance Score assumes all training classes share the same covariance structure to avoid overfitting, but this assumption breaks down when different classes have different shapes or variances.
The authors show that a Dirichlet Process Mixture Model provides a principled Bayesian framework for OOD detection by computing the probability that a test point belongs to a previously unseen cluster rather than any known class.
Under tied covariance assumptions, the Relative Mahalanobis Distance Score turns out to be approximately equivalent to the log-odds from Bayesian inference in a Gaussian mixture model, providing a new probabilistic interpretation of this widely-used heuristic.
On the OpenOOD benchmark with ImageNet, the coupled diagonal covariance model achieved the best Near OOD detection performance at 80.98 AUROC, outperforming standard Relative Mahalanobis by allowing classes to have different variances while sharing statistical strength to avoid overfitting.
This Bayesian nonparametric perspective opens a path between two extremes: forcing all classes to look identical versus estimating each class independently, and you can explore the full paper and create your own video explanations at EmergentMind.com.
