Hypothesis-Driven Deep Learning for Out of Distribution Detection (2403.14058v1)

Abstract: Predictions of opaque black-box systems are frequently deployed in high-stakes applications such as healthcare. For such applications, it is crucial to assess how models handle samples beyond the domain of training data. While several metrics and tests exist to detect out-of-distribution (OoD) data from in-distribution (InD) data to a deep neural network (DNN), their performance varies significantly across datasets, models, and tasks, which limits their practical use. In this paper, we propose a hypothesis-driven approach to quantify whether a new sample is InD or OoD. Given a trained DNN and some input, we first feed the input through the DNN and compute an ensemble of OoD metrics, which we term latent responses. We then formulate the OoD detection problem as a hypothesis test between latent responses of different groups, and use permutation-based resampling to infer the significance of the observed latent responses under a null hypothesis. We adapt our method to detect an unseen sample of bacteria to a trained deep learning model, and show that it reveals interpretable differences between InD and OoD latent responses. Our work has implications for systematic novelty detection and informed decision-making from classifiers trained on a subset of labels.

• The paper reformulates OoD detection as a hypothesis testing problem by ensembling diverse deep learning metrics on latent responses.
• It leverages the Multi-Response Permutation Procedure (MRPP) to quantify differences between in-distribution and out-of-distribution data.
• Experiments on MNIST, CIFAR10, and AMRB datasets show improved detection consistency and reliability across various model architectures.

Hypothesis-Driven Deep Learning for Out-of-Distribution Detection

Introduction

The reliability of black-box machine learning models, particularly deep neural networks (DNNs), in high-stakes domains such as healthcare, is critically dependent on their ability to recognize and handle data that differ significantly from the training samples. Current methodologies for Out-of-Distribution (OoD) detection have shown varied success across different datasets, models, and tasks, which limits their applicability in practice. This paper introduces a novel hypothesis-driven framework for OoD detection that leverages hypothesis testing principles to evaluate whether a new sample can be considered In-Distribution (InD) or OoD based on latent response patterns obtained from a trained DNN.

Related Work

Prior efforts in OoD detection have attempted to either enhance metric robustness, propose new tests for detecting OoD samples, or adjust the training processes to better accommodate OoD data. Despite these advancements, a standard, universally effective solution remains elusive due to the significant variability in the performance of these metrics across different data types and model structures. This paper posits that an ensemble approach, combining multiple OoD metrics, can provide improved consistency and discrimination power for OoD detection.

Methodology

The authors propose a method that subjects a model's latent responses to an ensemble of OoD metrics, hence converting the task of OoD detection into a hypothesis testing problem. Utilizing the Multi-Response Permutation Procedure (MRPP) enables quantification of differences between groups of latent responses corresponding to InD and OoD samples. The MRPP statistic measures the dissimilarity within and between these groups, offering a formal mechanism to ascertain the OoD nature of new samples with statistical significance.

Ensembling OoD Detection Metrics

The core of the proposed methodology involves generating a multidimensional metric space for each input sample by extracting a set of responses from the model's hidden layers. These responses are then subject to hypothesis testing to assess their alignment with InD or OoD characteristics. Metrics such as K Nearest Neighbor Distance, Reconstruction Error, and Distance to Data Manifold are considered for constructing this ensemble, each providing a different perspective on the model's reaction to novel inputs.

Experimental Design

The researchers conducted experiments using various datasets, including benchmark datasets (MNIST and CIFAR10) and a domain-specific dataset (AMRB) comprising single-cell bacteria images. They trained multiple model architectures, including classifier-only, auto-encoder-only, and hybrid models, on subsets of these datasets. The effectiveness of the proposed hypothesis testing framework was measured in terms of its ability to distinguish between InD and OoD data.

Evaluation

The results confirmed the efficacy of the ensemble approach in improving the consistency and reliability of OoD detection across different data types and model architectures. Particularly notable was the framework's performance on the domain-specific AMRB dataset, where it demonstrated the potential for practical application in healthcare scenarios, such as the identification of unknown bacterial species.

Conclusion

This paper presents a significant step towards formalizing the OoD detection problem within a hypothesis testing framework, showcasing a method that is both model-agnostic and adaptable to various metric choices. The proposed method's ability to discern subtle differences in latent response patterns between InD and OoD data represents an advancement in the field of machine learning, improving model reliability in critical applications.

The authors convincingly demonstrate that by employing a carefully selected ensemble of OoD metrics and subjecting them to hypothesis-driven statistical testing, it is possible to achieve a higher level of precision in identifying OoD samples. This not only enhances the interpretability of OoD detection results but also supports more informed decision-making in the deployment of models in real-world scenarios.

Future work may explore the integration of additional OoD metrics into the ensemble, further enhancing the method's robustness. Additionally, extending the framework's applicability to broader domains and model architectures will be crucial for its adoption in diverse practical applications, particularly those involving high stakes and requiring high reliability.