NERO: Explainable Out-of-Distribution Detection with Neuron-level Relevance (2506.15404v1)

Abstract: Ensuring reliability is paramount in deep learning, particularly within the domain of medical imaging, where diagnostic decisions often hinge on model outputs. The capacity to separate out-of-distribution (OOD) samples has proven to be a valuable indicator of a model's reliability in research. In medical imaging, this is especially critical, as identifying OOD inputs can help flag potential anomalies that might otherwise go undetected. While many OOD detection methods rely on feature or logit space representations, recent works suggest these approaches may not fully capture OOD diversity. To address this, we propose a novel OOD scoring mechanism, called NERO, that leverages neuron-level relevance at the feature layer. Specifically, we cluster neuron-level relevance for each in-distribution (ID) class to form representative centroids and introduce a relevance distance metric to quantify a new sample's deviation from these centroids, enhancing OOD separability. Additionally, we refine performance by incorporating scaled relevance in the bias term and combining feature norms. Our framework also enables explainable OOD detection. We validate its effectiveness across multiple deep learning architectures on the gastrointestinal imaging benchmarks Kvasir and GastroVision, achieving improvements over state-of-the-art OOD detection methods.

Explainable Out-of-Distribution Detection in Medical Imaging

Out-of-Distribution (OOD) detection in deep learning models, particularly in the domain of medical imaging, is of paramount importance. The paper "NERO: Explainable Out-of-Distribution Detection with Neuron-level Relevance" addresses the critical need for reliable model predictions in medical diagnostics, focusing on gastrointestinal imaging. In the quest for enhancing model reliability, especially when confronting OOD samples—a crucial area where high-confidence, incorrect predictions may lead to severe misdiagnoses—this research introduces a novel approach termed NERO.

Methodology

The paper critiques existing OOD detection methodologies which primarily leverage logit space or feature space representations and suggests a potential inadequacy in capturing the full variance of OOD scenarios. To address these challenges, the authors propose an innovative neuron-level relevance-based scoring mechanism at the feature layer. This technique concentrates on clustering neuron-level relevance scores for each in-distribution class, thus forming representative centroids to facilitate the quantification of deviation for new samples. The relevance distance metric introduced quantifies these deviations, aiming to improve OOD separability substantially.

A notable aspect of NERO is its incorporation of scaled relevance in the bias term alongside feature norms. This addition enhances the model performance, contributing significantly to the detection process. The framework is designed to allow for explainable OOD detection, an essential requirement in sensitive fields like medical imaging where reliability and interpretability cannot be overstated.

Experimental Evaluation

The methodology was validated on various deep learning architectures using gastrointestinal imaging benchmarks such as Kvasir and GastroVision datasets. Across these benchmarks, the proposed NERO framework outperformed the state-of-the-art OOD detection methods, demonstrating superior accuracy in distinguishing OOD from in-distribution samples. Quantitative results showed improvement in metrics like AUROC and FPR95. Notably, NERO achieved an AUROC of 90.76% with ResNet-18 and 92.73% with DeiT, and FPR95 metrics of 28.84% and 18.96% respectively, underscoring its efficacy.

Theoretical and Practical Implications

The use of neuron-level relevance marks a departure from traditional OOD approaches, addressing the shortcomings of feature or logit space methods. This can theoretically offer a more granular understanding of the model's decision-making process by elucidating neuron contributions to the final prediction. Practically, the implications are profound, especially in medical imaging, where reliable model outputs can potentially assist in accurate diagnostics, improving healthcare outcomes.

Furthermore, the explainable nature of NERO is an important advancement towards integrating AI into clinical settings, where the ability to interpret model decisions is crucial. The framework’s reliance on neuron-level analysis dovetails with emerging trends in explainable AI, promising deeper insights into models’ operational intricacies.

Future Directions

Looking ahead, the potential developments in AI following this research could involve expanding the neuron relevance approach to other domains beyond medical imaging, adapting it to different architectures, or even exploring its applicability in other areas of healthcare. Additionally, further fine-tuning to make neuron-level relevance detection even more efficient and accurate could pave the way for broader adoption.

In conclusion, "NERO: Explainable Out-of-Distribution Detection with Neuron-level Relevance" represents a significant contribution to the field of medical imaging, offering an innovative method for improving reliability in critical diagnostic applications. By advancing both the theoretical understanding and practical application of OOD detection, it sets the stage for more trustworthy AI systems in sensitive domains.