Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection (2409.17485v1)

Abstract: Medical anomaly detection (AD) is crucial in pathological identification and localization. Current methods typically rely on uncertainty estimation in deep ensembles to detect anomalies, assuming that ensemble learners should agree on normal samples while exhibiting disagreement on unseen anomalies in the output space. However, these methods may suffer from inadequate disagreement on anomalies or diminished agreement on normal samples. To tackle these issues, we propose D2UE, a Diversified Dual-space Uncertainty Estimation framework for medical anomaly detection. To effectively balance agreement and disagreement for anomaly detection, we propose Redundancy-Aware Repulsion (RAR), which uses a similarity kernel that remains invariant to both isotropic scaling and orthogonal transformations, explicitly promoting diversity in learners' feature space. Moreover, to accentuate anomalous regions, we develop Dual-Space Uncertainty (DSU), which utilizes the ensemble's uncertainty in input and output spaces. In input space, we first calculate gradients of reconstruction error with respect to input images. The gradients are then integrated with reconstruction outputs to estimate uncertainty for inputs, enabling effective anomaly discrimination even when output space disagreement is minimal. We conduct a comprehensive evaluation of five medical benchmarks with different backbones. Experimental results demonstrate the superiority of our method to state-of-the-art methods and the effectiveness of each component in our framework. Our code is available at https://github.com/Rubiscol/D2UE.

Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection

The paper "Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection" by Yi Gu, Yi Lin, Kwang-Ting Cheng, and Hao Chen introduces innovative methodologies aimed at advancing the efficacy of medical anomaly detection (AD) using deep learning ensembles. This work addresses limitations in existing AD techniques, particularly those that hinge on uncertainty estimation within deep ensembles.

Core Contributions

The authors present a novel framework named Diversified Dual-space Uncertainty Estimation (D2UE) designed to enhance the performance of ensemble learning approaches in detecting medical anomalies. The framework is built on two key components: Redundancy-Aware Repulsion (RAR) and Dual-Space Uncertainty (DSU).

1. Redundancy-Aware Repulsion (RAR): RAR is engineered to promote diversity among ensemble learners without causing neural network redundancy. It ensures that learners trained on normal data capture different feature spaces, thereby ensuring disagreement on anomalies. This is achieved using a similarity kernel that is invariant to isotropic scaling and orthogonal transformations.
2. Dual-Space Uncertainty (DSU): DSU aims to enhance anomaly detection by exploiting uncertainties in both input and output spaces. It calculates gradients of reconstruction error concerning input images and integrates these gradients with outputs to estimate uncertainty. This approach ensures that anomalies can be detected even when there is minimal disagreement among ensemble learners in the output space.

Experimental Validation

The authors validate their methodology using five medical imaging benchmarks, covering a spectrum of data modalities including chest X-rays, brain MRI, and retinal fundus images. These datasets were studied to evaluate both image-level classification performance (AUC and AP metrics) and specific anomaly detection capabilities.

• Datasets: The research utilizes RSNA Pneumonia Detection Challenge, VinBigData Chest X-ray Abnormalities Detection, Chest X-ray Anomaly Detection (CXAD), Brain Tumor MRI, and Large-scale Attention-based Glaucoma (LAG) datasets.
• Results: Experimental outcomes demonstrate that the proposed D2UE framework consistently outperforms state-of-the-art methods. For instance, compared to the baseline models, D2UE achieved improvements in AUC and AP metrics across most datasets. Specifically, with the AEU backbone, D2UE exhibited performance enhancements such as 1.3% AUC and 0.4% AP for RSNA, 2.1% AUC and 1.3% AP for VinDr-CXR, 3.7% AUC for CXAD, and 2.1% AUC and 5.0% AP for the LAG dataset. These results underscore the method's enhanced detection accuracy and robustness.

Implications and Future Work

The paper presents several implications for both practical implementation and theoretical exploration:

• Practical Implications: The integration of RAR and DSU within ensemble frameworks demonstrates a more reliable approach to medical AD, potentially leading to more accurate early disease detection and pathological localization in clinical settings. The ability of D2UE to maintain high performance across various medical image datasets signals robustness and adaptability.
• Theoretical Implications: By addressing the simplicity bias and promoting learner diversity with redundancy-aware mechanisms, this work contributes to foundational aspects in ensemble learning theory. It provides a feasible solution to balancing agreement on normal samples and disagreement on anomalies.

Future Directions

The framework's promising results open several avenues for future research:

• Quantitative Analysis: Future work could explore the quantitative relationship between ensemble diversity and AD performance, potentially identifying more granular factors that influence model efficacy.
• Efficiency Enhancements: Optimization of the framework to reduce computational overhead and training time without compromising detection accuracy is an essential next step for translating this research into practical, real-world applications.
• Extension to Other Domains: While this paper focuses on medical imaging, the principles and methodologies could be adapted for anomaly detection in other domains, such as industrial inspection and cybersecurity, broadening the impact of the research.

In conclusion, "Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection" provides substantial advancements in the field of medical anomaly detection. The introduction of the D2UE framework, with its innovative components RAR and DSU, marks a significant contribution to achieving more accurate and reliable AD through deep ensemble learning. The empirical results and proposed methodology offer a robust foundation for continued exploration and application in broader contexts.