Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection

Abstract: Deep Learning has achieved state of the art performance in medical imaging. However, these methods for disease detection focus exclusively on improving the accuracy of classification or predictions without quantifying uncertainty in a decision. Knowing how much confidence there is in a computer-based medical diagnosis is essential for gaining clinicians trust in the technology and therefore improve treatment. Today, the 2019 Coronavirus (SARS-CoV-2) infections are a major healthcare challenge around the world. Detecting COVID-19 in X-ray images is crucial for diagnosis, assessment and treatment. However, diagnostic uncertainty in the report is a challenging and yet inevitable task for radiologist. In this paper, we investigate how drop-weights based Bayesian Convolutional Neural Networks (BCNN) can estimate uncertainty in Deep Learning solution to improve the diagnostic performance of the human-machine team using publicly available COVID-19 chest X-ray dataset and show that the uncertainty in prediction is highly correlates with accuracy of prediction. We believe that the availability of uncertainty-aware deep learning solution will enable a wider adoption of AI in a clinical setting.

• The paper introduces a Dropweights-based Bayesian CNN that quantifies predictive uncertainty in COVID-19 detection.
• It applies Monte Carlo Dropweights with transfer learning from a pre-trained ResNet50V2 to enhance classification accuracy.
• The results show that higher uncertainty correlates with misclassified cases, boosting diagnostic reliability and trust in AI.

Estimating Uncertainty and Interpretability in Deep Learning for COVID-19 Detection

This paper focuses on addressing an important aspect of AI applications in medical imaging: the quantification of uncertainty in predictive models. The study specifically targets the detection of COVID-19 from chest X-ray images using Deep Learning (DL) models with Bayesian techniques to estimate uncertainty. The primary contribution of this paper lies in the integration of Dropweights-based Bayesian Convolutional Neural Networks (BCNN) to enhance the interpretability and reliability of AI-assisted medical diagnostics.

Methodological Approach

The authors employ a Bayesian modeling framework, more precisely, Dropweights-based Bayesian Convolutional Neural Networks, to estimate and quantify uncertainty in deep learning models. In this context, uncertainty estimation serves a dual purpose. It provides additional insights into point predictions while enhancing clinicians' trust in AI models by distinguishing between certain and uncertain predictions.

Key methodologies include the use of Monte-Carlo Dropweights (MC Dropweights) for uncertainty estimation and a pre-trained ResNet50V2 model leveraged within a transfer learning paradigm. The BCNN model approximates predictive uncertainty by averaging stochastic Monte Carlo samples during inference. The Bayesian approach allows for the separation of epistemic (model-based) and aleatoric (data-based) uncertainties, which is critical in the real-world deployment of AI models in healthcare settings.

The study utilizes a composite dataset formed by augmenting publicly available COVID-19 X-ray images with Kaggle’s Chest X-ray Images dataset to train and validate their model.

Experimental Results

The experimental outcomes provide robust evidence of the utility of uncertainty estimation in enhancing classification accuracy. Key findings include:

• A significant correlation between the predictive uncertainty and the accuracy of predictions.
• The Bayesian model yielded a higher classification accuracy than conventional models (e.g., standard ResNet50V2), indicating the effectiveness of uncertainty estimation in improving model performance.
• Higher uncertainty measures were associated with incorrect predictions, which aids in identifying cases that require human intervention.

Utilizing Predictive Entropy (PH) as a measure of uncertainty, the Spearman's correlation coefficient demonstrated a strong relationship between uncertainty and prediction errors, underscoring the potential of uncertainty measures to act as reliable indicators for the accuracy of predictions.

Implications and Future Work

The implications of this research are multi-fold. Practically, the research provides a pathway to more reliable AI diagnostics by enabling a system that can flag uncertain cases for further investigation, thereby reducing false negatives. Theoretically, the study underscores the importance of incorporating Bayesian methods in DL models to address uncertainty quantification effectively.

The paper suggests a promising direction for future research in AI for medical imaging. Prospective investigations might explore integrating multi-modal data, such as genomics and radiomics (termed "multi-omics"), to further refine diagnostic accuracy and potentially unveil novel biomarkers for diagnostic and therapeutic purposes.

Lastly, visualizing and interpreting the uncertainty serves as a step toward demystifying deep learning models, often criticized as "black boxes," thereby fostering greater clinical acceptance and trust in AI technologies. Future studies could further the development of intuitive and insightful visualizations to complement uncertainty estimates, enhancing the interpretability of AI-assisted decisions.

In conclusion, the paper makes a significant technical contribution to medical AI by effectively demonstrating the integration and application of uncertainty quantification in deep learning models for COVID-19 detection, a nugget of insight pertinent to researchers and practitioners aiming to leverage AI within clinical workflows.