Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-rays (2004.08379v3)

Abstract: We demonstrate use of iteratively pruned deep learning model ensembles for detecting pulmonary manifestation of COVID-19 with chest X-rays. This disease is caused by the novel Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) virus, also known as the novel Coronavirus (2019-nCoV). A custom convolutional neural network and a selection of ImageNet pretrained models are trained and evaluated at patient-level on publicly available CXR collections to learn modality-specific feature representations. The learned knowledge is transferred and fine-tuned to improve performance and generalization in the related task of classifying CXRs as normal, showing bacterial pneumonia, or COVID-19-viral abnormalities. The best performing models are iteratively pruned to reduce complexity and improve memory efficiency. The predictions of the best-performing pruned models are combined through different ensemble strategies to improve classification performance. Empirical evaluations demonstrate that the weighted average of the best-performing pruned models significantly improves performance resulting in an accuracy of 99.01% and area under the curve of 0.9972 in detecting COVID-19 findings on CXRs. The combined use of modality-specific knowledge transfer, iterative model pruning, and ensemble learning resulted in improved predictions. We expect that this model can be quickly adopted for COVID-19 screening using chest radiographs.

• The paper presents a novel ensemble approach that iteratively prunes CNN models to enhance COVID-19 detection accuracy in chest X-rays.
• The methodology employs modality-specific transfer learning, achieving 99.01% accuracy and an AUC of 0.9972 for classifying pulmonary conditions.
• Iterative pruning reduces model complexity and improves computational efficiency, making the approach suitable for resource-limited clinical settings.

Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-rays

The paper presents a robust methodology for detecting COVID-19 manifestations in chest X-rays (CXRs) using iteratively pruned deep learning model ensembles. The paper focuses on automating the detection of COVID-19-related thoracic abnormalities through advanced machine learning techniques. By leveraging a combination of custom convolutional neural networks (CNNs) and ImageNet pretrained models, the researchers achieve exceptional performance metrics, facilitating potential applications in clinical screening protocols.

The methodology employs domain-specific transfer learning to adapt pretrained CNNs on large CXR datasets. The preexisting models are finetuned to categorize CXRs as normal, exhibiting bacterial pneumonia or having COVID-19-related abnormalities. The ensemble method implemented significantly augments the performance, evidenced by a reported accuracy of 99.01% and an area under the curve (AUC) of 0.9972, suggesting highly precise discrimination capabilities between COVID-19 and other pulmonary conditions.

Key Methodological Insights

1. Modality-Specific Transfer Learning: The transfer learning strategy involved fine-tuning CNNs on a diverse dataset of CXRs, thereby enhancing the feature representation through training on CXR-specific attributes. This results in an effective initialization of model weights, allowing for improved fine-tuning on the COVID-19 classification tasks.
2. Iterative Pruning Technique: The novelty of the approach lies in iteratively pruning the top-performing models. By systematically removing neurons with low activations, the method optimizes model architecture, reducing the network complexity while maintaining high prediction accuracy. This not only shrinks the model size but also enhances computational efficiency, making deployment feasible on limited-resource platforms.
3. Ensemble Learning: Several ensemble strategies were examined, including averaging and stacking. However, weighted averaging was noted to be particularly effective, indicating robustness against model variance. By combining the predictions of the pruned models, the ensemble approach capitalized on each model's strengths, ensuring improved generalizability and reliability over individual networks.
4. Visualization and Interpretability: The paper integrates Grad-CAM visualization techniques to illustrate the decision-making process of the models. This aspect ensures model transparency and aids in verifying the model's ability to focus on clinically relevant areas within CXRs that are crucial indicators of COVID-19 infections.

Implications and Future Directions

From a practical standpoint, the implementation of pruned ensembles offers a promising avenue for rapid COVID-19 screenings, especially pertinent in regions with constrained radiological resources. The method's efficiency in memory usage and computational requirements suggests potential adaptability to mobile devices, potentially broadening its clinical application during ongoing pandemics. Furthermore, the approach could be extended to analyze other forms of viral pneumonia, offering a versatile framework for thoracic disease diagnostics.

The theoretical implications of iterative pruning in deep learning extend beyond this application, possibly serving as a blueprint for optimizing other large-scale neural network models in diverse domains. However, the efficacy of such ensembles is naturally contingent on the availability and variability of training datasets, as well as relevant computational resources to facilitate the required extensive model training and evaluation phases.

Overall, this paper advances the understanding and capabilities of AI-driven diagnostics, highlighting the potential impact iterative model pruning and ensemble learning can have on the efficiency and performance of deep learning applications in medical imaging. Future research might explore the integration of these models with other diagnostic modalities such as CT scans, providing comprehensive, multimodal screening tools that leverage the strengths of each individual imaging technique.