COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction (2101.04909v2)

Published 13 Jan 2021 in cs.CV and cs.LG

Abstract: The rapid spread of COVID-19 cases in recent months has strained hospital resources, making rapid and accurate triage of patients presenting to emergency departments a necessity. Machine learning techniques using clinical data such as chest X-rays have been used to predict which patients are most at risk of deterioration. We consider the task of predicting two types of patient deterioration based on chest X-rays: adverse event deterioration (i.e., transfer to the intensive care unit, intubation, or mortality) and increased oxygen requirements beyond 6 L per day. Due to the relative scarcity of COVID-19 patient data, existing solutions leverage supervised pretraining on related non-COVID images, but this is limited by the differences between the pretraining data and the target COVID-19 patient data. In this paper, we use self-supervised learning based on the momentum contrast (MoCo) method in the pretraining phase to learn more general image representations to use for downstream tasks. We present three results. The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0.742 for predicting an adverse event within 96 hours (compared to 0.703 with supervised pretraining) and an AUC of 0.765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0.749 with supervised pretraining). We then propose a new transformer-based architecture that can process sequences of multiple images for prediction and show that this model can achieve an improved AUC of 0.786 for predicting an adverse event at 96 hours and an AUC of 0.848 for predicting mortalities at 96 hours. A small pilot clinical study suggested that the prediction accuracy of our model is comparable to that of experienced radiologists analyzing the same information.

Authors (10)

Anuroop Sriram (32 papers)
Matthew Muckley (12 papers)
Koustuv Sinha (31 papers)
Farah Shamout (5 papers)
Joelle Pineau (123 papers)
Krzysztof J. Geras (31 papers)
Lea Azour (2 papers)
Yindalon Aphinyanaphongs (11 papers)
Nafissa Yakubova (5 papers)
William Moore (9 papers)

Citations (60)

View on Semantic Scholar

Summary

Analysis of COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction

The paper, "COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction," provides a sophisticated approach to predicting patient deterioration in the context of COVID-19 using chest X-ray imaging. This work is situated in the dynamic field of AI-driven medical imaging, where machine learning models enhance clinical decision-making by assessing risk in healthcare settings. The authors have utilized advanced machine learning techniques to address the scarcity of labeled COVID-19 patient data and propose robust models capable of predicting patient deterioration through innovative architectures and training methodologies.

Methodology and Core Findings

The authors employ self-supervised learning to overcome limitations associated with traditional supervised pretraining methods. In particular, the paper introduces the momentum contrast (MoCo) method during the pretraining phase to learn more generalized image representations without requiring extensive labeled data. This approach benefits from contrastive loss functions that facilitate feature extraction independent of pretraining dataset labels or tasks.

Key results from this paper include an AUC of 0.742 for predicting adverse events within 96 hours using single-image prediction, an improvement over the 0.703 AUC achieved with supervised pretraining. Additionally, an AUC of 0.765 was reached for predicting oxygen requirements greater than 6 L/day at 24 hours, compared to 0.749 with supervised methods. The paper further proposes a transformer-based architecture capable of processing sequences of chest X-rays, significantly enhancing performance to an AUC of 0.786 for predicting adverse events at 96 hours and an AUC of 0.848 for mortalities at 96 hours.

The implementation of sequence-based prediction using transformers represents a notable shift towards incorporating temporal changes in imaging, akin to radiologist evaluations, which rely on examining changes across multiple scans to project patient trajectories.

Implications and Future Directions

The implications of this research are twofold. Practically, these models can assist in effective allocation of hospital resources during pandemics by improving triage processes. Theoretically, the integration of self-supervised methods and sequence-based prediction contributes to a deeper understanding of contrastive learning potential in medical imaging.

Potential future developments may explore further integration of weak supervision strategies or clustering methods alongside contrastive loss, capitalizing on emergent advances in self-supervised learning. Additionally, expanding datasets and optimizing model architectures could enhance predictive capabilities, leading to more generalized and robust applications across varied clinical settings.

Conclusion

This paper exemplifies a nuanced approach to AI applications in healthcare, particularly within the context of a global pandemic. By harnessing self-supervised learning and transformer models, the authors advance both the technical capabilities and clinical applications of AI-driven radiology, paving the way for more adaptive and responsive healthcare solutions. This research underscores the value of adapting machine learning techniques to the demands of clinical environments, reflecting a pivotal evolution in AI's role within medical imaging diagnostics and patient care.

PDF Markdown

Related Papers

Find Related Papers

YouTube

Show All Videos