Development of Machine Learning Classifiers for Blood-based Diagnosis and Prognosis of Suspected Acute Infections and Sepsis

Abstract: We applied machine learning to the unmet medical need of rapid and accurate diagnosis and prognosis of acute infections and sepsis in emergency departments. Our solution consists of a Myrna (TM) Instrument and embedded TriVerity (TM) classifiers. The instrument measures abundances of 29 messenger RNAs in patient's blood, subsequently used as features for machine learning. The classifiers convert the input features to an intuitive test report comprising the separate likelihoods of (1) a bacterial infection (2) a viral infection, and (3) severity (need for Intensive Care Unit-level care). In internal validation, the system achieved AUROC = 0.83 on the three-class disease diagnosis (bacterial, viral, or non-infected) and AUROC = 0.77 on binary prognosis of disease severity. The Myrna, TriVerity system was granted breakthrough device designation by the United States Food and Drug Administration (FDA). This engineering manuscript teaches the standard and novel machine learning methods used to translate an academic research concept to a clinical product aimed at improving patient care, and discusses lessons learned.

• The paper demonstrates an ML-based blood test that leverages 29 mRNAs to differentiate bacterial, viral, and non-infectious conditions while assessing sepsis severity.
• It employs a comprehensive methodology combining diverse gene expression platforms and grouped cross-validation to overcome batch effects and ensure clinical applicability.
• The system achieved AUROCs of 0.83 for infection classification and 0.77 for severity, highlighting its strong potential for rapid diagnosis in emergency departments.

Development of Machine Learning Classifiers for Blood-based Diagnosis and Prognosis of Suspected Acute Infections and Sepsis

Introduction

This paper addresses a critical gap in the medical field: the translation of ML innovations for diagnosing and prognosing acute infections and sepsis into clinical practice. The authors present a comprehensive study on the development of a blood-based diagnostic system consisting of the Myrna™ Instrument and embedded TriVerity™ classifiers. The solution utilizes the abundance of 29 messenger RNAs (mRNAs) in patient blood samples to generate intuitive test reports detailing the probabilities of bacterial or viral infections and the severity of the disease.

Methodology

Diagnosing infections in emergency departments (ED) involves two primary classification tasks: determining the type of infection (bacterial, viral, or non-infected) and assessing illness severity. The classifiers leverage gene expression data as input features and are trained using both publicly available and proprietary datasets, ensuring a robust and clinically relevant model.

The ground truth for infection type classification was derived from clinical adjudication, while the severity classification used 30-day survival data. Various platforms, including microarrays, RNA-Seq, and molecular barcoding technology (NanoString®), were employed for gene expression measurement during training, with Myrna™ using Loop-Mediated Isothermal Amplification (LAMP) as the target platform in clinical settings.

Batch Effects and Platform Transfer

The study data came from diverse sources, with variability in hospitals, gene expression measurement platforms, and patient demographics. To manage platform-induced batch effects, the training and validation data were grouped by platform. Cross-validation (CV) was performed using grouped approaches to mitigate the impact of study-induced batch effects, ensuring the classifiers' robustness across different settings.

Platform transfer is a crucial aspect, defined as the ability of classifiers trained on one platform to generalize to the Myrna™ platform. The Concordance Filtering (CF) approach was developed to address this, filtering out classifiers that failed to achieve adequate concordance between platforms during hyperparameter tuning.

Results

The classifiers demonstrated strong performance metrics. The BVN (bacterial, viral, non-infected) classifier achieved an AUROC of 0.83 in validation, and the SEV (severity) classifier achieved an AUROC of 0.77. These metrics underscore the classifiers' effectiveness in a clinical setting. The decision thresholds for the classifiers were optimized using a Genetic Algorithm for Thresholds (GAT), facilitating a fine balance between sensitivity and specificity in clinical applications.

Implications and Future Work

The successful development and validation of the Myrna™ and TriVerity™ system represent a significant step toward the practical implementation of ML in diagnosing and prognosing infections in clinical settings. The system's robustness across different platforms and patient populations suggests it is well-suited for deployment in emergency departments globally.

Future work will focus on prospective evaluations of the classifiers' clinical validity and utility through pivotal and randomized trials. The implications of this work are substantial, potentially leading to more rapid and accurate diagnoses, improved patient outcomes, and optimized resource allocation in healthcare settings.

Conclusion

This study provides a detailed account of developing and validating ML classifiers for diagnosing and prognosing acute infections and sepsis using blood-based mRNA measurements. The Myrna™ and TriVerity™ system demonstrates strong potential for clinical adoption, meeting the stringent requirements for accuracy, robustness, and FDA compliance. The ongoing and future trials will further establish the system's clinical utility and guide its broader implementation in healthcare.