Doctor AI: Predicting Clinical Events via Recurrent Neural Networks (1511.05942v11)

Abstract: Leveraging large historical data in electronic health record (EHR), we developed Doctor AI, a generic predictive model that covers observed medical conditions and medication uses. Doctor AI is a temporal model using recurrent neural networks (RNN) and was developed and applied to longitudinal time stamped EHR data from 260K patients over 8 years. Encounter records (e.g. diagnosis codes, medication codes or procedure codes) were input to RNN to predict (all) the diagnosis and medication categories for a subsequent visit. Doctor AI assesses the history of patients to make multilabel predictions (one label for each diagnosis or medication category). Based on separate blind test set evaluation, Doctor AI can perform differential diagnosis with up to 79% recall@30, significantly higher than several baselines. Moreover, we demonstrate great generalizability of Doctor AI by adapting the resulting models from one institution to another without losing substantial accuracy.

• The paper introduces Doctor AI, a RNN-based model that predicts clinical events with a recall@30 of up to 79% for diagnosis codes.
• It employs GRUs and Skip-gram embeddings to process sequential EHR data and generate multilabel predictions for diagnoses and medications.
• Transfer learning allows the model to generalize across institutions, improving predictive accuracy by over 10% on new datasets.

Doctor AI: Predicting Clinical Events via Recurrent Neural Networks

Abstract

The paper "Doctor AI: Predicting Clinical Events via Recurrent Neural Networks" presents a sophisticated temporal model tailored for the prediction of clinical events utilizing Recurrent Neural Networks (RNNs). The model, referred to as Doctor AI, leverages longitudinal electronic health record (EHR) data encompassing diagnoses, medication orders, and procedure codes. The evaluation over 260K patients reveals that Doctor AI achieves a recall@30 of up to 79%, outperforming traditional baselines with significant margins. Additionally, the paper evidences the capacity of transfer learning to adapt models across different medical institutions while preserving predictive accuracy.

Introduction

A prominent issue within healthcare is the deluge of patient data juxtaposed with limited computational tools for real-time interpretation. Traditional predictive models are often narrowly focused, limited to specific outcomes, thus not reflecting the multifaceted nature of day-to-day clinical practice. In response to this, the authors developed Doctor AI to provide a comprehensive predictive model capable of handling the heterogeneity of medical scenarios by utilizing RNNs.

Methods

Historical patient data from Sutter Health’s EHR system was examined, and Doctor AI was trained to predict future diagnoses and medication codes. Doctor AI’s design enables multilabel predictions, addressing the complexity of clinical events that involve multiple diagnostic and treatment categories at each patient visit. The RNN model employed in this work is specifically configured with Gated Recurrent Units (GRUs), known for their efficacy in handling sequential data. A distinctive feature is the use of Skip-gram embeddings for initializing the RNN, enhancing both accuracy and training speed.

Results

Numerical Results

The paper articulates the results of Doctor AI’s predictions through various experimental setups:

• Prediction Performance
  • Diagnosis Prediction: Doctor AI achieved a recall@30 of 79.58% for diagnosis codes, a significant improvement over baseline models. Notably, even without using Skip-gram embeddings, the RNN with two hidden layers reached a recall@30 of 78.71%.
  • Medication Prediction: For medication orders, the model obtained a recall@30 of 85.48%.
  • Joint Prediction: Doctor AI’s joint model for predicting diagnoses, medications, and visit timing exhibited robust performance across all tasks.
• Transfer Learning
  • The capability of the RNN model to transfer learned knowledge from a large dataset to a smaller one was also demonstrated. Transferring a model pre-trained on Sutter Health data to the MIMIC II dataset yielded a performance improvement exceeding 10%.

Model Implications

Doctor AI's pragmatic contributions are multi-fold:

1. Clinical Decision Support: The ability to predict diagnoses and medications based on historical patient data contributes to more informed and timely decisions during clinical practice.
2. Operational Efficiency: Predicting time to the next visit helps in identifying patients who may delay seeking care, enabling proactive interventions.
3. Transferability: The model's demonstrated ability to generalize across different institutions without substantial accuracy loss opens avenues for its application even in settings with limited data availability.

Future Directions

The paper elucidates potential advancements in enhancing Doctor AI, addressing current limitations such as optimizing time prediction accuracy and improving the model’s predictive power beyond average physician performance. Exploration into mechanisms that mitigate the impact of incorrect predictions on patient outcomes is proposed as a future direction.

Conclusion

Doctor AI embodies a significant stride in the application of RNNs for clinical event prediction, offering enhanced recall performance and underscoring the practical utility of transfer learning in medical data analytics. While promising, the continuous refinement and evaluation of the model in varied clinical environments are essential for its evolution into a robust tool that can aid physicians in making more precise and effective clinical decisions.

References

1. Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F., & Sun, J. (Year). Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. Journal/Conference.

Appendices

The appendices include detailed descriptions of the RNN architecture, methodology for Skip-gram embeddings in EHR data, and nuanced cases illustrating Doctor AI’s predictive accuracy in real-world scenarios. The empirical evidence underpins the potential improvements in clinical outcomes driven by advanced predictive modeling.