Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Clinical Risk Prediction Using Language Models: Benefits And Considerations (2312.03742v1)

Published 29 Nov 2023 in cs.CL and cs.LG

Abstract: The utilization of Electronic Health Records (EHRs) for clinical risk prediction is on the rise. However, strict privacy regulations limit access to comprehensive health records, making it challenging to apply standard machine learning algorithms in practical real-world scenarios. Previous research has addressed this data limitation by incorporating medical ontologies and employing transfer learning methods. In this study, we investigate the potential of leveraging LLMs (LMs) as a means to incorporate supplementary domain knowledge for improving the performance of various EHR-based risk prediction tasks. Unlike applying LMs to unstructured EHR data such as clinical notes, this study focuses on using textual descriptions within structured EHR to make predictions exclusively based on that information. We extensively compare against previous approaches across various data types and sizes. We find that employing LMs to represent structured EHRs, such as diagnostic histories, leads to improved or at least comparable performance in diverse risk prediction tasks. Furthermore, LM-based approaches offer numerous advantages, including few-shot learning, the capability to handle previously unseen medical concepts, and adaptability to various medical vocabularies. Nevertheless, we underscore, through various experiments, the importance of being cautious when employing such models, as concerns regarding the reliability of LMs persist.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Using Electronic Health Records To Generate Phenotypes For Research. Current Protocols in Human Genetics, page e80, December 2018.
  2. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J. Am. Med. Inform. Assoc., 24(1):198–208, January 2017.
  3. Gram: Graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, page 787–795. Association for Computing Machinery, 2017.
  4. Pre-training of graph augmented transformers for medication recommendation. CoRR, abs/1906.00346, 2019.
  5. Icd-10: History and context.
  6. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digital Medicine, 4(1):86, December 2021.
  7. RareBERT: Transformer Architecture for Rare Disease Patient Identification using Administrative Claims. Proceedings of the AAAI Conference on Artificial Intelligence, 35:453–460, May 2021.
  8. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, May 2019.
  9. BEHRT: Transformer for Electronic Health Records. Scientific Reports, 10:7155, December 2020.
  10. Feijun Luo. State-Level Economic Costs of Opioid Use Disorder and Fatal Opioid Overdose — United States, 2017. MMWR. Morbidity and Mortality Weekly Report, 70, 2021.
  11. Gensyn: A multi-stage framework for generating synthetic microdata using macro data sources. In 2022 IEEE International Conference on Big Data (Big Data), pages 685–692, 2022.
  12. Exploring county-level spatio-temporal patterns in opioid overdose related emergency department visits. PLOS ONE, 17(12):1–15, 12 2022.
  13. A Thomas McLellan. Substance misuse and substance use disorders: Why do they matter in healthcare? Trans. Am. Clin. Climatol. Assoc., 128:112–130, 2017.
  14. Llama: Open and efficient foundation language models, 2023.
  15. Palm: Scaling language modeling with pathways, 2022.
  16. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020.
  17. Normalized names for clinical drugs: Rxnorm at 6 years, July 2011.
  18. Machine learning based opioid overdose prediction using electronic health records. AMIA. In Annual Symposium proceedings. AMIA Symposium, pages 389–398. 2019.
  19. Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. JMLR workshop and conference proceedings, 56:301–318, August 2016.
  20. Patient2vec: A personalized interpretable deep representation of the longitudinal electronic health record. IEEE Access, 2018.
  21. Prediction of hospitalization due to heart diseases by supervised learning methods. International Journal of Medical Informatics, 2015.
  22. Sreekanth Rallapalli and T. Suryakanthi. Predicting the risk of diabetes in big data electronic health records by using scalable random forest classification algorithm. In 2016 International Conference on Advances in Computing and Communication Engineering (ICACCE), pages 281–284, 2016.
  23. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 3512–3520. Curran Associates Inc., 2016.
  24. Benchmarking deep learning architectures for predicting readmission to the icu and describing patients-at-risk. Scientific Reports, 2020.
  25. Mimic-iv, 2021.
  26. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific Data, 5:180178, September 2018.
  27. A comparison of deep learning methods for icd coding of clinical records. Applied Sciences, 10(15), 2020.
  28. Homicidal event forecasting and interpretable analysis using hierarchical attention model. In Robert Thomson, Halil Bisgin, Christopher Dancy, Ayaz Hyder, and Muhammad Hussain, editors, Social, Cultural, and Behavioral Modeling, pages 140–150, Cham, 2020. Springer International Publishing.
  29. Cehr-bert: Incorporating temporal information from structured ehr data to improve prediction tasks. In Subhrajit Roy, Stephen Pfohl, Emma Rocheteau, Girmaw Abebe Tadesse, Luis Oala, Fabian Falck, Yuyin Zhou, Liyue Shen, Ghada Zamzmi, Purity Mugambi, Ayah Zirikly, Matthew B. A. McDermott, and Emily Alsentzer, editors, Proceedings of Machine Learning for Health, volume 158 of Proceedings of Machine Learning Research, pages 239–260. PMLR, 04 Dec 2021.
  30. Clinicalbert: Modeling clinical notes and predicting hospital readmission. CoRR, abs/1904.05342, 2019.
  31. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records, 2022.
  32. Use of the systematized nomenclature of medicine clinical terms (snomed ct) for processing free text in health care: Systematic scoping review. J Med Internet Res, Jan 2021.
  33. MIMIC-IV.
  34. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. Journal of the American Medical Informatics Association, 2017.
  35. A deep learning method to detect opioid prescription and opioid use disorder from electronic health records. International Journal of Medical Informatics, 171:104979, 2023.
  36. Deep ehr: A survey of recent advances on deep learning techniques for electronic health record (ehr) analysis. CoRR, abs/1706.03446, 2017.
  37. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. 2019. Publisher: arXiv Version Number: 1.
  38. Triplet Loss in Siamese Network for Object Tracking. In Computer Vision – ECCV 2018, volume 11217, pages 472–488. Springer International Publishing, 2018.
  39. Universal Sentence Encoder. 2018. Publisher: arXiv Version Number: 2.
  40. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019.
  41. A comprehensive overview of large language models, 2023.
  42. Understanding catastrophic forgetting and remembering in continual learning with optimal relevance mapping. CoRR, abs/2102.11343, 2021.
  43. Self-supervised pre-training for semantic segmentation in an indoor scene, 2022.
  44. Sources of hallucination by large language models on inference tasks, 2023.
  45. Yarn: Efficient context window extension of large language models, 2023.
  46. Extending context window of large language models via positional interpolation, 2023.
  47. Focal loss for dense object detection. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2999–3007, 2017.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Angeela Acharya (2 papers)
  2. Sulabh Shrestha (3 papers)
  3. Anyi Chen (1 paper)
  4. Joseph Conte (1 paper)
  5. Sanja Avramovic (1 paper)
  6. Siddhartha Sikdar (8 papers)
  7. Antonios Anastasopoulos (111 papers)
  8. Sanmay Das (19 papers)
Citations (4)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets