Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

WisPerMed at "Discharge Me!": Advancing Text Generation in Healthcare with Large Language Models, Dynamic Expert Selection, and Priming Techniques on MIMIC-IV (2405.11255v1)

Published 18 May 2024 in cs.CL and cs.LG

Abstract: This study aims to leverage state of the art LLMs to automate generating the "Brief Hospital Course" and "Discharge Instructions" sections of Discharge Summaries from the MIMIC-IV dataset, reducing clinicians' administrative workload. We investigate how automation can improve documentation accuracy, alleviate clinician burnout, and enhance operational efficacy in healthcare facilities. This research was conducted within our participation in the Shared Task Discharge Me! at BioNLP @ ACL 2024. Various strategies were employed, including few-shot learning, instruction tuning, and Dynamic Expert Selection (DES), to develop models capable of generating the required text sections. Notably, utilizing an additional clinical domain-specific dataset demonstrated substantial potential to enhance clinical language processing. The DES method, which optimizes the selection of text outputs from multiple predictions, proved to be especially effective. It achieved the highest overall score of 0.332 in the competition, surpassing single-model outputs. This finding suggests that advanced deep learning methods in combination with DES can effectively automate parts of electronic health record documentation. These advancements could enhance patient care by freeing clinician time for patient interactions. The integration of text selection strategies represents a promising avenue for further research.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Phi-3 technical report: A highly capable language model locally on your phone. Preprint, arXiv:2404.14219.
  2. AI@Meta. 2024. Llama 3. https://github.com/meta-llama/llama3/. Accessed: 2024-05-15.
  3. Malaikannan Sankarasubbu Ankit Pal. 2024. OpenBioLLMs: Advancing open-source large language models for healthcare and life sciences. https://huggingface.co/aaditya/OpenBioLLM-Llama3-70B. Accessed: 2024-05-15.
  4. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
  5. Readability Revisited: The New Dale-Chall Readability Formula, volume 1. Brookline Books.
  6. Chatbot arena: An open platform for evaluating llms by human preference. Preprint, arXiv:2403.04132.
  7. Meri Coleman and Ta Lin Liau. 1975. A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2):283–284.
  8. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  9. What’s up with llama 3? arena data analysis. Accessed: 2024-05-15.
  10. Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220.
  11. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  12. Mistral 7B. Preprint, arXiv:2310.06825.
  13. MIMIC-IV.
  14. Mimic-iv, a freely accessible electronic health record dataset. Scientific Data, 10:1.
  15. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel.
  16. Publicly shareable clinical large language model built on synthetic clinical notes. Preprint, arXiv:2309.00237.
  17. SummaC: Re-visiting nli-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163–177.
  18. Quantifying the carbon emissions of machine learning. CoRR, abs/1910.09700.
  19. DeepZensols: A deep learning natural language processing framework for experimentation and reproducibility. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 141–146, Singapore, Singapore. Empirical Methods in Natural Language Processing.
  20. A new public corpus for clinical section identification: MedSecId. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3709–3721, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  21. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  22. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net.
  23. Large language models: A survey. arXiv preprint arXiv:2402.06196.
  24. OpenAI. 2023. ChatGPT. https://www.openai.com. Accessed: 2024-05-15.
  25. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  26. SB Patel and K Lam. 2023. Chatgpt: the future of discharge summaries? Lancet Digit Health, 5(3):e107–e108. Epub 2023 Feb 6.
  27. Instruction tuning with gpt-4. Preprint, arXiv:2304.03277.
  28. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108.
  29. Discharge summary hospital course summarisation of in patient electronic health record text with clinical concept guided deep pre-trained transformer models. Journal of Biomedical Informatics, 141:104358.
  30. Allocation of physician time in ambulatory practice: A time and motion study in 4 specialties. Annals of Internal Medicine, 165(11):753.
  31. Experimental standards for deep learning in natural language processing research. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2673–2692, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  32. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys, 53(3).
  33. WizardLM: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations.
  34. J. Xu. 2024. Discharge me: Bionlp acl’24 shared task on streamlining discharge documentation.
  35. Overview of the first shared task on clinical text generation: Rrg24 and “discharge me!”. In The 23rd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, Bangkok, Thailand. Association for Computational Linguistics.
  36. Aci-bench: A novel ambient clinical intelligence dataset for benchmarking automatic visit note generation. Scientific Data, 10(1):586.
  37. AlignScore: Evaluating factual consistency with a unified alignment function. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11328–11348, Toronto, Canada. Association for Computational Linguistics.
  38. BERTScore: Evaluating text generation with bert. In International Conference on Learning Representations.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Hendrik Damm (2 papers)
  2. Tabea M. G. Pakull (2 papers)
  3. Bahadır Eryılmaz (2 papers)
  4. Helmut Becker (3 papers)
  5. Ahmad Idrissi-Yaghir (6 papers)
  6. Henning Schäfer (6 papers)
  7. Sergej Schultenkämper (1 paper)
  8. Christoph M. Friedrich (17 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets