README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP (2312.15561v5)
Abstract: The advancement in healthcare has shifted focus toward patient-centric approaches, particularly in self-care and patient education, facilitated by access to Electronic Health Records (EHR). However, medical jargon in EHRs poses significant challenges in patient comprehension. To address this, we introduce a new task of automatically generating lay definitions, aiming to simplify complex medical terms into patient-friendly lay language. We first created the README dataset, an extensive collection of over 50,000 unique (medical term, lay definition) pairs and 300,000 mentions, each offering context-aware lay definitions manually annotated by domain experts. We have also engineered a data-centric Human-AI pipeline that synergizes data filtering, augmentation, and selection to improve data quality. We then used README as the training data for models and leveraged a Retrieval-Augmented Generation method to reduce hallucinations and improve the quality of model outputs. Our extensive automatic and human evaluations demonstrate that open-source mobile-friendly models, when fine-tuned with high-quality data, are capable of matching or even surpassing the performance of state-of-the-art closed-source LLMs like ChatGPT. This research represents a significant stride in closing the knowledge gap in patient education and advancing patient-centric healthcare solutions.
- A meta-evaluation of faithfulness metrics for long-form hospital-course summarization. arXiv preprint arXiv:2303.03948.
- Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511.
- Giving patients their own records in general practice: experience of patients and staff. Br Med J (Clin Res Ed), 292(6520):596–598.
- Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
- Susan B Bastable. 2016. Essentials of patient education. Jones & Bartlett Learning.
- Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucleic acids research, 32(suppl_1):D267–D270.
- Peter A. Boling. 2009. Care transitions and home health care. Clinics in Geriatric Medicine, 25(1):135–148. The Past, Present and Future of Home Health Care.
- Language models are few-shot learners.
- Paniniqa: Enhancing patient education through interactive question answering. arXiv preprint arXiv:2308.03253.
- A natural language processing system that links medical terms in electronic health record notes to lay definitions: system development using physician reviews. Journal of medical Internet research, 20(1):e26.
- A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 65(5):545–563.
- Angela Coulter. 2012. Patient engagement—what works? The Journal of ambulatory care management, 35(2):80–89.
- Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA: A Cancer Journal for Clinicians, 48(3):151–162.
- Teaching patients with low literacy skills. AJN The American Journal of Nursing, 96(12):16M.
- Readability of patient education materials on the american association for surgery of trauma website. Archives of trauma research, 3(2).
- Thomas Golper. 2001. Patient education: can it maximize the success of therapy? Nephrology Dialysis Transplantation, 16(suppl_7):20–24.
- From patient education to patient engagement: Implications for the field of patient education. Patient Education and Counseling, 78(3):350–356. Changing Patient Education.
- Enriching consumer health vocabulary through mining a social q&a site: A similarity-based approach. Journal of biomedical informatics, 69:75–85.
- Enriching consumer health vocabulary using enhanced glove word embedding. arXiv preprint arXiv:2004.00150.
- Sanjay Krishnan and Eugene Wu. 2019. Alphaclean: Automatic generation of data cleaning pipelines. arXiv preprint arXiv:1904.11827.
- Medjex: A medical jargon extraction model with wiki’s hyperlink span and contextualized masked language model score. arXiv preprint arXiv:2210.05875.
- Evaluating the effectiveness of noteaid in a community hospital setting: Randomized trial of electronic health record note comprehension interventions with patients. Journal of medical Internet research, 23(5):e26354.
- Comprehenotes, an instrument to assess patient reading comprehension of electronic health record notes: development and validation. Journal of medical Internet research, 20(4):e139.
- Evaluating the efficacy of noteaid on ehr note comprehension among us veterans through amazon mechanical turk. International Journal of Medical Informatics, page 105006.
- Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
- Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
- BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6). Bbac409.
- Giving Patients Access to Their Medical Records via the Internet: The PCASSO Experience. Journal of the American Medical Informatics Association, 9(2):181–191.
- Understanding patient–provider conversations: what are we talking about? Academic Emergency Medicine, 20(5):441–448.
- Synthetic imitation edit feedback for factual alignment in clinical summarization. arXiv preprint arXiv:2310.20033.
- Robert Munro Monarch. 2021. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster.
- Data-centric ai competition. DeepLearning AI.
- OpenAI. 2023. Gpt-4 technical report.
- Karen Patrias and Dan Wendling. 2007. Citing Medicine:. Department of Health and Human Services, National Institutes of Health, US ….
- Secure multiparty computation for synthetic data generation from distributed data. arXiv preprint arXiv:2210.07332.
- Alina Petrova. 2014. Learning formal definitions for biomedical concepts. Ph.D. thesis, Master thesis. Technische Universität Dresden, Germany. Alina Petrova 51.
- Formalizing biomedical concepts from textual definitions. Journal of biomedical semantics, 6:1–17.
- Language models are unsupervised multitask learners.
- Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084.
- François Remy and Thomas Demeester. 2023. Automatic glossary of clinical terminology: a large-scale dictionary of biomedical definitions generated from ontological knowledge. arXiv preprint arXiv:2306.00665.
- Effects of self-management support on structure, process, and outcomes among vulnerable patients with diabetes: a three-arm practical clinical trial. Diabetes care, 32(4):559–566.
- A source data privacy framework for synthetic clinical trial data. In NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research.
- Advanced practice profiles and work activities of nurse navigators: An early-stage evaluation. Collegian, 26(1):103–109.
- Can language models be biomedical knowledge bases? arXiv preprint arXiv:2109.07154.
- Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
- Data collection and quality challenges in deep learning: A data-centric ai perspective. The VLDB Journal, 32(4):791–813.
- Performance of multimodal gpt-4v on usmle with image: Potential for imaging diagnostic support with explanations. medRxiv, pages 2023–10.
- Extracting biomedical factual knowledge using pretrained language model and electronic health record context. In AMIA Annual Symposium Proceedings, volume 2022, page 1188. American Medical Informatics Association.
- Context variance evaluation of pretrained language models for prompt-based biomedical knowledge probing. AMIA Summits on Translational Science Proceedings, 2023:592.
- Qing T Zeng and Tony Tse. 2006. Exploring and developing consumer health vocabularies. Journal of the American Medical Informatics Association, 13(1):24–29.
- Data-centric artificial intelligence: A survey. arXiv preprint arXiv:2303.10158.
- Ehr interaction between patients and ai noteaid ehr interaction. AAAI2024 Workshop on AI for Education (AI4ED).
- Ehrtutor: Enhancing patient understanding of discharge instructions. arXiv preprint arXiv:2310.19212.