Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation (2402.13408v1)

Published 20 Feb 2024 in cs.CL

Abstract: The copilot framework, which aims to enhance and tailor LLMs for specific complex tasks without requiring fine-tuning, is gaining increasing attention from the community. In this paper, we introduce the construction of a Healthcare Copilot designed for medical consultation. The proposed Healthcare Copilot comprises three main components: 1) the Dialogue component, responsible for effective and safe patient interactions; 2) the Memory component, storing both current conversation data and historical patient information; and 3) the Processing component, summarizing the entire dialogue and generating reports. To evaluate the proposed Healthcare Copilot, we implement an auto-evaluation scheme using ChatGPT for two roles: as a virtual patient engaging in dialogue with the copilot, and as an evaluator to assess the quality of the dialogue. Extensive results demonstrate that the proposed Healthcare Copilot significantly enhances the capabilities of general LLMs for medical consultations in terms of inquiry capability, conversational fluency, response accuracy, and safety. Furthermore, we conduct ablation studies to highlight the contribution of each individual module in the Healthcare Copilot. Code will be made publicly available on GitHub.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Gpt-4 technical report. arXiv preprint.
  2. Palm 2 technical report. arXiv preprint.
  3. Language models are few-shot learners. In NeurIPS.
  4. Bianque: Balancing the questioning and suggestion ability of health llms with multi-turn health conversations polished by chatgpt. arXiv preprint.
  5. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna.lmsys.org (accessed 14 April 2023).
  6. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.
  7. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence.
  8. Glm: General language model pretraining with autoregressive blank infilling. In ACL.
  9. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint.
  10. Medalpaca–an open-source collection of medical conversational ai models and training data. arXiv preprint.
  11. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. arXiv preprint.
  12. Sparseadapter: An easy approach for improving the parameter-efficiency of adapters. In Findings of EMNLP.
  13. The virtual patient as a learning tool: a mixed quantitative qualitative study. BMC Medical Education, 18(1):1–10.
  14. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  15. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint.
  16. Prompted llms as chatbot modules for long open-domain conversation. In ACL.
  17. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).
  18. Yen-Ting Lin and Yun-Nung Chen. 2023. Llm-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models. In NLP4ConvAI.
  19. Biosignal copilot: Leveraging the power of llms in drafting reports for biomedical signals. medRxiv, pages 2023–06.
  20. Gpteval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint.
  21. Error analysis prompting enables human-like translation evaluation in large language models: A case study on chatgpt. arXiv preprint.
  22. Becky McCall. 2020. Could telemedicine solve the cancer backlog? The Lancet Digital Health, 2(9):e456–e457.
  23. Swaroop Mishra and Elnaz Nouri. 2022. Help me think: A simple prompting strategy for non-experts to create customized content with models. In ACL.
  24. Training language models to follow instructions with human feedback. In NeurIPS.
  25. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In CHIL, pages 248–260. PMLR.
  26. A study of generative large language model for medical research and healthcare. arXiv preprint.
  27. Revisiting demonstration selection strategies in in-context learning. arXiv preprint.
  28. Towards making the most of ChatGPT for machine translation. In Findings of EMNLP.
  29. Crest–copilot for real-world experimental scientist. ChemRxiv.
  30. On efficient training of large-scale deep learning models: A literature review. arXiv preprint.
  31. Large language models encode clinical knowledge. Nature, 620(7972):172–180.
  32. Towards expert-level medical question answering with large language models. arXiv preprint.
  33. Evaluation metrics in the era of GPT-4: reliably evaluating large language models on sequence to sequence tasks. arXiv preprint.
  34. Mirac Suzgun and Adam Tauman Kalai. 2024. Meta-prompting: Enhancing language models with task-agnostic scaffolding. arXiv preprint.
  35. Gemini: a family of highly capable multimodal models. arXiv preprint.
  36. Llama: Open and efficient foundation language models. arXiv preprint.
  37. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint.
  38. Towards conversational diagnostic ai. arXiv preprint.
  39. Recursively summarizing enables long-term dialogue memory in large language models. arXiv preprint.
  40. Chatgpt as your vehicle co-pilot: An initial attempt. IEEE Transactions on Intelligent Vehicles.
  41. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS.
  42. Doctorglm: Fine-tuning your chinese doctor is not a herculean task. arXiv preprint.
  43. Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. arXiv preprint.
  44. Gatortron: A large clinical language model to unlock patient information from unstructured electronic health records. arXiv preprint.
  45. Meddialog: Large-scale medical dialogue datasets. In EMNLP, pages 9241–9250.
  46. Huatuogpt, towards taming language model to be a doctor. In EMNLP.
  47. GPTBIAS: A comprehensive framework for evaluating bias in large language models. arXiv preprint.
  48. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zhiyao Ren (2 papers)
  2. Yibing Zhan (73 papers)
  3. Baosheng Yu (51 papers)
  4. Liang Ding (159 papers)
  5. Dacheng Tao (829 papers)
Citations (8)
View on Semantic Scholar