Med42-v2: A Suite of Clinical LLMs (2408.06142v1)

Abstract: Med42-v2 introduces a suite of clinical LLMs designed to address the limitations of generic models in healthcare settings. These models are built on Llama3 architecture and fine-tuned using specialized clinical data. They underwent multi-stage preference alignment to effectively respond to natural prompts. While generic models are often preference-aligned to avoid answering clinical queries as a precaution, Med42-v2 is specifically trained to overcome this limitation, enabling its use in clinical settings. Med42-v2 models demonstrate superior performance compared to the original Llama3 models in both 8B and 70B parameter configurations and GPT-4 across various medical benchmarks. These LLMs are developed to understand clinical queries, perform reasoning tasks, and provide valuable assistance in clinical environments. The models are now publicly available at \href{https://huggingface.co/m42-health}{https://huggingface.co/m42-health}.

• The paper introduces Med42-v2, an enhanced clinical LLM suite that surpasses GPT-4 on multiple medical benchmarks.
• It employs a two-stage training process combining instruction fine-tuning with chain-of-thought reasoning and Direct Preference Optimization.
• The models promise improved diagnostic accuracy and clinical decision support, while addressing common ethical and data quality challenges.

Med42-v2: A Suite of Clinical LLMs

The paper introduces Med42-v2, a collection of clinical LLMs based on the Llama3 architecture, designed to address the shortcomings of general-purpose models in healthcare environments. These models are fine-tuned with domain-specific clinical data and undergo a multi-stage preference alignment process, enabling them to provide effective responses to clinical queries. The Med42-v2 suite includes models with 8B and 70B parameter configurations, and their performance surpasses the original Llama3 models and GPT-4 on multiple medical benchmarks.

Introduction

Recent advancements in LLMs have demonstrated their capabilities across a wide range of applications. However, their deployment in specialized fields, particularly healthcare, remains limited due to the need for in-depth domain knowledge and adherence to strict ethical and safety guidelines. Generic LLMs suffer from hallucinations, biases, and privacy concerns, which hinder their effectiveness in medical applications, such as diagnostics, interpretation of medical literature, and generation of patient education materials.

To mitigate these issues, Med42-v2 is introduced, which refines the generic model architecture of Llama3 through fine-tuning with specialized clinical data and preference alignment, thereby enhancing its suitability for clinical settings. Comparative analysis indicates that Med42-v2 significantly outperforms the baseline Llama3 models and GPT-4 across various medical benchmarks.

Method

The development of Med42-v2 involves a two-stage training process aimed at creating specialized clinical LLMs:

1. Instruction Fine-Tuning:
   • Involves fine-tuning Llama3 and Llama3.1 models with a curated clinical dataset.
   • The training dataset includes a diverse collection of medical and biomedical resources, augmented with examples of chain-of-thought reasoning and chat interactions to enhance the model's conversational and reasoning abilities.
   • The prompt format adheres to the established format of the Llama3 models, and the training was conducted over two epochs with specific hyperparameters detailed in the paper.
2. Preference Alignment:
   • Utilizes Direct Preference Optimization (DPO) to align the clinically fine-tuned models with human preferences.
   • Preference datasets like UltraFeedback and Snorkel-DPO were used to refine the model's responses through iterative alignment.
   • Hyperparameters and a staged iterative alignment approach were employed to progressively enhance the model's performance.

Benchmarks

Med42-v2's performance was evaluated using Eleuther AI’s evaluation harness framework on several medical benchmarks, including MMLU, MMLU-Pro, MedMCQA, MedQA, USMLE, PubmedQA, and ToxiGen. The results indicate:

• Med42-v2 models exhibit substantial improvements across all benchmarks compared to their predecessors.
• In zero-shot performance, Med42-v2 models notably outperform other general-purpose and domain-specific models, including proprietary models like GPT-4.

These results highlight the advantages of targeted medical instruction and alignment, underscoring the model's enhanced clinical knowledge and reasoning capabilities. However, the performance improvements are less significant on safety-focused benchmarks like ToxiGen.

Conclusions and Limitations

Med42-v2 represents a significant step forward in the development of clinical LLMs, demonstrating enhanced performance over the original Llama3 models and GPT-4. Despite these advancements, challenges remain, particularly in mitigating hallucinations, biases, and ethical concerns inherent in deploying LLMs in healthcare settings. The reliance on high-quality domain-specific data introduces potential risks associated with data gaps and biases.

Future work will focus on developing a new evaluation framework to assess the clinical utility of LLMs in real-world scenarios, emphasizing clinical data understanding, safety, and reasoning capabilities. This rigorous evaluation aims to identify and address potential risks, ensuring that models like Med42-v2 can be safely and effectively integrated into healthcare environments.

The release of Med42-v2 models, now publicly available, represents a valuable resource for the broader research community, providing a foundation for further exploration and refinement of clinical LLMs. This ongoing development promises to enhance the application of AI in healthcare, improving diagnostic accuracy, patient education, and clinical decision support.