Aloe: A Family of Fine-tuned Open Healthcare LLMs (2405.01886v1)

Abstract: As the capabilities of LLMs in healthcare and medicine continue to advance, there is a growing need for competitive open-source models that can safeguard public interest. With the increasing availability of highly competitive open base models, the impact of continued pre-training is increasingly uncertain. In this work, we explore the role of instruct tuning, model merging, alignment, red teaming and advanced inference schemes, as means to improve current open models. To that end, we introduce the Aloe family, a set of open medical LLMs highly competitive within its scale range. Aloe models are trained on the current best base models (Mistral, LLaMA 3), using a new custom dataset which combines public data sources improved with synthetic Chain of Thought (CoT). Aloe models undergo an alignment phase, becoming one of the first few policy-aligned open healthcare LLM using Direct Preference Optimization, setting a new standard for ethical performance in healthcare LLMs. Model evaluation expands to include various bias and toxicity datasets, a dedicated red teaming effort, and a much-needed risk assessment for healthcare LLMs. Finally, to explore the limits of current LLMs in inference, we study several advanced prompt engineering strategies to boost performance across benchmarks, yielding state-of-the-art results for open healthcare 7B LLMs, unprecedented at this scale.

• The paper introduces a family of fine-tuned healthcare LLMs that balance competitive performance with rigorous ethical safeguards.
• The paper employs state-of-the-art base models and synthetic Chain of Thought data to optimize domain-specific performance through advanced inference techniques.
• The paper demonstrates superior accuracy on medical benchmarks while mitigating unsafe outputs via Direct Preference Optimization and targeted red teaming.

Aloe: A Family of Fine-tuned Open Healthcare LLMs

The paper "Aloe: A Family of Fine-tuned Open Healthcare LLMs" presents the development of the Aloe models, a series of open-source, LLMs specifically fine-tuned for the healthcare domain. As the landscape of LLMs continues to expand, the need for models that not only perform competitively but also adhere to ethical standards becomes paramount, especially in sensitive areas like healthcare. This work provides an intricate overview of the methodological advancements implemented to enhance the efficacy of open healthcare LLMs, focusing not just on performance, but also on ethical alignment and safety.

Key Contributions and Methodology

Central to the paper is the advancement in model training and fine-tuning strategies adapted for Aloe models. The researchers utilize state-of-the-art base models such as Mistral and LLaMA 3, which are further refined using a custom dataset enriched with synthetic Chain of Thought (CoT) data. This approach aids in the instruct tuning phase, aimed at optimizing the models’ domain-specific adaptability through supervised fine-tuning (SFT) and advanced inference techniques. Moreover, model merging methods are implemented, blending different instantiations to leverage individual capabilities, thereby achieving a robust and high-performing composite model.

One of the distinctive aspects of the Aloe project is its strong emphasis on ethical performance and fairness, achieved through policy alignment via Direct Preference Optimization (DPO). This positions Aloe among the first healthcare LLMs to incorporate preference alignment with policy adjustments, heralding a new benchmark in ethical AI deployment in the medical domain. The alignment phase of Aloe improves the handling of biased and potentially toxic responses, thanks to a targeted red teaming initiative aimed at identifying and mitigating unsafe outputs.

The advanced inference scheme explored includes novel prompt engineering strategies that significantly boost Aloe's performance on medical benchmarks, setting state-of-the-art results for models at the 7B scale. This illustrates that even optimized versions of smaller models can exceed specific performance thresholds previously attributed only to larger models.

Evaluation and Performance

Through comprehensive evaluations using medical benchmarks such as MedQA, MedMCQA, and PubMedQA, Aloe models not only demonstrate superior accuracy compared to many existing open-source healthcare models but also achieve significant safety and robustness enhancements. Additionally, the paper highlights the effects of advanced prompt engineering techniques, such as the Medprompt, which integrates task-specific embeddings to enhance the model's context sensitivity and accuracy.

The researchers conduct a thorough ablation paper within the Aloe family, elucidating the impact of various training stages and methodological choices, including model merging and DPO alignment, showcasing their cumulative contribution to the model's impressive performance benchmarks.

Implications and Future Directions

The implications of the Aloe family are twofold: practically, these models reduce the dependency on proprietary healthcare models by offering a competitive, open alternative. Theoretically, this work underscores the significance of integrating robust alignment protocols in the deployment of healthcare LLMs, fostering a safer and ethically conscious AI ecosystem.

Looking forward, the Aloe project establishes several avenues for future research. These include extending the scope of alignment methodologies to counteract novel adversarial tactics continually. Similarly, further exploration into the scalability of these models across diverse medical domains can empower broader and more inclusive healthcare applications.

In summary, the Aloe models represent a significant step toward democratizing access to sophisticated, reliable, and ethically attuned LLMs in healthcare, ensuring that advancements in AI contribute positively to societal welfare without compromising ethical standards.