MKA: A Scalable Medical Knowledge Assisted Mechanism for Generative Models on Medical Conversation Tasks (2312.02496v1)

Abstract: Using NLP technologies to develop medical chatbots makes the diagnosis of the patient more convenient and efficient, which is a typical application in healthcare AI. Because of its importance, lots of research have been come out. Recently, the neural generative models have shown their impressive ability as the core of chatbot, while it cannot scale well when directly applied to medical conversation due to the lack of medical-specific knowledge. To address the limitation, a scalable Medical Knowledge Assisted mechanism, MKA, is proposed in this paper. The mechanism aims to assist general neural generative models to achieve better performance on the medical conversation task. The medical-specific knowledge graph is designed within the mechanism, which contains 6 types of medical-related information, including department, drug, check, symptom, disease, food. Besides, the specific token concatenation policy is defined to effectively inject medical information into the input data. Evaluation of our method is carried out on two typical medical datasets, MedDG and MedDialog-CN. The evaluation results demonstrate that models combined with our mechanism outperform original methods in multiple automatic evaluation metrics. Besides, MKA-Bert-GPT achieves state-of-the-art performance. The open-sourced codes are public: https://github.com/LIANGKE23/Knowledge_Assisted_Medical_Dialogue_Generation_Mechanism

• The paper introduces the MKA mechanism, which integrates a six-category medical knowledge graph to improve generative models for medical conversations.
• It demonstrates clear performance gains by reducing perplexity and boosting BLEU, NIST, and METEOR scores compared to conventional methods.
• The findings suggest that incorporating domain-specific knowledge can lead to more accurate and reliable AI-driven conversational agents in healthcare.

MKA: A Scalable Medical Knowledge Assisted Mechanism for Generative Models on Medical Conversation Tasks

The paper "MKA: A Scalable Medical Knowledge Assisted Mechanism for Generative Models on Medical Conversation Tasks" by Ke Liang, Sifan Wu, and Jiayi Gu presents a novel mechanism to enhance the performance of neural generative models applied in medical conversation tasks. The authors propose the Medical Knowledge Assisted (MKA) mechanism, which addresses a notable gap in current generative models: their limited capacity to scale effectively in medical contexts due to the lack of domain-specific knowledge.

Introduction

The primary motivation for this research is rooted in the practical challenges facing medical practice, such as difficulty in accessing medical services, lengthy queuing times, and complexities in scheduling appointments. The integration of AI technologies, particularly NLP, offers potential solutions. Despite recent advancements in neural generative models, their direct application to medical conversations is hindered by insufficient incorporation of medical-specific knowledge. This paper presents the MKA mechanism as an intervention to bridge this gap.

Main Contributions

1. Medical-Specific Knowledge Graph:
   • The MKA mechanism is designed to integrate a medical-specific knowledge graph encompassing six types of medical-related information: departments, drugs, check-ups, symptoms, diseases, and food.
   • This graph is used to inject domain-specific knowledge into the input data via a defined token concatenation policy.
2. Evaluation and Results:
   • The proposed MKA mechanism is evaluated on two datasets, MedDG and MedDialog-CN.
   • The inclusion of MKA resulted in consistent performance improvements across a variety of automatic evaluation metrics.

Experimental Results

The paper presents robust numerical evidence supporting the efficacy of the MKA mechanism:

• Perplexity: MKA-based models (MKA-Transformer: 5.95, MKA-BERT-GPT: 5.95) exhibit reduced perplexity compared to their non-MKA counterparts (Dialog-GPT: 8.53, Transformer: 8.52, BERT-GPT: 5.98).
• BLEU Scores: MKA-BERT-GPT achieved superior BLEU-2 (8.09%) and BLEU-4 (2.87%) scores, indicating better overlap between generated responses and ground truth.
• NIST Scores: Modest improvements were observed in NIST-2 and NIST-4 scores with the adoption of MKA, demonstrating enhanced precision.
• METEOR: MKA-BERT-GPT achieved the highest METEOR score at 16.63%, highlighting greater linguistic quality.
• Diversity Metrics: The inclusion of MKA also led to minor improvements in diversity as evidenced by Entropy and Distinct-1/2 scores.

Implications

The integration of medical-specific knowledge into neural generative models significantly enhances their ability to perform in medical dialogue settings. This implies a substantial potential for improving patient interaction systems, offering more accurate and informative conversational agents for healthcare. The enhanced performance metrics suggest that MKA-based models are better equipped to handle the complexities inherent in medical conversations, such as providing relevant medical advice and understanding nuanced medical symptoms and conditions.

Future Directions

The authors suggest several avenues for future exploration:

1. Graph Neural Networks:
   • Applying graph neural networks to further enhance the extraction and prediction of medical knowledge, potentially leading to even more accurate and contextually appropriate dialogue systems.
2. Hybrid Approaches:
   • Combining information retrieval methods with neural generative methods to optimize dialogue generation systems.

These future developments, if realized, may lead to even more responsive and contextually aware medical conversational agents, thereby improving the overall efficiency and effectiveness of healthcare delivery through AI.

Conclusion

The MKA mechanism represents a significant step forward in the application of neural generative models to medical conversation tasks. By addressing the domain-specific knowledge gap, the proposed mechanism not only enhances current model performance but also sets a foundation for future research in hybrid dialogue generation systems. Overall, the empirical results demonstrate the practical scalability and effectiveness of MKA in real-world medical conversation datasets, paving the way for more advanced AI-driven healthcare solutions.