DDO: Dual-Decision Optimization via Multi-Agent Collaboration for LLM-Based Medical Consultation

Abstract: LLMs demonstrate strong generalization and reasoning abilities, making them well-suited for complex decision-making tasks such as medical consultation (MC). However, existing LLM-based methods often fail to capture the dual nature of MC, which entails two distinct sub-tasks: symptom inquiry, a sequential decision-making process, and disease diagnosis, a classification problem. This mismatch often results in ineffective symptom inquiry and unreliable disease diagnosis. To address this, we propose \textbf{DDO}, a novel LLM-based framework that performs \textbf{D}ual-\textbf{D}ecision \textbf{O}ptimization by decoupling and independently optimizing the the two sub-tasks through a collaborative multi-agent workflow. Experiments on three real-world MC datasets show that DDO consistently outperforms existing LLM-based approaches and achieves competitive performance with state-of-the-art generation-based methods, demonstrating its effectiveness in the MC task.

• The paper introduces a dual-decision optimization approach that decouples symptom inquiry from disease diagnosis for improved accuracy.
• It employs four collaborative agents—Diagnosis, Policy, Inquiry, and Patient—to efficiently process and aggregate decision-making.
• The framework achieves 94.2% accuracy on the DXY dataset, demonstrating superior performance with reduced model complexity.

DDO: Dual-Decision Optimization via Multi-Agent Collaboration for LLM-Based Medical Consultation

The paper "DDO: Dual-Decision Optimization via Multi-Agent Collaboration for LLM-Based Medical Consultation" presents a novel framework aimed at optimizing LLM-based medical consultation tasks by decoupling symptom inquiry and disease diagnosis. The DDO framework utilizes multiple collaborative agents to enhance decision-making capabilities essential for medical consultations, an application that requires both effective information gathering and precise diagnostic capabilities.

Framework and Methodology

The DDO framework is comprised of four main agents: Diagnosis Agent, Policy Agent, Inquiry Agent, and Patient Agent, all operating over a shared memory. This architecture allows for a structured consultation process where each agent plays a specific role:

• The Diagnosis Agent estimates disease confidences utilizing Binary Token Probability (BTP), a logit-based method that efficiently assigns confidence scores to potential diagnoses.
• The Policy Agent generates reliable candidate actions using an external RL-based policy model structured through masked sampling.
• The Inquiry Agent strategically selects symptoms to inquire about, leveraging the candidate actions produced by the Policy Agent.
• The Patient Agent simulates realistic patient responses, either indicating the presence or absence of symptoms.

  Figure 1: Overview of the proposed DDO framework, comprising four collaborative agents operating over a shared memory to execute the consultation workflow.

Performance Evaluation

Diagnostic Performance

DDO excels in aligning high interim diagnostic confidence with accurate final diagnosis, significantly outperforming existing LLM-based methods in various datasets. It achieves a notable 94.2% accuracy on the DXY dataset, showcasing its adeptness in symptom inquiry and diagnosis. The framework also maintains competitive accuracy compared to state-of-the-art generation-based methods such as MTDiag and HAIformer, yet with reduced training complexities due to the efficient use of LLMs.

Efficiency in Information Gathering

The framework's ability to decouple and independently optimize symptom inquiry and diagnosis allows for more targeted and effective information gathering. Compared to traditional methods, DDO provides substantial gains in diagnostic accuracy with fewer model parameters, thanks to the collaborative nature of its multi-agent setup.

Exploration of Multi-Turn Symptom Inquiry

The framework's performance improves with an increased number of turns in patient interaction, suggesting that it effectively accumulates critical diagnostic evidence over time. However, the diagnostic gains diminish slightly as the inquiry reaches a saturation point, particularly evident in extensive datasets like GMD.

Figure 2: Effect of max turns L.

Ablation and Comparative Studies

A comprehensive ablation study reveals that each agent's function is integral to the overall framework's performance. Removing the diagnostic adapter or disabling the multi-agent collaboration results in significant drops in accuracy, particularly highlighting the importance of the collaborative inquiry process.

Conclusion

In summary, the DDO framework provides a robust, LLM-based multi-agent system for medical consultations by independently optimizing the dual aspects of symptom inquiry and disease diagnosis. It leverages multi-agent collaboration to enhance both the efficiency and accuracy of medical decision-making tasks. The framework shows promise in improving AI-driven healthcare applications, setting a foundation for future developments that could include more sophisticated LLM integration and exploration of API-based LLM deployment strategies to increase accessibility and efficiency in clinical environments.