Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators

Abstract: Recently, LLMs have shown great potential to transform online medical consultation. Despite this, most research targets improving diagnostic accuracy with ample information, often overlooking the inquiry phase. Some studies try to evaluate or refine doctor models by using prompt-engineered patient agents. However, prompt engineering alone falls short in accurately simulating real patients. We need to explore new paradigms for patient simulation. Furthermore, the relationship between inquiry and diagnosis remains unexplored. This paper extracts dialogue strategies from real doctor-patient conversations to guide the training of a patient simulator. Our simulator shows higher anthropomorphism and lower hallucination rates, using dynamic dialogue strategies. This innovation offers a more accurate evaluation of diagnostic models and generates realistic synthetic data. We conduct extensive experiments on the relationship between inquiry and diagnosis, showing they adhere to Liebig's law: poor inquiry limits diagnosis effectiveness, regardless of diagnostic skill, and vice versa. The experiments also reveal substantial differences in inquiry performance among models. To delve into this phenomenon, the inquiry process is categorized into four distinct types. Analyzing the distribution of inquiries across these types helps explain the performance differences. The weights of our patient simulator are available https://github.com/PatientSimulator/PatientSimulator.

• The paper demonstrates that the quality of patient inquiry significantly impacts the accuracy of diagnosis in online medical consultations.
• The study identified critical inquiry types, including chief complaint and medical history, highlighting key areas for improving AI model performance in online consultations.
• The findings emphasize the need to optimize inquiry strategies in medical AI models to improve patient interactions and outcomes in online consultations.

Analysis of the Inquiry-Diagnosis Dynamics Using Advanced Patient Simulators

The paper "Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators" investigates the dynamic between the inquiry phase and diagnostic accuracy within the context of Online Medical Consultation (OMC). The authors address a pivotal challenge in OMC environments—the lack of direct physical examination, which emphasizes the significance of strategic inquiry for effective diagnosis. Notably, the researchers observe that the current focus in OMC has been disproportionately placed on enhancing the diagnostic processes rather than on optimizing the inquiry phase.

In their methodology, the authors utilize actual doctor-patient communication patterns to refine the performance of a patient simulator. This innovative approach involves extracting and analyzing real conversation strategies to guide the simulator, aiming to mirror authentic patient behaviors. The constructed patient simulator, underpinned by this strategy, was designed to functionally simulate nuanced patient interactions that incorporate emotions and proactive questioning.

The study uncovers a key interdependence between inquiry quality and diagnostic effectiveness, adhering to Liebig's law. This finding underscores that deficiencies in either the inquiry process or diagnostic capabilities constrain the overall consultation outcomes. Moreover, the research highlights notable disparities among models concerning inquiry efficiency, pointing towards chief complaint inquiry, specification of known symptoms, inquiry about accompanying symptoms, and gathering family or medical history as critical types of inquiries. These inquiry dimensions provide a framework for both assessing existing models and illuminating potential enhancements in the inquiry process.

Quantitative experimental results corroborate these findings by demonstrating varying diagnostic accuracies contingent on the quality of the inquiry—irrespective of the model's diagnostic prowess. This result emphasizes the indispensable role of a well-conducted inquiry in achieving accurate medical diagnoses. Consequently, this research calls attention to the necessity of optimizing inquiry strategies within AI models to facilitate high-quality patient interactions in OMC.

The practical implications of these results are multifaceted. In the short term, enhancing inquiry strategies could vastly improve patient satisfaction and outcomes in remote consultations. Theoretically, these insights may propel further research in AI-driven medical interfaces, challenging the community to rethink how conversational agents in healthcare are developed and assessed.

Looking forward, this work lays foundational guidelines for creating more interactive and responsive patient simulators that could be adapted to various medical domains, thereby fostering substantial advancements in AI-driven healthcare services. The authors' pledge to open-source their patient simulator's code further solidifies their contribution to the field, potentially setting a new benchmark for future model development and deployment in medical diagnostics.