DocCHA: Towards LLM-Augmented Interactive Online diagnosis System (2507.07870v1)

Abstract: Despite the impressive capabilities of LLMs, existing Conversational Health Agents (CHAs) remain static and brittle, incapable of adaptive multi-turn reasoning, symptom clarification, or transparent decision-making. This hinders their real-world applicability in clinical diagnosis, where iterative and structured dialogue is essential. We propose DocCHA, a confidence-aware, modular framework that emulates clinical reasoning by decomposing the diagnostic process into three stages: (1) symptom elicitation, (2) history acquisition, and (3) causal graph construction. Each module uses interpretable confidence scores to guide adaptive questioning, prioritize informative clarifications, and refine weak reasoning links. Evaluated on two real-world Chinese consultation datasets (IMCS21, DX), DocCHA consistently outperforms strong prompting-based LLM baselines (GPT-3.5, GPT-4o, LLaMA-3), achieving up to 5.18 percent higher diagnostic accuracy and over 30 percent improvement in symptom recall, with only modest increase in dialogue turns. These results demonstrate the effectiveness of DocCHA in enabling structured, transparent, and efficient diagnostic conversations -- paving the way for trustworthy LLM-powered clinical assistants in multilingual and resource-constrained settings.

• The paper presents a novel modular framework, DocCHA, that transforms static LLM-based CHAs into adaptive diagnostic tools.
• It decomposes the diagnostic process into three confidence-driven stages—symptom collection, history acquisition, and causal graph construction—to enhance clinical dialogue.
• Evaluation on Chinese datasets shows DocCHA improves diagnostic accuracy by up to 5.18% and symptom recall by over 30%, promising greater healthcare efficiency.

DocCHA: Towards LLM-Augmented Interactive Online Diagnosis System

The paper "DocCHA: Towards LLM-Augmented Interactive Online Diagnosis System" proposes a novel framework, DocCHA, that aims to transform the static capabilities of current LLM-based Conversational Health Agents (CHAs) into dynamic, adaptive systems capable of robust clinical dialogue. By decomposing the diagnostic process into three modular stages—symptom elicitation, history acquisition, and causal graph construction—DocCHA represents a significant step towards achieving transparency, accountability, and interaction efficiency in healthcare applications.

Motivation and Background

Traditional CHAs often depend on static prompt-based strategies, which are insufficient for real-world medical diagnosis due to their inability to dynamically adapt and reason iteratively about patient input (Figure 1). The process of clinical reasoning necessitates a series of nuanced, multi-turn interactions to accurately ascertain a patient's condition. Previous models typically lack this adaptive capability, resulting in oversimplified dialogs and unreliable diagnostic outputs.

Figure 1: Overview of the DocCHA framework. The left panel shows a multi-turn dialogue between the user and the diagnostic agent, while the right panel illustrates the corresponding backend modules, including symptom collection, history acquisition, and causal reasoning.

Methodology

DocCHA consists of three principal modules, each defined by specific confidence-driven mechanisms that enhance the diagnostic process:

Module 1: Symptom Collection

In this phase, symptoms are collected with a focus on both breadth and detailed specificity, relying on a dual-score system for coverage and detail, ensuring more meaningful and discriminative attributes are surfaced to differentiate between diagnoses. This structured framework allows DocCHA to ask targeted questions based on discriminative power, significantly reducing diagnostic uncertainty.

Module 2: History Acquisition

This module dynamically assesses the most relevant aspects of a patient’s medical history through a confidence-scored mechanism. The modular approach prioritizes obtaining comprehensive disease-relevant information while handling ambiguous inputs. The history sufficiency is measured across dimensions such as coverage, relevance, and certainty, which direct focused follow-up questions.

Module 3: Causal Graph Construction and Refinement

This module aims to construct an explicit causal reasoning path that aligns symptoms and historical data with potential diagnoses. It uses a scoring function to evaluate coherence and medical plausibility, leveraging medical knowledge bases such as UMLS to refine and validate the accuracy of these causal inferences. Any detected weak links prompt additional queries for better clarification (Figure 2).

Figure 2: An end-to-end diagnosis example with DocCHA. The left shows the multi-turn interaction between patient and system; the right illustrates the corresponding backend modules that drive symptom elicitation, history acquisition, and causal reasoning.

Implementation and Evaluation

Implemented using advanced LLMs like GPT-4o and LLaMA-3, DocCHA was evaluated on two Chinese consultation datasets—IMCS21 and DX. The evaluation showed significant improvements: DocCHA achieved a diagnostic accuracy of up to 5.18% higher than established LLM baselines and improved symptom recall by over 30%, with a modest increase in dialogue turns (Figure 3). These results highlight DocCHA's ability to handle complexity with nuanced and clinically grounded questioning techniques.

Figure 3: Sensitivity of DocCHA’s modules across Accuracy, Cosine Similarity, Information Recall, and Turn Count on IMCS21 and DX.

Discussion and Implications

DocCHA significantly advances the field by providing a more adaptable and transparent conversational diagnostic framework. The modular and confidence-aware design of DocCHA allows for greater interaction efficiency and diagnostic reliability compared to traditional CHAs. This design is particularly well-suited for deployment in multilingual or low-resource environments due to its interpretability and effectiveness in eliciting comprehensive medical information.

Future Directions

Future work will explore enhancing DocCHA’s capacity to include more diverse patient interaction scenarios, refining module interactions further, and integrating other sensory data inputs for a holistic diagnostic approach. Additionally, expanding the framework to support a broader range of medical conditions could potentially increase its utility in healthcare settings.

Conclusion

DocCHA introduces a substantial innovation in LLM-augmented diagnostic systems, bridging the gap between current conversational models and the complex, iterative nature of clinical diagnosis. By modularizing the diagnostic process and incorporating confidence-driven methodologies, DocCHA sets the stage for the next generation of interactive, LLM-powered healthcare agents that prioritize accuracy, transparency, and efficiency.