MACD-Human Diagnostic Workflow
- MACD-Human is a human-in-the-loop extension of the MACD framework that uses iterative LLM-based consultations with physician escalation for unresolved cases.
- The workflow features a three-tier process—independent agent diagnosis, evaluator-guided consensus, and physician adjudication—to mirror clinical consultation practices.
- The self-learned knowledge pipeline refines disease-specific diagnostic insights, significantly boosting model accuracy and case efficiency compared to traditional methods.
MACD-human collaborative workflow denotes the human-in-the-loop extension of the Multi-Agent Clinical Diagnosis (MACD) framework, a diagnostic consultation architecture in which multiple LLM-based diagnostician agents engage in iterative consultations, an evaluator agent determines whether consensus has been reached, and physicians intervene only when the agents fail to converge (Li et al., 24 Sep 2025). Within this design, diagnosis is not treated as an isolated inference problem. Instead, the workflow couples reusable disease-specific “Self-Learned Knowledge” with a staged consultation-and-escalation process intended to mirror how physicians accumulate expertise through experience and consult one another in difficult cases. The resulting system is positioned as a collaborative workflow rather than a replacement model: agents perform most case-level reasoning, but human physicians remain the final authority in unresolved disagreements.
1. Conceptual basis and problem setting
The MACD-human workflow was proposed in response to a limitation of both conventional prompting and many multi-agent designs: they typically optimize isolated inferences and neglect the accumulation of reusable clinical experience (Li et al., 24 Sep 2025). In the MACD view, diagnosing each case from scratch is suboptimal because clinical expertise is cumulative. The framework therefore aims to let LLMs “learn from experience” by extracting reusable disease-specific knowledge from prior cases, refining that knowledge, and reusing it in later diagnoses.
This design is also motivated by a specific organizational analogy. The workflow is intended to mimic how real clinicians develop expertise through experience, while also using consultation to resolve difficult cases. In MACD-human, that analogy is operationalized as a division of labor between autonomous diagnostician agents, an arbitration layer, and physician oversight. Agents handle routine and moderately difficult cases through iterative consultation; physicians adjudicate only the unresolved subset. The paper explicitly characterizes this as a collaborative consultation workflow in which human oversight is reserved for cases where agreement is not reached, rather than being applied uniformly to every case (Li et al., 24 Sep 2025).
A central implication is that MACD-human shifts the role of the human physician from primary first-pass diagnostician to final authority in difficult disagreements. This does not remove physician judgment. Rather, it repositions that judgment at escalation points where disagreement persists after structured machine consultation. In that sense, the workflow combines cumulative machine memory with selective human adjudication.
2. Core architecture and the Self-Learned Knowledge pipeline
At the framework level, MACD comprises three core components: a knowledge summarizer agent, a knowledge refiner agent, and a diagnostician agent (Li et al., 24 Sep 2025). The summarizer reviews a set of learning cases for a disease and extracts salient diagnostic concepts. The refiner then performs redundancy filtering to remove near-duplicate concepts and importance filtering to remove concepts that are harmful or misleading for diagnosis. The resulting output is the disease-specific Self-Learned Knowledge, which is then inserted into the diagnostician’s prompt as prior knowledge.
The paper emphasizes that this is not parameter training. The knowledge is explicitly written into prompts and can be updated as new cases are learned. Learning cases are drawn from the larger dataset and are defined as cases that at least one diagnostician model initially got correct. Each model team receives its own exclusive subset of these learning cases, so each model learns its own experience rather than sharing a universal memory. The summarizer organizes disease concepts into “general” and “rare” criteria, and the refiner deduplicates them using BioBERT semantic embeddings and a greedy maximal marginal relevance strategy.
The importance-filtering stage is more than cosmetic pruning. It uses concept-based causal intervention: each concept is ablated and the change in diagnostic accuracy is measured. The paper denotes semantic similarity between concept embeddings as , and the intervention effect as , defined by the accuracy change when a concept is removed versus retained (Li et al., 24 Sep 2025). If removing a concept hurts performance, the concept is treated as positive; if removing it improves or does not change performance, the concept is treated as negative and may be removed. This yields a curated prompt-level knowledge memory intended to resemble accumulated physician experience.
In the MACD-human extension, three diagnostician agents are instantiated, each based on a different base model and each using its own self-learned knowledge. An evaluator agent is added as the arbitration layer. The evaluator does not diagnose by itself; instead, it governs the multi-round consultation and checks whether the diagnostician agents are sufficiently aligned to stop discussion or whether another round is needed. Human physicians enter the workflow only when this machine-level convergence process fails.
3. Consultation loop, consensus formation, and escalation to physicians
The consultation/refinement process in MACD-human is explicitly iterative (Li et al., 24 Sep 2025). Each diagnostician first makes an independent diagnosis. These outputs are then collected and fed back into the next round as “past diagnosis results.” The prompt instructs each agent not to merely echo prior opinions, but to conduct an impartial re-evaluation based on the case and the prior outputs. This detail is important because the system is designed as a consultation mechanism, not a voting scheme.
Consensus is evaluated after each discussion round. It is not based on exact string matching alone. Diagnoses are first normalized using tolerant disease-name matching rules and then semantically compared using BioBERT cosine similarity. Consensus is confirmed only if pairwise similarity exceeds a threshold. If consensus is reached, the agreed diagnosis is returned. If consensus is not reached after a predefined maximum number of discussion rounds, the final round’s outputs become the “effective opinions” for the physician, and the physician makes the final diagnosis.
The human consultation prompt is itself structured. It instructs the agent to verify prior opinions, treat them only as references, and produce one final diagnosis plus diagnostic criteria. This means that physician escalation is not an abrupt handoff from automation to manual diagnosis. Rather, it is a transition from agent-only consultation to physician-supported adjudication with machine-generated evidence already organized.
Operationally, the workflow thereby has three decision layers: independent agent diagnosis, evaluator-governed multi-round consultation, and physician adjudication for unresolved cases. This layered structure is the defining procedural feature of MACD-human.
4. Human oversight, authority, and relation to broader collaborative workflow research
MACD-human belongs to a broader family of human-agent workflow designs in which human participation is concentrated at semantically meaningful intervention points rather than dispersed across continuous supervision. In “Human Tool,” a human contributor is represented through structured schemas of Capabilities, Information, and Authority, and the agent invokes human input under conditions of capability complementarity, information exchange, and authority control (Tang et al., 13 Feb 2026). This suggests one useful interpretation of MACD-human: the physician is invoked not because the system requires constant supervision, but because unresolved diagnostic disagreement activates a human authority boundary.
A related pattern appears in asynchronous high-performance workflows. The CIF framework treats human review as a declared workflow task and summarizes execution as
so that human judgment can redirect the workflow without blocking the entire compute pipeline (Mendoza et al., 5 May 2026). A plausible implication is that MACD-human’s escalation logic can be understood as a domain-specific instantiation of the same principle: human oversight is built into the workflow as a first-class checkpoint, rather than as an external interruption.
These related frameworks differ in domain and implementation, but they converge on a common organizational idea. The agent-led system handles orchestration, decomposition, or high-throughput execution; human authority is preserved at points involving approval, unresolved ambiguity, or context-sensitive judgment. In MACD-human, this general workflow principle takes a clinical form: physicians are the final authority in difficult disagreements, while the agents function as a pre-consultation system that can often settle cases before human intervention (Li et al., 24 Sep 2025).
5. Dataset, evaluation protocol, and empirical results
The MACD evaluation is based on MIMIC-MACD, a dataset built from MIMIC-IV v2.2 and related clinical sources, containing 4,390 real-world cases spanning seven diseases: appendicitis, cholecystitis, diverticulitis, pancreatitis, pericarditis, pneumonia, and pulmonary embolism (Li et al., 24 Sep 2025). Each case includes four modalities of text: history of present illness, physical examination, laboratory results, and radiology reports. The framework was evaluated with diverse open-source LLMs, including Llama-3.1 8B/70B and DeepSeek-R1-Distill-Llama 70B.
For physician comparison, the paper created a MIMIC-MACD-human subset by randomly sampling 20 cases per disease from the test set. Seven attending emergency medicine physicians participated; three handled chest diseases and four handled abdominal diseases. The main quantitative outcomes were primary diagnostic accuracy, consensus rate in MACD-human, effective opinion rate, and diagnosis time.
The reported results indicate improvements both from self-learned knowledge and from the human-collaborative consultation workflow. Across the seven diseases, MACD improved diagnostic accuracy over baseline guideline knowledge by 10.3% for Llama-8B, 8.7% for DeepSeek-70B, and 15.8% for Llama-70B, averaging an 11.6% gain. Compared with zero-knowledge prompting, average accuracy rose to 70.9% for Llama-8B, 71.4% for DeepSeek-70B, and 84.4% for Llama-70B. Against the Mayo Clinic guideline, Llama-70B improved by 22.3%; against the Professional Guideline, the gain was also 22.3% in the reported comparison table for that model (Li et al., 24 Sep 2025).
On the physician benchmark subset, physicians averaged 65.0% accuracy, while the Llama-70B diagnostician reached 81.0%, a statistically significant increase. The MACD-human workflow improved further to 83.6%, corresponding to an 18.6% improvement over physicians-only diagnosis. Efficiency also differed sharply: the Llama-70B agent took about 13.29 seconds per case versus 209.71 seconds for physicians.
| Comparison | Result |
|---|---|
| MACD vs guideline knowledge | 10.3%, 8.7%, and 15.8% gains; 11.6% average |
| Zero-knowledge to MACD accuracy | 70.9%, 71.4%, 84.4% for Llama-8B, DeepSeek-70B, Llama-70B |
| Llama-70B vs Mayo Clinic guideline | 22.3% improvement |
| Physicians-only diagnosis | 65.0% accuracy |
| Llama-70B diagnostician | 81.0% accuracy |
| MACD-human workflow | 83.6% accuracy; 18.6% improvement over physicians-only |
| Diagnosis time | 13.29 seconds per case for Llama-70B vs 209.71 seconds for physicians |
These outcomes support the paper’s main claim that the collaborative workflow is synergistic rather than merely additive. Self-learned knowledge improves the diagnostician agents, and selective physician adjudication improves the unresolved cases further.
6. Interpretability, stability, and limitations
The MACD-human workflow is designed to be at least partially interpretable. The diagnostician is required to output both a diagnosis and diagnostic criteria, and the knowledge refiner traces how individual concepts affect performance through causal intervention (Li et al., 24 Sep 2025). The paper presents direct evidence that refinement removes harmful concepts rather than merely shortening prompts. For diverticulitis, removing three negatively influential concepts improved Llama-70B accuracy by 3.5%. Human physicians also rated the self-learned knowledge as clinically relevant, with average concept relevance scores generally around 3.3 to 4.2 on a 5-point scale.
The paper also argues that the learned knowledge shows strong stability, transferability, and personalization. Stability means that stronger base models still outperform weaker ones when using the same knowledge. Transferability means that self-learned knowledge improves multiple models, not only the model that produced it. Personalization, however, is especially pronounced: each model performs best on the self-learned knowledge produced by its own agent team. When knowledge is transferred between model teams, performance drops, especially when knowledge from a stronger model is applied to a weaker one. The authors interpret this as evidence that each LLM develops a model-specific cognitive style and a distinctive linguistic representation of disease (Li et al., 24 Sep 2025).
Several limitations are stated explicitly. The workflow is still manually structured rather than fully autonomous; the data are mostly text reports rather than raw images; the dataset is English-language and U.S.-centric; and the observed performance gains do not imply immediate readiness for direct clinical deployment. A more subtle limitation is model mismatch: external knowledge, even if clinically authoritative, may be less useful than self-generated knowledge because different LLMs interpret and encode medical cues differently. The collaborative consensus mechanism also depends on semantic alignment thresholds and thus may not perfectly capture true clinical disagreement.
These constraints help delimit common misconceptions. MACD-human is not presented as an autonomous clinical replacement system, nor as evidence that human review is obsolete. Its more precise claim is narrower: a structured workflow that turns episodic case solving into cumulative expertise, then escalates only unresolved disagreements to physicians, can outperform physician-only diagnosis under the reported evaluation protocol while preserving human final authority (Li et al., 24 Sep 2025).
