MedGuards: Medical LLM Error Guardrails

• MedGuards is a modular medical safety guardrail that treats clinical error detection and correction as a multi-agent in-context learning problem.
• It employs specialized agents for detection, localization, and correction using confidence-guided arbitration and XML-like structured outputs.
• The framework introduces the Keyword-Prioritized Correction Score (KPCS) to assess correction quality, ensuring clinically critical errors are addressed.

Searching arXiv for the target paper and closely related work on guardrails and medical LLM safety. MedGuards is a medical safety guardrail for LLMs that treats medical error detection and correction as a multi-agent in-context learning problem. It is designed to detect whether a clinical note contains a medical error, localize the erroneous sentence, and generate a corrected version without additional training of the underlying LLMs. The framework combines specialized agents, confidence-guided arbitration over reasoning traces, and a clinically oriented evaluation metric, the Keyword-Prioritized Correction Score (KPCS), to target the failure mode in which broadly similar text still contains clinically decisive mistakes (Ma et al., 24 Jun 2026).

1. Clinical-text safety objective

MedGuards is motivated by the claim that, in healthcare, even a small wording error can change a diagnosis, treatment, pathogen, medication, or management plan, and therefore create direct patient-safety risks. The framework is aimed at clinical text rather than generic toxicity moderation or open-ended factuality checking. Its central criticism of conventional quality-control practice is that metrics such as BLEU, ROUGE, and BERTScore can assign high similarity to a sentence whose clinically decisive entity is wrong. The paper illustrates this with the contrast between “hepatitis A” and “Schistosoma mansoni”: most of the sentence may remain unchanged while the medically relevant content is incorrect (Ma et al., 24 Jun 2026).

The task is decomposed into three conditional stages. Given a clinical note $x$, MedGuards first predicts whether an error exists, then identifies the erroneous sentence, then produces a corrected sentence: $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$ Here $d \in \{0,1\}$ is the detection decision, $s$ is the erroneous sentence or sentence set, and $\hat{s}$ is the corrected sentence. This decomposition is clinically significant because it separates coarse note-level judgment from sentence-level fault isolation and from the generative act of rewriting, which the paper treats as distinct reasoning problems rather than a single monolithic generation task (Ma et al., 24 Jun 2026).

The framework is also explicitly training-free. It is described as “plug-and-play,” uses base LLMs as-is, and does not require fine-tuning or additional parameter updates. This positions MedGuards as an overlay on existing medical or general-purpose LLMs rather than as a new base clinical model (Ma et al., 24 Jun 2026).

2. Multi-agent decomposition and arbitration

The most distinctive design element is the use of multiple specialized agents in the detection and localization stages. In error detection, two base agents independently inspect the same note and each produce a binary prediction, a reasoning trace, and a confidence score: $$(d_i, r_i^{\mathrm{Det}}, c_i^{\mathrm{Det}}) = \mathrm{Agent}_i^{\mathrm{Det}}(x), \quad i \in \{1,2\}.$$ If the two agents agree, their decision is accepted. If they disagree, a third arbitration agent is invoked and receives the original note together with both candidate predictions, rationales, and confidences: $$d = \mathrm{Agent}_3^{\mathrm{Det}}\big(x \,\Vert\, (d_1, r_1^{\mathrm{Det}}, c_1^{\mathrm{Det}}), (d_2, r_2^{\mathrm{Det}}, c_2^{\mathrm{Det}})\big).$$

Localization is organized analogously. Two localization agents each produce a candidate error sentence, a rationale, and a confidence score: $$(s_i, r_i^{\mathrm{Loc}}, c_i^{\mathrm{Loc}}) = \mathrm{Agent}_i^{\mathrm{Loc}}(x), \quad i \in \{1,2\}.$$ Agreement yields the output directly; disagreement triggers a third localization arbitrator: $$s = \mathrm{Agent}_3^{\mathrm{Loc}}\Big(x \,\Vert\, (s_1, r_1^{\mathrm{Loc}}, c_1^{\mathrm{Loc}}), (s_2, r_2^{\mathrm{Loc}}, c_2^{\mathrm{Loc}})\Big).$$

The correction stage is intentionally simpler. Rather than using multi-agent consensus for rewriting, MedGuards uses a single correction agent conditioned on the localized sentence and the full note. The paper reports that adding a second correction agent hurt generation quality, which the authors interpret as evidence that correction is a generative task with many valid outputs and is therefore less suited to consensus-style arbitration than detection and localization (Ma et al., 24 Jun 2026).

After localization, MedGuards aligns the predicted sentence back to the original document using longest-common-subsequence-style string similarity: $$s^* = \arg\max_{x_i \subset x} \operatorname{sim}(x_i, s), \qquad \operatorname{sim}(x_i, s) = \frac{2 \cdot M}{|x_i| + |s|}.$$ Here $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$0 is the number of matching characters in the longest common subsequences. The aligned sentence $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$1 is then sent to the correction agent. This soft matching is operationally important because localization agents may paraphrase or slightly alter the sentence they intend to quote (Ma et al., 24 Jun 2026).

3. Prompting, reasoning traces, and implementation details

MedGuards treats arbitration as another in-context reasoning problem rather than as a fixed numerical fusion rule. The paper is explicit that confidence is one of the structured inputs to the arbitrator, but there is no hand-coded closed-form aggregation formula over confidence scores. “Confidence-guided arbitration” therefore refers to an LLM-based adjudication process that reads predictions, rationales, and confidence values jointly (Ma et al., 24 Jun 2026).

The system standardizes intermediate outputs with XML-like tags. Agent prompts ask models to emit chain-of-thought in > ..., confidence in <confidence>...</confidence>, and final answers in <result>...</result>. These fields are parsed with regular expressions and passed downstream. Confidence is requested as a 1–100 score, although the abstract description sometimes idealizes it as lying in $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$2. This structured output format makes arbitration and post-processing feasible without task-specific fine-tuning (Ma et al., 24 Jun 2026).

The prompts are role-specialized. Detection prompt A frames the model as “a skilled medical doctor reviewing the clinical text,” while detection prompt B additionally says to “put more focus” on diagnosis-related errors. The localization prompt frames the model as a “medical report quality assurance expert” and includes the heuristic that the error sentence is “more likely to be found in a conclusion sentence.” The detection and localization arbitration prompts cast the third agent as a senior medical expert reviewing two junior doctors’ outputs. The paper emphasizes that self-consistency is implemented using multiple models with distinct prompts, rather than repeated prompting of a single prompt template (Ma et al., 24 Jun 2026).

The evaluated backbones include Gemini 2.0 Flash, Gemini 2.5 Flash Lite, GPT-4o-mini, Doubao-1.5-thinking-pro, DeepSeek-V3, and Llama-3.3-70B-Instruct. For GPT-4o-mini, the reported settings are temperature $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$3, top_p $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$4, max_tokens $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$5, and logprobs = True. DeepSeek-V3 was invoked via the Doubao API interface in the authors’ setup. The implementation therefore remains model-agnostic at the orchestration level even though concrete API settings are model-specific (Ma et al., 24 Jun 2026).

4. Benchmarks and the Keyword-Prioritized Correction Score

MedGuards is evaluated on MEDEC and MedErrBench. MEDEC contains 3,848 clinical texts, each either correct or containing a single error in one of five categories: Diagnosis, Management, Treatment, Pharmacotherapy, or Causal Organism. MedErrBench contains 2,506 clinical texts across English (1,024), Chinese (1,000), and Arabic (482), and covers ten clinician-defined error categories: diagnosis, management, treatment, pharmacotherapy, causal organism/pathogen, lab/serum value interpretation, physiology, histology, anatomy, and epidemiology (Ma et al., 24 Jun 2026).

Benchmark	Scope	Error structure
MEDEC	3,848 clinical texts	Single error in 5 categories
MedErrBench	2,506 texts in English, Chinese, Arabic	10 clinician-defined categories

Detection and localization are evaluated with accuracy. Correction is evaluated with ROUGE-1, BERTScore, BLEURT, and KPCS. True negatives—cases where a note had no error and the model correctly predicted no error—are counted as successful cases and included in final scores. The localization evaluation distinguishes “soft matching,” which aligns generated sentences back to the note, from “hard matching,” which requires exact sentence-index prediction (Ma et al., 24 Jun 2026).

KPCS is the paper’s main metric contribution. It is intended to reflect whether clinically critical keywords are preserved or corrected properly, rather than treating all tokens as equally important. Given corrected sentence $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$6 and reference $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$7, the keyword component is: $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$8 where $$d = f^{\mathrm{Det}}(x), \quad s = f^{\mathrm{Loc}}(x \mid d), \quad \hat{s} = f^{\mathrm{Corr}}(s \mid x).$$9 is the number of reference keywords found in $d \in \{0,1\}$0 and $d \in \{0,1\}$1 is the number of keywords in $d \in \{0,1\}$2. Semantic and lexical similarity is averaged as: $d \in \{0,1\}$3 The final score is: $d \in \{0,1\}$4 with $d \in \{0,1\}$5 in all reported experiments. Sensitivity analysis with $d \in \{0,1\}$6 reportedly preserved model rankings reasonably well, while $d \in \{0,1\}$7 aligned best with human evaluation trends (Ma et al., 24 Jun 2026).

5. Empirical performance and ablation results

On MEDEC, MedGuards with Doubao-1.5-thinking-pro achieved error detection accuracy 0.770, localization accuracy 0.716, ROUGE-1 0.724, BERTScore 0.731, and BLEURT 0.695. The strongest prior baseline on detection in the provided comparison, Medifact, reached 0.737 detection but much lower correction quality, with ROUGE-1 0.454, BERTScore 0.444, and BLEURT 0.439. The paper presents this as evidence that the framework improves both upstream identification and downstream correction quality rather than trading one for the other (Ma et al., 24 Jun 2026).

The wrapper-style gains across backbones are large. On MEDEC, Gemini 2.0 Flash improved from detection 0.589 / localization 0.415 / ROUGE-1 0.379 / BERTScore 0.380 / BLEURT 0.395 / KPCS 0.370 to 0.713 / 0.609 / 0.537 / 0.549 / 0.549 / 0.472 under MedGuards, with an average improvement of +35.6%. GPT-4o-mini improved from 0.539 / 0.374 / 0.300 / 0.296 / 0.335 / 0.210 to 0.695 / 0.535 / 0.571 / 0.581 / 0.574 / 0.461, a +66.2% average improvement. DeepSeek-V3 improved from 0.533 / 0.413 / 0.562 / 0.547 / 0.557 / 0.257 to 0.681 / 0.679 / 0.708 / 0.712 / 0.682 / 0.556. All reported improvements were statistically significant with $d \in \{0,1\}$8 (Ma et al., 24 Jun 2026).

The multilingual results support the paper’s claim of dataset transfer. On MedErrBench English, Gemini 2.0 Flash improved from detection 0.514 / localization 0.168 / ROUGE-1 0.281 / BERTScore 0.294 / BLEURT 0.288 to 0.755 / 0.322 / 0.651 / 0.694 / 0.613. On Chinese, GPT-4o-mini improved from 0.505 / 0.115 / 0.244 / 0.390 / 0.257 to 0.600 / 0.235 / 0.595 / 0.536 / 0.529. On Chinese with DeepSeek-V3, MedGuards reached 0.815 detection, 0.420 localization, 0.748 ROUGE-1, 0.782 BERTScore, and 0.708 BLEURT. These gains are important because MedErrBench is explicitly multilingual and not just a held-out English benchmark (Ma et al., 24 Jun 2026).

The ablation studies clarify which components contribute. Replacing ICL-based arbitration with simple voting reduced performance: for Gemini 2.0 Flash, voting alone achieved detection 0.693 and localization 0.557, while voting plus ICL arbitration reached 0.713 and 0.609. Soft matching also clearly outperformed hard matching: for Gemini 2.0 Flash, soft localization accuracy was 0.61 versus 0.36 under hard matching; for GPT-4o-mini, 0.54 versus 0.29. The self-consistency analysis found that arbitration was needed much more often for localization than for detection, which supports the view that locating the error sentence is the harder subproblem (Ma et al., 24 Jun 2026).

The multi-agent structure itself also matters. Using Gemini 2.0 Flash as base, a single-agent setup achieved an average score of 0.432. A two-agent setup improved this by 17.6%; a four-agent setup by 28.9%; and the default structure of three detection agents, three localization agents, and one correction agent produced the best average score, 0.591, a 36.8% improvement. Adding a second correction agent reduced average improvement to 12.5%, reinforcing the paper’s design choice to use consensus chiefly for classification-style subproblems (Ma et al., 24 Jun 2026).

Human evaluation by two clinical doctors on 80 sampled corrections from MEDEC also supported the automated metrics. Gemini 2.0 Flash improved from a normalized mean opinion score of 0.425 to 0.483 with MedGuards, and GPT-4o-mini improved from 0.383 to 0.475. Clinician agreement was 57.5%, which the authors interpret as reflecting genuine variability in medical judgment rather than purely annotation noise (Ma et al., 24 Jun 2026).

6. Limitations, deployment status, and relation to adjacent guardrail work

MedGuards is not presented as a clinically validated autonomous correction system. The appendix states that the current work is methodology-oriented and mainly a proof of concept, and the authors explicitly frame the system as supporting human-in-the-loop workflows in which clinicians review and validate outputs before use. The framework is therefore best understood as a guardrail layer for safer drafting, auditing, or correction assistance rather than as an independent clinical authority (Ma et al., 24 Jun 2026).

Several limitations are explicit. KPCS currently relies on exact string matching for keywords and can therefore miss synonyms or clinically equivalent expressions. The framework also incurs extra latency and token cost because it makes multiple API calls across detection, localization, and correction. On MEDEC with Gemini 2.0 Flash, baseline inference took 1.61 seconds with 1,927 input tokens and 48 output tokens, whereas MedGuards took 6.99 seconds and much higher token usage across stages. In addition, raw confidence scores were not strongly calibrated: correct and wrong agreement cases had almost identical average confidence in the paper’s analysis, suggesting that arbitration benefits more from joint access to rationales and disagreements than from confidence alone (Ma et al., 24 Jun 2026).

The broader guardrail landscape helps locate MedGuards more precisely. ExpGuard is a domain-specialized external moderation model that classifies prompt and response harmfulness across financial, medical, and legal domains using a 7B LLM backbone; it is aimed at content moderation rather than error localization and correction in clinical text (Choi et al., 3 Mar 2026). MedVIGIL, by contrast, evaluates trustworthy medical VLMs under broken visual evidence using a 300-case suite, silent-failure rate, and the medvigil Composite Score; its focus is evidence failure and refusal behavior in medical vision-language systems rather than textual correction workflows (Jiang et al., 8 May 2026). Relative to these systems, MedGuards occupies a narrower but distinct niche: post hoc medical text safety, specifically the detect–localize–correct pipeline for clinically meaningful errors.

A common misconception is that “confidence-guided arbitration” implies a formal confidence-weighted ensemble. The paper does not implement such a closed-form fusion rule. Another is that “training-free” means low-cost or calibration-free deployment; the reported latency and token costs show otherwise. A more substantive limitation is scope: the system improves medical error detection and correction in benchmarked clinical text, but it does not verify clinical truth in the general sense, does not replace expert review, and does not by itself solve the broader problems of content moderation, multimodal evidence failure, or institution-specific policy compliance. Those would require additional guardrail layers beyond the architecture MedGuards provides (Ma et al., 24 Jun 2026).