MedGuards: Multi-Agent System for Reliable Medical Error Detection and Correction

Abstract: As LLMs are increasingly deployed in healthcare settings, accurate error detection and correction in generated or existing text becomes critical, as even minor mistakes can pose risks to patient safety. Existing methods for error detection and correction, including automated checks and heuristic-based approaches, do not generalize well across unseen datasets. In this paper, we propose MedGuards as a medical safety guardrail, which is a new framework that treats medical error detection and correction as a multi-agent in-context learning task. Specialized agents separately detect, localize, and correct errors, while a confidence-guided arbitration mechanism resolves disagreements using reasoning traces and confidence scores. This design enhances interpretability, robustness, and adaptability, without requiring additional training of the base LLMs. Additionally, we introduce the Keyword-Prioritized Correction Score (KPCS), a new evaluation metric that considers whether critical keywords within the reference text are generated correctly, providing a more comprehensive assessment than conventional metrics. Experiments across four multilingual medical datasets consisting of clinical notes demonstrate significant improvements by the proposed framework across several metrics and models. Our aim is to enable safer deployment of LLMs in real-world healthcare applications. For reproducibility, we make our code publicly available at https://github.com/congboma/MedErrBench.

• The paper presents a novel multi-agent framework that decomposes medical error detection and correction into sequential stages using chain-of-thought reasoning.
• It employs specialized agents and in-context arbitration to enhance detection accuracy and interpretability, achieving significant performance gains on benchmark datasets.
• A new evaluation metric, KPCS, is introduced to prioritize critical clinical entities, aligning automated assessments with expert human opinion.

MedGuards: Multi-Agent In-Context Learning for Medical Error Detection and Correction

Motivation and Background

The reliable detection and correction of medical errors in clinical notes is an unresolved challenge in the deployment of LLMs for healthcare, with significant ramifications for patient safety. Traditional approaches based on heuristic checks or single-agent LLM evaluations do not generalize well to unseen datasets and often fail to capture errors related to domain-specific clinical entities such as diagnoses, pharmacotherapy, or causal organisms. The MIQA-CORR shared tasks and recently released datasets (e.g., MEDEC, MedErrBench) highlight both the critical gap in automated error control mechanisms and the deficiencies of existing evaluation metrics that treat all tokens as equally important. MedGuards addresses these challenges by leveraging a principled multi-agent framework that exploits the synergy between Chain-of-Thought (CoT) decomposition, self-consistency, and in-context arbitration to deliver robust and interpretable semantic error management.

MedGuards System Architecture

MedGuards operationalizes medical error detection and correction as a collaborative, multi-agent in-context learning task. The workflow is partitioned into three sequential stages:

• Error Detection: Multiple specialized agents independently analyze the input clinical note, outputting binary predictions (error/no error) with explicit reasoning traces and confidence scores. Disagreements trigger an arbitration agent conditioned on all prior outputs, rationales, and confidences.
• Error Localization: Agents identify erroneous sentences, leveraging character-level alignment via SequenceMatcher to robustly map generated error spans to the source text. Arbitration is again employed for cases of disagreement, resulting in a consensus on the specific error location.
• Error Correction: A dedicated correction agent generates clinically meaningful corrections for the erroneous sentences, referencing the original context to maintain semantic fidelity.

This modular decomposition not only mirrors authentic clinical reasoning but also facilitates traceability and interpretability, as each sub-task is accompanied by chain-of-thought rationales and explicit confidence quantification.

Self-Consistency and Arbitration Dynamics

MedGuards extends classical self-consistency sampling into a multi-agent context. Multiple independent models (or variants of prompts) are invoked for each stage, and consensus is reached via comparison. Where outputs diverge, arbitration is triggered, wherein an adjudicator agent integrates the outputs, reasoning traces, and confidences as prompt context, resulting in higher-order deliberation. This meta-reasoning, realized via ICL, does not require retraining the base models, enabling plug-and-play integration across any LLM infrastructure. Statistical analyses reveal that consensus dominates the detection stage (arbitration rates as low as 11%-20%), whereas localization is inherently ambiguous and benefits disproportionately from arbitration (31.9%-37.8%).

Domain-Critical Evaluation: KPCS Metric

Conventional metrics (BLEU, ROUGE, BERTScore) are insufficient for clinical safety as they fail to prioritize domain-specific entities. MedGuards introduces Keyword-Prioritized Correction Score (KPCS), which explicitly penalizes corrections lacking critical medical keywords while balancing semantic and fluency alignment through a tunable weighting parameter. This metric directly captures the clinical reliability of corrections, as evidenced by alignment with human Mean Opinion Scores. Sensitivity analyses confirm robustness of the metric across a range of $\alpha$ values, with $\alpha = 0.5$ yielding best concordance with expert assessments.

Empirical Results and Ablation Analyses

MedGuards demonstrates substantial improvements over state-of-the-art single-agent and hybrid models across multiple datasets and languages. On MEDEC, MedGuards with Doubao-1.5-thinking-pro as the base LLM achieves detection accuracy of 0.770, localization accuracy of 0.716, and KPCS of 0.554, outperforming the next-best baseline by statistically significant margins (all $p \ll 0.05$). Across English, Arabic, and Chinese subsets of MedErrBench, MedGuards consistently outperforms plain and voting-based LLM variants by 20%-96% on detection and correction metrics. Human evaluation attests to the clinical meaningfulness of the corrections, with MedGuards improving the normalized MOS score compared to baseline LLMs.

Ablations further underscore the impact of structured collaboration and arbitration: ICL-based debating strategies for consensus resolution yield substantial gains over naive voting, especially for weaker LLM backbones. Multi-agent configurations also demonstrate that increasing the number of detection and localization agents monotonically improves performance up to 36.8%, while excessive redundancy in correction agents degrades generation quality due to the inherent variability in generation outputs.

Practical Implications, Limitations, and Future Directions

MedGuards delivers a model-agnostic, plug-and-play framework for enhancing safety in medical NLP applications. Its architecture is amenable to integration into clinical documentation pipelines, enabling human-in-the-loop auditing, quality assurance, and error correction. The rigorous prioritization of clinical entities through KPCS provides a mechanism for aligning automated evaluations with the demands of medical practitioners.

Practically, the computational overhead from multi-agent stages is offset by substantial improvements in reliability and clinical safety, particularly in time-sensitive environments. Parallelization and caching strategies can further mitigate latency concerns.

Current limitations include reliance on exact string matching for keyword detection in KPCS (which may miss clinically equivalent synonyms), scarcity of high-quality annotated clinical datasets, and the need for further validation in real-world hospital workflows. MedGuards is designed to facilitate downstream human verification and integration with domain knowledge bases, and future work will focus on clinical deployment, integration with medical ontologies, schema-constrained generation, and large-scale real-world validation.

Theoretically, the MedGuards paradigm represents a generically applicable approach to leveraging multi-agent reasoning as a meta-level control mechanism for generative LLMs, advancing the frontier of interpretability, reliability, and robustness in medical AI.

Conclusion

MedGuards introduces a principled multi-agent framework for medical error detection and correction, yielding improved robustness, interpretability, and clinical fidelity across diverse languages and datasets. Coupled with the domain-critical KPCS metric, MedGuards sets a new benchmark for safety-oriented deployment and evaluation of medical LLMs, providing an actionable foundation for real-world integration and future research in agentic medical AI (2606.25651).