Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification

Abstract: Vision-LLMs (VLMs) reliability in medical diagnosis is challenged by trust-undermining hallucinations. Existing hallucination detection approaches mainly focus on identifying factual inconsistencies between generated text and reference data. While some studies analyze where models attend in images, they seldom verify whether such attention truly reflects the visual evidence supporting the generated text. To address this gap, we propose Co}unter-Evidence Verification (CoEV), a training-free plug-and-play framework that detects and corrects hallucinations through evidence-based factual consistency verification. CoEV performs bidirectional verification between textual assertions and visual evidence, testing whether each statement is supported by its corresponding evidence region, and assigns each statement into a four-quadrant diagnostic map capturing combinations of text factuality and visual grounding. CoEV detects hallucinated content and serves as a post hoc refinement tool, correcting hallucinations without retraining. Extensive experiments on four medical datasets show that CoEV combats hallucinations in VLMs.For hallucination detection, CoEV consistently outperforms existing methods, improving average PR-AUC and ROC-AUC by 3.0% and 3.9% absolute points respectively, with notable gains of up to 18.5% in specific VQA scenarios. For hallucination correction, it improves Micro-F1 by up to 12.5%, reduces hallucination rates by over 11.9% on medical report generation, and also boosts medical VQA accuracy. These results show that CoEV enables reliable detection and correction of hallucinations, providing clinicians with dependable, evidence-based cues for diagnosis. Code will be released upon acceptance.

• The paper introduces CoEV, a training-free, post-hoc method for detecting and correcting hallucinations by verifying clinical claims against visual evidence.
• It employs a four-quadrant diagnostic mapping that categorizes outputs based on factual correctness and evidence grounding using a bidirectional verification process.
• Experimental results show improvements in PR-AUC, ROC-AUC, and VQA accuracy, ensuring enhanced diagnostic reliability and clinical trust.

Counter-Evidence Verification for Hallucination Management in Medical Vision-LLMs

Background and Motivation

Medical Vision-LLMs (VLMs) have been integrated into clinical workflows for tasks such as Medical Visual Question Answering (Med-VQA), Radiology Report Generation (MRG), and phrase grounding, leveraging multimodal representations to facilitate diagnostic support. Despite their utility, VLMs are susceptible to hallucinations—outputs not substantiated by visual evidence—which can result in inaccurate clinical assessments, jeopardize patient safety, and erode clinician trust. Existing hallucination detection strategies predominantly rely on uncertainty estimation, cross-model consistency, or linguistic evaluation metrics (e.g., BLEU, ROUGE), but these approaches inadequately address the nuanced evidentiary demands in medical contexts. Furthermore, attention visualization and saliency mapping provide only superficial interpretability, lacking the causal verification required to differentiate evidence-grounded from prior-driven generation.

Methodological Framework

Counter-Evidence Verification (CoEV)

CoEV introduces a training-free, plug-and-play post-hoc framework that diagnoses and corrects hallucinations in medical VLMs without necessitating model retraining or external architectures. The core innovation is a bidirectional verification process that directly tests the factual and visual grounding of individual clinical claims. For each textual assertion in the generated content, CoEV localizes supporting visual regions using the Diagnosis-Grounded Module (DGM). CoEV then applies a counterfactual intervention by masking the evidence region and re-evaluating the assertion via the VLM. If the claim persists after evidence removal, it is attributed to linguistic priors rather than visual grounding, enabling rigorous hallucinatory profiling.

Four-Quadrant Diagnostic Mapping

CoEV formalizes claim classification into a four-quadrant diagnostic map based on two axes—factual correctness ($f_i$) and visual grounding ($z_i$):

• Q1: Consistent & grounded (correct, evidence-based)
• Q2: Inconsistent & grounded (incorrect but evidence-based)
• Q3: Inconsistent & ungrounded (incorrect, prior-driven)
• Q4: Consistent & ungrounded (correct but prior-driven)

This taxonomy stratifies hallucination types, allowing isolation of purely linguistic bias, visual misinterpretation, and total failure. Claims outside Q1 are flagged as hallucinated.

Correction Mechanisms

CoEV integrates post-hoc refinement across Med-VQA and MRG tasks:

• Med-VQA: Answers are verified for both factual and visual consistency. Hallucinated responses are revised based on CoEV’s quadrantic diagnosis.
• MRG: Each sentence in a radiology report is subjected to claim-evidence verification. Hallucinated sentences are autonomously rewritten with grounded, evidence-aligned alternatives using LLM prompting.

Experimental Evaluation

Datasets and Tasks

CoEV was evaluated on VQA-RAD and MIMIC-VQA for Med-VQA, and MIMIC-CXR and IU-Xray for MRG, using ground-truth labels from the GREEN model. Diverse VLMs—including InternVL3-2B, Qwen2-3B, Lingshu-7B, and LLaVA-Med—were benchmarked to assess architectural robustness.

Metrics

Performance for hallucination detection was measured using PR-AUC and ROC-AUC; report-level correction was assessed by CheXpert-style F1 scores (Mi-F1-5, Mi-F1-14), CHAIR-I/S (sentence-level error rates), MediHall (clinically weighted penalties), and VQA accuracy.

Results

CoEV achieved consistent, architecture-agnostic improvements over baselines:

• Hallucination Detection: Average PR-AUC and ROC-AUC increased by 3.0% and 3.9%, respectively, with gains up to 18.5% in VQA-RAD scenarios.
• Correction: Micro-F1 improved by up to 12.5%, hallucination rates decreased by more than 11.9%, and VQA accuracy rose by 13.04% on average, with a +27.16% gain for Qwen2-3B on MIMIC-VQA.
• Ablation Study: Combining textual and visual axes maximized F1-Score (0.6112), confirming their complementary contribution to discriminative hallucination characterization.

CoEV’s causal intervention isolates model errors and ensures that correction is localized, preserving valid findings and mitigating unsupported clinical claims.

Implications and Future Directions

Practical Impact

CoEV enables reliable, evidence-based trust calibration in safety-critical domains without retraining, supporting clinical adoption of VLMs by providing granular, actionable diagnostics and automated correction. The plug-and-play architecture facilitates seamless integration into existing infrastructures, empowering clinicians with interpretable model oversight.

Theoretical Advancements

The four-quadrant diagnostic mapping enriches hallucination taxonomy, offering a structured paradigm to analyze claim-evidence relationships. The framework advances the definition of grounding by exposing causal dependencies between multimodal inputs and generated text, contrasting with correlation-based interpretability.

Forward-Looking Perspectives

Application of CoEV to real-time clinical decision support and broader multimodal inconsistencies is poised to further enhance reliability in medical AI. Future iterations may extend to longitudinal patient data, multi-modal imaging, and adaptive retraining scenarios, providing holistic verification and correction in complex clinical pipelines.

Conclusion

CoEV establishes a robust, causally interpretable solution for post-hoc hallucination detection and correction in medical VLMs, consistently improving diagnostic accuracy across open-ended and closed-ended tasks. The framework’s architecture-agnostic, evidence-driven methodology advances model trustworthiness and practical deployment in clinical settings. Future expansion into broader domains and real-time applications will extend CoEV’s impact as a foundation for reliable multimodal AI (2606.18609).