Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

Abstract: Retrieval-augmented generation (RAG) is a widely adopted paradigm for enhancing LLMs in medical applications by incorporating expert multimodal knowledge during generation. However, the underlying retrieval databases may naturally contain, or be intentionally injected with, adversarial knowledge, which can perturb model outputs and undermine system reliability. To investigate this risk, prior studies have explored knowledge poisoning attacks in medical RAG systems. Nevertheless, most of them rely on the strong assumption that adversaries possess prior knowledge of user queries, which is unrealistic in deployments and substantially limits their practical applicability. In this paper, we propose M\textsuperscript{3}Att, a knowledge-poisoning framework designed for medical multimodal RAG systems, assuming only limited distribution knowledge of the underlying database. Our core idea is to inject covert misinformation into textual data while using paired visual data as a query-agnostic trigger to promote retrieval. We first propose a unified framework that introduces imperceptible perturbations to visual inputs to manipulate retrieval probabilities. Besides, due to the prior medical knowledge in LLMs, naively poisoned medical content with explicit factual errors can be corrected during generation. Thus, we leverage the inherent ambiguity of medical diagnosis and design a covert misinformation injection strategy that degrades diagnostic accuracy while evading model self-correction. Experiments on five LLMs and datasets demonstrate that M\textsuperscript{3}Att consistently produces clinically plausible yet incorrect generations. Codes: https://github.com/ypr17/M3Att.

• The paper introduces a dual-stage protocol using PGD-crafted image perturbations and subtle textual misinformation to hijack retrieval pipelines.
• Experimental results show top-1 attack success rates up to 32.56% and over 75% in top-5 under <1% poisoning, impacting diagnostic accuracy.
• Findings expose critical vulnerabilities in medical RAG systems, emphasizing the need for robust knowledge base curation and defense measures.

Knowledge Poisoning Attacks in Medical Multimodal Retrieval-Augmented Generation

Introduction and Background

This paper addresses the security and reliability issues in medical multimodal retrieval-augmented generation (RAG) systems, which combine large vision-LLMs (LVLMs) with retrieval pipelines over paired image-text medical knowledge bases. While RAG enhances the evidential grounding of LLM outputs, it also opens new vulnerabilities: the injection of adversarial medical knowledge—in either visual or textual form—into the external knowledge base can directly perturb downstream diagnostic outputs. Distinct from prior work, this study focuses on a realistic threat model for knowledge poisoning: adversaries are assumed to have limited knowledge about the underlying data distribution, no access to user queries, and no control over model parameters.

Threat Model and Technical Challenges

The medical multimodal RAG setting presents unique technical barriers for poisoning attacks. Medical image embeddings are highly clustered due to anatomical homogeneity, so naively injected poisoned samples are likely to be overwhelmed by benign data in retrieval unless injected at high, easily-detectable rates. Moreover, LLMs used in medical settings encode strong domain priors—overly explicit factual errors in retrieved samples are likely to be corrected automatically by the generator, mitigating the effect of the attack and exposing the malicious sample. Therefore, effective attacks must balance three objectives: they must:

1. Stealthily hijack retrieval pipelines under distributional uncertainty;
2. Manipulate the generator via misinformation that is not self-corrected;
3. Maintain high clinical plausibility to avoid detection by automated or manual corpus defenses.

The M3.Att Framework

M3.Att addresses these challenges via a dual-stage poisoning protocol:

Distribution-Guided Retrieval Hijacking

Poisoned images are crafted via a cluster-centric approach. The adversary extracts a reference subset of the knowledge base—accessible via black-box oracles—and performs clustering in embedding space to identify proxy targets representative of typical query regions. Projected gradient descent (PGD) perturbations are then applied to selected candidate images to maximize their retrieval probability with respect to the cluster centers, subject to strict $l_\infty$ constraints that ensure visual imperceptibility. Both white-box (gradient access) and black-box (finite-difference) optimization settings are explored, demonstrating the practicality of the method under realistic deployment constraints.

Clinical Ambiguity-Guided Misinformation Injection

Textual poisoning leverages inherent ambiguities in medical language rather than overt factual errors. Three strategies are used in combination:

• Fine-grained Severity Migration: Alters the urgency of diagnoses without changing disease category, via up- or down-scaling of severity (e.g., from "severe" to "mild").
• Prior-Constrained Diagnosis Distortion: Reframes the diagnosis to visually similar but distinct conditions matched for prior probability, evading LLM self-correction.
• Risk Association Corruption: Modifies actionable statements and recommendations to suppress or exaggerate clinical urgency.

These strategies exploit low-confidence regions of the LLM’s prior, where clinical uncertainty is intrinsic and self-correction is less likely.

Experimental Evaluation

Experiments are conducted on five representative medical multimodal datasets (IU-XRay, MIMIC-CXR, CRC100k, MHIST, PCam) and across five LVLMs, including both general-purpose (GPT-4o, GPT-5, Gemini-2.5, Claude-4.5) and specialized medical models (LLaVA-Med). Three major VL retrievers (CLIP, BGE-VL, SigLIP) are tested.

Numerical Results

• Retrieval Attack Success: For moderate poisoning budgets ($<1\%$ poisoned samples), M3.Att achieves top-1 retrieval attack success rates (ASR) up to $32.56\%$, and top-5 ASR exceeding $75\%$ in many cases. Retrieval hijacking is robust to access assumptions: black-box optimization performs comparably to white-box, showing practical deployability.
• Downstream Utility Degradation: Across all LVLM architectures and medical tasks (VQA, report generation, image classification), M3.Att reduces clinical utility by an average of 8.78% compared to clean-RAG baselines at the same retrieval budgets. In filtered scenarios—where a hijacked poison is retrieved—the model’s accuracy and factual consistency can drop to near-random levels.
• Stealthiness and Defense Evasion: Expert annotation experiments demonstrate that at realistic injection ratios (1%), the F1 for human detection of poisoned texts is as low as 0.0870. Automated corpus defenses based on clustering and image-text consistency fail to detect poisoned entries, reflected by unchanged retrieval ASR metrics post-filtering.

Ablations and Hyperparameter Sensitivity

Disabling either the visual retrieval hijacking or the ambiguity-guided text poisoning component significantly restores LVLM utility, confirming the necessity of both components. Poisoning effect is sensitive to the number of semantic clusters and perturbation budgets, but efficacy saturates at moderate values—excessive perturbations are unnecessary.

Implications and Future Directions

Practical Implications: The findings expose a critical vulnerability in medical RAG systems: even with a small amount of clinically plausible poisoned data, and under minimal adversarial knowledge, downstream outputs can be reliably corrupted without triggering typical model defenses. These results generalize across retriever architectures and LVLMs, highlighting the importance of both robust knowledge base curation and secure RAG pipeline design.

Theoretical Implications: The work suggests that the interplay between semantic clustering in visual embedding spaces and the low-confidence regions of LLM priors is a fundamental attack surface in multimodal retrieval-generation. Standard alignment and safety tuning in LVLMs does not suffice against attacks that probe regions of ontological uncertainty rather than make flagrant errors.

Future Developments: Future research directions should investigate:

• Extending attacks and defenses to high-dimensional (3D, video) modalities in medical imaging.
• Advanced knowledge base validation methods capable of detecting subtle but coordinated visual-textual anomalies.
• Fine-grained provenance tracking in RAG pipelines and further adversarial training strategies leveraging medical uncertainty models.

Conclusion

M3.Att establishes a new, query-agnostic standard for knowledge poisoning in medical multimodal RAG. It demonstrates that security assumptions underlying many current RAG deployments are insufficient, as attacks can succeed without query information or gradient access, and with tight stealth constraints. The study motivates a fundamental reevaluation of knowledge base construction, retrieval reliability, and model alignment methods in high-stakes AI applications in healthcare.