- The paper identifies deceptive grounding as a failure mode where accurate evidence is misattributed to the wrong clinical entities, leading to potential patient harm.
- It uses a controlled benchmark of 264 drug–disease–drug triples and evaluates 13 state-of-the-art models to quantify DG rates across diverse clinical scenarios.
- Results show high DG rates (up to 87%) under partial evidence conditions, emphasizing the need for explicit entity-attribution checks in clinical RAG systems.
Deceptive Grounding in Clinical Retrieval-Augmented Generation: Entity Attribution Failure and Its Implications
Problem Characterization
This paper exposes a critical and previously under-examined failure mode in clinical retrieval-augmented generation (RAG); termed deceptive grounding (DG), this phenomenon occurs when a model's output is factually accurate, fully grounded in real, correctly cited documents, and passes all standard automated evaluations—yet attributes evidence to the wrong clinical entity. Specifically, DG arises when a system is queried about a drug X but retrieves and synthesizes evidence from documents pertaining to a distinct drug Y (active in the same disease context), presenting Y's clinical data as if it applied to X. This misattribution is invisible to existing hallucination, faithfulness, and citation-check frameworks because all claims remain faithful to retrieved content.
The implications in clinical decision support are pronounced: clinicians may be provided with accurate evidence for an incorrect therapy, leading to potentially serious patient harm. In settings like the clinical literature—replete with similar entities and partial evidence for novel indications—this blind spot represents a significant, under-recognized risk for AI-driven medical tools.
Methodology and Experimental Design
The study introduces a highly controlled and fully-factorial benchmark, constructing 264 drug–disease–alternate-drug (X, C, Y) triples across diverse subdomains, and systematically manipulates both the completeness of evidence for the queried drug (Cx: absent, partial, complete) and the presence/type of clinical evidence for alternate drugs (Cy: null control, class proximate, context adjacent, prior completing, synthetic_Y). The experiment evaluates 13 state-of-the-art LLMs and clinical RAG pipelines, encompassing both general-purpose and medically/biomedically fine-tuned models.
Automated evaluation is performed using Kimi-K2.5, a high-precision LLM-based judge, with human adjudication on a subset for calibration. The principal metric is the DG rate—cases where evidence about Y is faithfully presented as evidence about X, undetectable by existing RAG evaluation frameworks. Complementary mechanistic experiments—ablation studies, activation patching, entity-label substitution, and prompt-based mitigation—are performed to establish causal mechanisms and scope for mitigation.
Results
Prevalence and Conditions of Deceptive Grounding
DG rates are alarmingly high under adversarial retrieval conditions, spanning 8–87% across models; medical and biomedical fine-tuned LLMs register the highest rates (up to 86.7%). Domain specialization amplifies, rather than remedies, the failure. Completing-information presence (e.g., trial names, NCT numbers) in retrieved documents about Y is the decisive risk factor; confabulation dominates only when such information is absent.
The presence of complete, entity-specific evidence for the queried drug (complete Cx) confers near-total protection (DG ≤6.4%), but partial evidence leaves substantial risk (e.g., up to 44% DG in some regimes). Critically, the model's misattribution is driven by information content, not just entity label: synthetic or anonymized Y-entities induce at least as much DG as real drugs, showing that entity-recognition is not an effective protective factor.
Mechanistic Insights
Cross-model and within-model analysis reveal a two-stage mechanism:
- Stage 1: Parametric attribution prior is activated by disease-context overlap or pharmacological class similarity between X and Y, "licensing" the attribution of retrieved evidence across entity boundaries.
- Stage 2: Availability of completing information (CITs) in Y-documents directs the model either towards deceptive grounding (misattribution) or confabulation (unsupported claims about X).
Ablation and activation patching confirm causality: removing Y's clinical evidence eliminates DG in favor of confabulation, and intentionally inserting pharmacological-class representations shifts attribution as predicted.
Detection and Production Analysis
The study introduces entity-attribution verification (EAV)—a per-claim check that explicitly requires cited evidence to match the queried entity—which detects DG at 97.0% precision and 98.7% recall. EAV is absent from standard faithfulness/citation frameworks (e.g., RAGAS, ARES, FaithDial, FActScore). In a real deployed clinical RAG system, 7.8% of responses showed DG under naturalistic conditions; the risk is markedly higher for recently approved drugs with sparse entity-specific evidence (13.6%).
Model-Specific Susceptibility Profiles
Medically fine-tuned models encode stronger pharmacological class structure in their hidden states, which paradoxically increases susceptibility to cross-entity misattribution: fine-tuning that improves domain knowledge also amplifies DG risk. Six model profiles are delineated, spanning from parametric-prior and identity-fusion (greatest risk, in biomedical SFT), to profiles where confabulation dominates. Explicit entity-anchoring instructions are effective mitigation for some profiles, but ineffectual for others—full solution requires architectural as well as prompt-based modifications.
Theoretical and Practical Implications
The work demonstrates that current evaluation protocols for RAG in clinical settings are structurally incapable of surfacing entity attribution errors, even though such errors can have critical downstream consequences. It thus challenges a core assumption behind widespread adoption of faithfulness-based and citation-based evaluation.
From a safety and regulatory perspective, the omission of entity-attribution as an evaluation axis—both in academic benchmarks and regulatory guidelines (e.g., FDA)—constitutes a significant gap. The results suggest that clinical and other high-stakes RAG deployments must adopt explicit entity-attribution checks both in development and in live systems.
On the modeling front, the results suggest that standard SFT, particularly with entity-rich, class-oriented clinical corpora, hardens pharmacological class representations and increases cross-entity "permission" for evidence misattribution. This calls for architectural or procedural enforcement of entity consistency, such as intermediate entity-matching layers, improved retrieval filtering, or post-generative entity-attribution audits.
Recommendations and Future Directions
Three interventions, each addressing a distinct layer of the RAG stack, are advocated:
- Evaluation: Incorporate entity-attribution verification as a first-class check alongside faithfulness and citation validation in all clinical RAG benchmarks and audits.
- Retrieval: Prioritize entity-precise over merely topically or class-adjacent evidence in retrieval modules; complete Cx is the single most effective risk suppressor.
- Training: Enforce hard entity-identity constraints (e.g., via loss terms or prompting) during generation. Noticing entity mismatches is insufficient—corrective mechanisms must prevent generative misattribution.
Empirical investigation is warranted to optimize retrieval filtering, fine-tuning corrections, and to explore the causal pathways through which SFT increases DG susceptibility (including but not limited to pharmacological class structure).
Conclusion
Deceptive grounding represents a systemic, evaluation-invisible failure mode in clinical RAG; a response can be simultaneously accurate, grounded, and clinically wrong due to entity misattribution. The problem is exacerbated—not mitigated—by domain-specialized modeling regimes. As RAG systems increasingly mediate access to biomedical evidence, robust entity-attribution evaluation and enforcement are imperative for safety and reliability. This work thus compels a reassessment of both modeling strategies and evaluation protocols for high-stakes RAG applications.
