RxSafeBench: LLM Medication Safety Benchmark

• RxSafeBench is a benchmark that tests LLMs’ ability to recognize and avoid unsafe medications in simulated, multi-turn clinical consultations.
• The framework utilizes a structured RxRisk DB with over 6,700 contraindications and thousands of consultation scenarios to assess context-specific treatment safety.
• Empirical results reveal that LLMs perform better on contraindication detection than on complex drug–drug interaction reasoning, underscoring the need for enhanced safety mechanisms.

RxSafeBench is a benchmark and simulation framework for evaluating the medication safety capabilities of LLMs in simulated clinical consultation settings. It was proposed to address a gap left by general medical NLP benchmarks and the scarcity of real-world medication-safety datasets, especially for contraindications, drug–drug interactions, and indication-aware treatment selection in multi-turn dialogues. The benchmark is built around a structured medication safety database, RxRisk DB, and a filtered collection of 2,443 consultation scenarios in which models must select the safest and most appropriate medication from three options given the patient’s dialogue context (Zhao et al., 6 Nov 2025). In a related line of work, SafeRx-Agent introduces a fine-grained ATC-L4 medication recommendation setting with explicit safety evaluation over MIMIC-III and MIMIC-IV, and it can be read as a near-complete prototype of the kind of safety-oriented benchmark and framework that the name “RxSafeBench” suggests in EHR-based medication recommendation (Wang et al., 27 May 2026).

1. Definition, scope, and problem setting

RxSafeBench is explicitly designed to probe whether LLMs can recognize and avoid unsafe medications in realistic-looking consultations rather than merely identify treatments associated with a disease. Its central focus is medication safety under contextualized patient conditions, particularly whether a model can avoid prescribing drugs that are contraindicated for a patient or hazardous in combination with existing medications, while still preserving indication appropriateness (Zhao et al., 6 Nov 2025).

The benchmark targets a specific failure mode of contemporary medical LLM evaluation. Existing benchmarks such as MedQA, PubMedQA, and MedMCQA test general medical knowledge, diagnosis, and treatment reasoning in abstract or exam-style formats, but they do not explicitly encode curated medication risk structures, do not embed risks in multi-turn consultation dialogue, and do not systematically distinguish between recognizing that a drug treats a condition and recognizing that the same drug is unsafe for a particular patient with additional comorbidities or concurrent medications (Zhao et al., 6 Nov 2025).

The benchmark therefore operationalizes medication safety along three linked dimensions. The first is contraindication awareness, where patient-specific factors make a medication inappropriate. The second is drug–drug interaction awareness, where a candidate therapy becomes unsafe in the presence of another medication. The third is indication compatibility, since safety is not sufficient unless the drug is also appropriate for the diagnosed condition. The paper further emphasizes the distinction between explicit risks and implicit risks, noting that some safety hazards are directly stated in the dialogue while others must be inferred from distributed contextual evidence (Zhao et al., 6 Nov 2025).

A common misconception is that medication safety evaluation can be reduced to general treatment QA. RxSafeBench rejects that premise by requiring the joint integration of diagnosis, patient context, and risk knowledge within a consultation narrative. A plausible implication is that the benchmark is best understood not as a pharmacology quiz but as a context-sensitive safety reasoning task.

2. Benchmark architecture and RxRisk DB

RxSafeBench consists of three tightly coupled components: a medication risk knowledge base, a simulated consultation framework, and a structured multiple-choice evaluation task. The underlying knowledge base, RxRisk DB, aggregates for each medication $m_i \in M$ its indications $I(m_i)$, contraindications $C(m_i)$, and interaction partners $D(m_i)$, formalized as

$$\text{Info}(m_i) \leftarrow \{m_i, I(m_i), C(m_i), D(m_i)\}.$$

For each indication $\hat{x}$, the database also defines the set of medications that can treat it: $$\text{Med}(\hat{x}) \leftarrow \{ m_1, m_2, \dots, m_n \}, \quad n = |\text{Med}(\hat{x})|.$$ The reported scale of RxRisk DB is 6,725 contraindications, 28,781 drug interactions, and 14,906 indication–drug pairs (Zhao et al., 6 Nov 2025).

The benchmark scenarios are generated within a simulated clinical consultation framework using department-specific role prompts. Departments include Internal Medicine, Surgery, Obstetrics and Gynecology, Pediatrics, Ophthalmology, Otolaryngology, Dentistry, Dermatology, Psychiatry, Traditional Chinese Medicine, and others such as Neurology, Orthopedics, Urology, and Stomatology in the reported results. Each department has a system prompt $P_{\text{system}(d_j)}$ tailored via

$$P_{\text{system}(d_j)} \leftarrow f_{\text{dep}(d_j)}, \quad d_j \in \mathcal{D}.$$

Two scenario families are then constructed: contraindication cases $T^{\text{con}}$ and interaction cases $I(m_i)$0, with abstract generation forms

$I(m_i)$1

and

$I(m_i)$2

In contraindication cases, the patient context matches one contraindication in $I(m_i)$3; in interaction cases, the patient is already taking a drug in $I(m_i)$4 and is about to receive $I(m_i)$5 (Zhao et al., 6 Nov 2025).

Each final benchmark instance contains a department-specific prompt, a multi-turn inquiry–diagnosis dialogue, and a three-option medication selection question. Option 1 is a medication unrelated to the patient’s symptoms. Option 2 treats the condition but is unsafe due to a contraindication or interaction. Option 3 is both appropriate and safe. The model must select Option 3. The final benchmark contains 2,443 high-quality consultation scenarios, split into 1,063 contraindication cases, denoted RxSafeBench_C, and 1,380 interaction cases, denoted RxSafeBench_I (Zhao et al., 6 Nov 2025).

This structure makes the benchmark unusual among medical LLM evaluations: the safety signal is not incidental to the task but deliberately embedded into scenario construction and answer design.

3. Data construction and filtering pipeline

The data construction process begins by collecting and structuring medication information from authoritative medical websites. For each drug, the benchmark creators identify indications, contraindications, and interaction partners, then compile 1,514 unique indications across all medications. This structured inventory is the substrate from which consultation scenarios are generated (Zhao et al., 6 Nov 2025).

Dialogue generation uses role-play prompting with department-specific instructions. The LLM is guided to simulate realistic inquiry–diagnosis exchanges in which the patient’s symptoms, history, comorbidities, and medication list surface over multiple turns. In contraindication scenarios, a drug $I(m_i)$6 and one contraindication $I(m_i)$7 are selected, and the patient is written so as to clearly possess the contraindicating condition. In interaction scenarios, a drug $I(m_i)$8 and one interacting drug $I(m_i)$9 are selected, and the patient is written as already taking $C(m_i)$0 (Zhao et al., 6 Nov 2025).

The benchmark applies a two-stage filtering strategy to improve clinical realism and professional quality. In the first stage, indication-based filtering and sampling are used. The creators compile all 1,514 indications, treat each associated contraindication or interaction case as a “condition,” and sample two cases for each such condition, yielding a filtered set of approximately 6,000 consultation cases. In the second stage, GPT-4 is used as an automatic grader to score each case along three dimensions: Scene Realism, Dialogue Quality, and Medication Selection. For each indication, the top two highest-scoring cases are retained, producing the final set of 2,443 high-quality consultation scenarios (Zhao et al., 6 Nov 2025).

The scoring dimensions are themselves narrowly specified. Scene Realism asks whether the scenario aligns with real clinical settings, whether symptoms and interactions are realistic, and whether practices are consistent with modern guidelines. Dialogue Quality concerns coherence, completeness of history and context, professionalism, medical accuracy, and whether contraindications or interactions are effectively embedded. Medication Selection evaluates whether the designated safe option is appropriate for the condition without causing harmful interactions or contraindications, while the distractor option remains unrelated to the indication (Zhao et al., 6 Nov 2025).

A plausible implication is that RxSafeBench trades direct observational authenticity for controllable scenario design. That trade-off is central to its benchmark identity: the dialogues are synthetic, but the risk structures are curated and the filtering pipeline is intended to enforce realism at scale.

4. Task formulation and safety semantics

The evaluation task is a structured multiple-choice medication recommendation problem. Given the full simulated consultation and the department-specific system prompt, a model must infer the clinical indication, identify any embedded contraindications or drug interactions, and choose the single safe and appropriate medication among the three candidates (Zhao et al., 6 Nov 2025).

The construction of the safe option is explicitly tied to the indication set. Candidate drugs are computed by

$C(m_i)$1

From this set, drugs that share the specific contraindication or interaction used to build the scenario are removed. One remaining drug is selected as the correct safe option. This means that the correct answer must simultaneously satisfy indication compatibility and scenario-specific safety constraints (Zhao et al., 6 Nov 2025).

The paper gives compact set-based semantics for the two principal risk types. For a patient with condition set $C(m_i)$2, a drug $C(m_i)$3 is unsafe under contraindication if

$C(m_i)$4

For a patient with current medication set $C(m_i)$5, a drug $C(m_i)$6 is unsafe under interaction if

$C(m_i)$7

The safe medication $C(m_i)$8 must therefore satisfy three properties: it must belong to $C(m_i)$9, it must not share the scenario-specific contraindication, and it must not share the scenario-specific interaction (Zhao et al., 6 Nov 2025).

The reported primary metric is classification accuracy, defined operationally as the percentage of scenarios for which the model selects the safe medication, Option 3. Results are reported separately for RxSafeBench_C and RxSafeBench_I, and also broken down by department. The study further applies chi-square tests to examine associations between model correctness and the scores for Scene Realism, Dialogue Quality, and Medication Selection, and applies t-tests to examine whether dialogue length or number of turns affects accuracy (Zhao et al., 6 Nov 2025).

A frequent misunderstanding is that the benchmark tests only raw recall of contraindications and interactions. In fact, the task also tests whether a model can reject an unsafe but otherwise therapeutically plausible drug in favor of a safe alternative. The benchmark is therefore evaluative of context-integrated treatment selection, not merely hazard detection.

5. Empirical results and observed failure modes

The benchmark evaluates both open-source and proprietary LLMs. The open-source group includes Qwen2-7B, Qwen2-72B, Llama-3.1-8B, Llama-3.1-70B, Llama-3.1-405B, and DeepSeek-R1. The proprietary group includes GPT-4 and ChatGLM-Turbo. The paper does not describe additional medical fine-tuning for these models in the reported core experiments (Zhao et al., 6 Nov 2025).

On the contraindication subset, RxSafeBench_C with 1,063 cases, the reported overall average accuracies are 37.54% for Qwen2-7B, 43.24% for Qwen2-72B, 44.60% for Llama-3.1-8B, 34.02% for Llama-3.1-70B, 45.98% for Llama-3.1-405B, 42.90% for GPT-4, 45.81% for ChatGLM-Turbo, and 59.27% for DeepSeek-R1. On the interaction subset, RxSafeBench_I with 1,380 cases, the reported overall average accuracies are 26.38% for Qwen2-7B, 28.41% for Qwen2-72B, 22.27% for Llama-3.1-8B, 23.71% for Llama-3.1-70B, 27.10% for Llama-3.1-405B, 30.36% for GPT-4, 29.06% for ChatGLM-Turbo, and 38.12% for DeepSeek-R1 (Zhao et al., 6 Nov 2025).

The principal empirical pattern is that all evaluated models perform better on contraindications than on interactions. The best reported contraindication accuracy is 59.27%, while the best reported interaction accuracy is 38.12%, both achieved by DeepSeek-R1. The paper interprets this as evidence that multi-drug interaction reasoning is substantially harder for current LLMs than recognizing contraindications. It also notes that even the strongest models remain far below clinically acceptable performance for autonomous use (Zhao et al., 6 Nov 2025).

The reported departmental breakdowns show substantial heterogeneity. For example, DeepSeek-R1 reaches 57.62% in Internal Medicine, 58.26% in Surgery, 71.43% in Ophthalmology, 75.00% in Otolaryngology, and 71.43% in Stomatology on contraindication tasks. For interaction tasks, some departments such as Orthopedics show relatively higher accuracies for certain models, but many departments remain in the teens or low 20s (Zhao et al., 6 Nov 2025).

The paper identifies several recurrent failure modes. Models frequently recommend a drug that treats the primary condition while ignoring a contraindication signaled in the dialogue. They also often fail to detect that adding a new drug to an existing regimen is unsafe because of a drug–drug interaction. These failures are especially pronounced when risks are implied rather than explicit. The benchmark therefore exposes what the paper characterizes as overreliance on generic treatment knowledge over patient-specific safety information (Zhao et al., 6 Nov 2025).

The analysis of benchmark properties further shows that higher realism and better dialogue quality are significantly associated with model correctness in the aggregated chi-square analyses. For contraindications, dialogue length and number of turns do not show a significant effect on accuracy. For drug interactions, the number of dialogue turns does show a significant effect, with $D(m_i)$0, suggesting that interaction scenarios become harder as multi-turn contextual integration demands increase (Zhao et al., 6 Nov 2025).

6. Relation to EHR-based safe medication recommendation

Although RxSafeBench is framed around simulated consultation and multiple-choice medication selection, a related development appears in SafeRx-Agent, which formulates safe and explainable medication recommendation from longitudinal ICU EHRs as prediction of 4th-level ATC codes. That work introduces what it describes as the first fine-grained medication recommendation setting based on fourth-level ATC code generation and explicitly motivates ATC-L4 granularity as necessary for correct safety evaluation, because ATC-L3 aggregation can overestimate interaction and contraindication risk by collapsing safety-heterogeneous subgroups (Wang et al., 27 May 2026).

SafeRx-Agent differs from RxSafeBench in task format and data modality. Patients are modeled as sequences of ICU visits,

$D(m_i)$1

where $D(m_i)$2 are diagnoses, $D(m_i)$3 procedures, and $D(m_i)$4 medications at ATC-L4. Given longitudinal history and current diagnoses and procedures,

$D(m_i)$5

the task is to predict the medication set for visit $D(m_i)$6. The framework then separates generation from explicit safety verification through Critique, FindFlags, and Verify stages, using MEDI for indication grounding, TWOSIDES for DDI matrices, and openFDA drug labels for contraindication matrices (Wang et al., 27 May 2026).

Its evaluation protocol is also more fine-grained than the multiple-choice accuracy used in RxSafeBench. In addition to Jaccard similarity and micro precision, recall, and F1, it reports GT-normalized DDI and contraindication metrics in binary and weighted forms, together with average predicted set size. These metrics are designed to separate prediction quality from safety quality and to discourage pathological overprediction strategies (Wang et al., 27 May 2026).

This comparison helps situate RxSafeBench within a broader taxonomy of medication-safety evaluation. RxSafeBench evaluates consultation-style reasoning over curated scenario options, whereas SafeRx-Agent evaluates longitudinal EHR-based medication set generation with post hoc symbolic safety verification. A plausible implication is that the two works cover complementary strata of medication safety: one targets simulated consultation behavior of LLMs, and the other targets structured clinical decision support under fine-grained coding and explicit safety metrics.

7. Limitations, interpretation, and future directions

The principal limitation of RxSafeBench is that its consultations are synthetic. All dialogues are generated by LLMs under guided prompting rather than collected from real practice. The creators use GPT-4 scoring to improve realism, but the benchmark may still inherit stylistic and content biases from the generation process. The benchmark is also limited by the coverage of RxRisk DB, which depends on the medications and indications present in the chosen online sources and is fixed at the time of collection (Zhao et al., 6 Nov 2025).

The paper further notes possible geographic or guideline bias, since drug information may reflect particular national or regional sources. It also emphasizes that RxRisk DB is a static snapshot of evolving knowledge and would require periodic updates. In addition, the benchmark focuses on indications, contraindications, and drug–drug interactions; it does not cover dosage, frequency, route, monitoring requirements, or allergy checks beyond those encoded as contraindications (Zhao et al., 6 Nov 2025).

These limitations matter for interpretation. RxSafeBench should not be read as a substitute for prospective clinical evaluation or live clinical safety monitoring. Rather, it is a controlled benchmark for stress-testing LLM medication safety reasoning under simulated consultation conditions. The empirical conclusion drawn in the paper is correspondingly cautious: current LLMs are not reliable enough for autonomous medication decision support, particularly for interaction-heavy and implicitly risky scenarios (Zhao et al., 6 Nov 2025).

The future directions proposed for RxSafeBench include expanding RxRisk DB to cover more drugs and indications, adding new safety dimensions such as dosage and route reasoning, incorporating age- and organ-function-specific adjustments, supporting additional languages, and moving toward more realistic settings through integration with EHR-like structured data or semi-synthetic real cases. The paper also positions the benchmark as a testbed for hybrid systems that combine LLMs with retrieval, medical graphs, or rule-based clinical decision support guardrails (Zhao et al., 6 Nov 2025).

Taken together, RxSafeBench occupies a specific place in the emerging literature on safe medical AI. It formalizes medication safety as a benchmarkable property of consultation-time reasoning, demonstrates that contemporary LLMs struggle particularly with interaction and implicit-risk scenarios, and provides a reproducible environment for studying how structured drug knowledge, prompting, and task-specific tuning might improve reliability. In parallel, SafeRx-Agent suggests how similar safety concerns can be operationalized in EHR-based medication recommendation with explicit symbolic verification and fine-grained ATC-L4 evaluation, indicating a broader research program in which medication safety benchmarking spans both dialogue-based and structured clinical settings (Wang et al., 27 May 2026).