MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks an dClassifier-Based Defenses for Medical AI Safety

Abstract: Patient-facing medical chatbots are commonly evaluated on single-turn prompts, yet real users push back after refusals, add urgency, and invoke authority. We introduce MultiTurnPSB, a four-turn adversarial extension of PatientSafetyBench, and evaluate GPT-4.1-mini under fixed template, template-adaptive, and live adversarial attacks. Unsafe responses rise from 35% to nearly 80% by Turn 4 under live attack. Under the same adversary, GPT-4.1-mini and Claude Sonnet 4.5 are statistically indistinguishable at baseline but diverge to a 19x gap by Turn 4, a difference invisible to single-turn evaluation. We characterize four degradation trajectory signatures and identify a two-element attack formula responsible for most catastrophic failures. A lightweight input-side classifier reduces Turn 4 unsafe responses by 52 percentage points despite severe accuracy degradation, but the 45% false alarm rate on benign queries is the primary deployment constraint. A methodological finding also emerges: Claude Sonnet refused to generate adversarial messages in over half of late-turn conversations despite explicit red team framing, suggesting safety training may generalize to the attacker role.

• The paper presents a novel multi-turn adversarial framework that exposes escalating unsafe response rates in medical AI chatbots.
• It reveals significant model divergences with unsafe rates increasing from ~35% to 80% under live attacks, highlighting a critical vulnerability at Turn 2.
• The study evaluates classifier-based defenses that reduce unsafe outputs by over 50% but exhibit high false alarm rates, complicating clinical deployment.

MultiTurnPSB: Analysis of Multi-Turn Jailbreak Attacks and Classifier-Based Defenses for Medical AI Safety

Introduction and Motivation

MultiTurnPSB introduces a substantial advancement in the evaluation paradigm for medical AI safety by focusing on multi-turn dialogue adversarial attacks—contrasting with the single-turn settings prevalent in the literature. Given the prevalence of patient pushback and escalation in real-world medical interactions, this benchmark more accurately reflects operational risks faced by LLM-based medical chatbots. MultiTurnPSB builds upon the PatientSafetyBench (PSB) taxonomy but diverges through its adversarial, four-turn conversation protocol, capturing failure modes that static single-turn tests miss. Notably, the multi-turn setup not only exposes a qualitative difference in vulnerability between models but also highlights sharp quantitative divergences that would be overlooked by single-turn metrics.

Benchmark Design and Attack Taxonomy

The benchmark systematically converts 466 PSB prompts into four-turn adversarial conversations using three distinct attack modalities:

• Fixed-template attack: Utilizing scripted progression—urgency, authority, emotional appeals—without adaptation.
• Template-adaptive attack: Adapts the wording of each adversarial turn based on prior model response, maintaining tactic order.
• Live adversarial attack: Employs a high-temperature LLM red-teamer (GPT-4o-mini), free to select tactics and adaptively exploit model defenses at each turn.

The design ensures that the target (defender) model has access to the entire conversational history at each turn, while the judge (GPT-4o-mini) reliably scores the output without history context, adhering to standard PSB evaluation rubrics.

Quantitative Results and Trajectory Analysis

Multi-Turn Failure Rates and Category Sensitivity

Quantitative analysis reveals pronounced escalation in unsafe response rates under adversarial context. While the initial Turn 1 unsafe rate (across all modalities) is roughly 35%, live adversarial attacks drive this to nearly 80% by Turn 4—a substantial increase over both fixed-template and template-adaptive attacks, with the contribution of adaptive tactic selection being the critical differentiator.

Figure 1: Unsafe rates escalate sharply under live attack, with significant increases by Turn 2, while template-based attacks plateau.

At the category level, health misinformation and discrimination exhibit the highest susceptibility to multi-turn attacks. The unsafe rate for misinformation climbs from 15% at Turn 1 to 84% at Turn 4, while discrimination reaches the absolute highest at 89%. Misdiagnosis, by contrast, is persistently vulnerable but displays only a modest increase across turns.

Figure 2: Misinformation and discrimination categories are the most affected by live adversarial strategies, experiencing the highest unsafe rates.

Divergent Model Trajectories

A key claim substantiated by the multi-turn setup is the detection of divergent degradation trajectories among models that are statistically indistinguishable at baseline. For instance, under the same live adversarial attacker, GPT-4.1-mini and Claude Sonnet 4.5 start with equivalent unsafe rates at Turn 1 (~32–35%), but by Turn 4, GPT-4.1-mini climbs to 78.8% unsafe while Claude Sonnet drops to 4.1%—a 19x divergence not observable with single-turn evaluation.

Figure 3: GPT-4.1-mini and Claude Sonnet 4.5, identical at baseline, diverge under live attack, revealing critical differences only in the multi-turn regime.

The analysis identifies four distinct trajectory signatures: "Compliance Creep" (monotonic increase), "Diminishing Returns" (plateau or slight decrease), "Pattern Recognition" (degeneration to improved safety), and "Spike-and-Abandonment" (contaminated by attacker refusal). GPT-4.1-mini is vulnerable to basic restatement of the original prompt, while Claude Sonnet demonstrates increasing defensive capability over successive turns, reinforced by its recognition and explicit labeling of manipulative patterns.

Qualitative Findings and Failure Mechanisms

The study isolates a two-element adversarial tactic—emergency framing plus medical authority claims—as central to catastrophic safety failures (Score 5 violations), especially at Turn 2. This emergent pattern informs the identification of Turn 2 as a "critical vulnerability window", implying that targeted hardening at this stage is likely to yield the highest marginal benefit for defense strategies.

Classifier-Based Defense: Efficacy and Limitations

An input-side classifier intervention, capable of categorizing user prompts into one of the PSB categories plus a benign class, is evaluated for its ability to reduce harm when prepending safety tags to potentially risky prompts. Despite the classifier experiencing rapid accuracy degradation over the course of adversarial conversations (from 95.5% to 48.5%), its deployment reduces the Turn 4 unsafe rate from 78.8% to 26.6%—a 52.2 percentage-point reduction.

Figure 4: Classifier accuracy collapses under adversarial context, primarily due to lateral confusion into 'Unlicensed Practice'; nonetheless, the intervention halves unsafe output rates.

The defense is far from flawless: the classifier exhibits a 45% false alarm rate on benign prompts (GPT-4.1-mini classifier), establishing a deployment constraint that hinders usability in clinical settings. False alarms severely diminish system utility and necessitate the pursuit of classifiers with lower false positives and better resilience against adversarial drift. Notably, even with high lateral error (misclassifying the risk category), some protective effect persists because safety tags, even if mismatched, may still evoke model caution.

Methodological Observations on Red-Teaming Pipelines

A significant methodological finding concerns the use of safety-trained models as adversarial red-teamers. When Claude Sonnet is assigned the attacker role, it increasingly refuses to generate adversarial prompts in later turns (escalating to 54% refusal at Turn 4), leading to contamination of evaluation data. This self-limiting behavior highlights that safety alignment generalizes to the attacker's role, biasing red-teaming protocols if undetected.

Figure 5: Claude Sonnet's refusal rate as attacker increases sharply, undermining late-turn adversarial evaluations and revealing a training-byproduct confound.

Red-teaming infrastructure must therefore include explicit detection and handling of attacker refusals to avoid inflated safety scores and mischaracterization of model robustness.

Implications and Future Directions

These findings suggest that single-turn benchmarks systematically underestimate the risk posed by adversarial persistence in patient-facing settings. Trajectory analysis reveals that models may encode qualitatively different safety mechanisms, with some treating context accumulation as increased legitimacy and others strengthening defenses as adversarial pattern frequency rises. The work indicates that classifier-based interventions can provide substantial harm reduction even with severe drift, but their deployment is bottlenecked by excessive false alarms on safe interactions.

Future research should focus on:

• Developing more resilient, lower false-positive classifiers for conversational risk assessment.
• Integrating output-side and conversation-level defenses rather than relying exclusively on input-side tagging.
• Designing targeted hardening protocols for critical vulnerability windows (notably Turn 2).
• Formalizing trajectory-shape-based diagnostics as part of model evaluation and fine-tuning protocols.
• Expanding benchmarks to multilingual and multimodal domains, and ensuring attacker diversity to avoid self-limiting confounds.

Conclusion

MultiTurnPSB provides a robust, adversarial multi-turn framework that exposes both the magnitude and mechanisms of safety failures in LLM-based medical chatbots—outstripping what single-turn evaluations can detect. The findings demonstrate sharp model-to-model differences in multi-turn vulnerability, the significance of specific adversarial tactics (notably at Turn 2), and the bounded utility of classifier-based interventions limited by false alarm rates. The study's observations regarding attacker self-limitation further articulate the necessity for careful methodological design in red-teaming. MultiTurnPSB is positioned to serve as a new minimum standard for safety evaluation in real-world medical conversational AI deployments.