Multi-turn Resistance Indices in LLMs

• Multi-turn resistance indices are standardized metrics that quantify LLM robustness against adversarial attacks over multiple conversational turns.
• Key measurements such as Attack Success Rate and Differential Vulnerability Metric enable direct comparisons of model safety and context sensitivity.
• Evaluation methodologies combine automated scenario generation and human validation to assess vulnerabilities like prompt injection and context-dependent jailbreaks.

Multi-turn resistance indices quantify the robustness of LLMs to @@@@1@@@@ executed over multiple conversational turns. These metrics expose the capacity of advanced models to withstand prompt injection, psychological manipulation, and context-dependent jailbreak techniques. While originally motivated by the need to evaluate safety alignment vulnerabilities in LLMs, the multi-turn resistance index formalism provides a standardized, model-agnostic framework for reporting and benchmarking contextual adversarial robustness at scale (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

1. Formal Definition and Metrics

The central metric for multi-turn robustness is the Attack Success Rate (ASR), operationalized as the proportion of adversarial conversation threads that successfully induce a model to produce fully harmful content. For a model $m$, the multi-turn resistance index is

$$ASR_m = \frac{\text{Number of Successful Adversarial Conversations}_m}{\text{Total Adversarial Conversations}_m} \times 100\%$$

A lower ASR corresponds to higher resistance. Some studies use the complementary resistance index $R_m = 1 - ASR_m$ for interpretability, though ASR remains the primary reporting convention (Young, 8 Dec 2025).

Differential metrics further illuminate context sensitivity. The Differential Vulnerability Metric (DVM) or $\Delta ASR$ compares attack success with and without dialogue history:

$$\Delta ASR = ASR_{\text{multi}} - ASR_{\text{single}}$$

A large positive $\Delta ASR$ indicates increased vulnerability due to conversational context; a negative value indicates that history disables certain attacks (Kumarappan et al., 24 Nov 2025).

2. Evaluation Methodologies

Multi-turn resistance indices are grounded in adversarial testbeds employing rigorous automation and prompt design protocols. Leading frameworks include TEMPEST (Tree-based Exploration of Multi-turn Prompts for Eliciting Safety Thresholds), which adaptively branches adversarial scenarios, and pipelines leveraging psychological escalation mechanisms such as the Foot-in-the-Door principle (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

Key workflow phases:

• Automated scenario generation: Large, diverse banks of multi-turn attacks, including escalation and contextual manipulation, are synthesized with programmatic templates.
• Model interaction: Each model is exposed to sequences (typically 5–6 turns deep, with parallel branches), preserving conversation history.
• Safety evaluation: Independent automated classifiers (e.g., DeepSeek V3.1, Gemini 1.5 Flash) assign harm scores to candidate responses; thresholds define jailbreak success (e.g., score $\geq 10$ for full harm).
• Human validation: Subsets of attack results are manually reviewed to calibrate classifier bias and compute agreement metrics (Cohen’s $\kappa \geq 0.5$ observed) (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

Robustness is generally summarized across large, diverse threat surfaces (e.g., 1,000–1,500 behaviors spanning misinformation, hate speech, illegal activities, and privacy violations). Measurements include both the overall ASR and the average number of turns to first jailbreak as an auxiliary resistance signal.

3. Empirical Results Across Frontier Models

Large-scale studies have characterized multi-turn adversarial resistance across leading commercial and open models. Attack surfaces extend over tens of thousands of adversarial conversations per model, enabling statistically meaningful inter-model comparison (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

Representative Benchmark Table

Model	Multi-Turn ASR (%)	Avg. Turns to Jailbreak	ΔASR (pp) vs. Single-Turn
Gemma3 12B	100.0	1.1	n/a
Mistral Large 3 675B	100.0	1.0	n/a
DeepSeek V3.1 671B	99.0	1.6	n/a
Kimi K2 1T	97.0	1.6	n/a
Cogito 2.1 671B	96.0	3.6	n/a
GPT-OSS 20B	78.0	9.8	n/a
MiniMax M2 230B	55.0	22.7	n/a
Kimi K2 (Thinking) 1T	42.0	17.2	n/a
GPT-4o Mini	65.45	n/a	+21.10
Claude 3 Haiku	1.35	n/a	+1.00
Gemini 2.5 Flash	0.10	n/a	–0.55

A substantial cluster of models (Gemma3, Mistral, DeepSeek, Kimi K2) exhibit nearly total brittleness ($ASR \geq 96\%$). Several models demonstrate partial resistance ($ASR \approx 42-78\%$), while context-agnostic architectures (e.g., Gemini 2.5 Flash) approach immunity ($ASR \leq 0.1\%$) (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

GPT-family models are particularly susceptible to context-priming, with $\Delta ASR$ up to +32 percentage points for illegal activity scenarios. In contrast, Gemini 2.5 Flash exhibits a negative $\Delta ASR$ (multi-turn more resistant than single-turn), suggesting robust decoupling of safety policy from dialogue history (Kumarappan et al., 24 Nov 2025).

4. Architectural and Methodological Factors

Resistance indices reveal key distinctions in LLM safety architecture:

• Alignment Quality over Scale: No observable correlation exists between parameter count and multi-turn ASR (Spearman $\rho = -0.12$, $p = 0.74$): parameter count is neither necessary nor sufficient for adversarial robustness. Rigorous safety alignment dominates capacity scaling as the critical determinant (Young, 8 Dec 2025).
• Inference Protocol – Extended Reasoning: Mechanisms such as 'thinking mode' (MoE-driven chain-of-thought) considerably increase resistance; for Kimi K2, enabling deliberative inference reduced ASR from $97\%$ to $42\%$ and increased mean turns to jailbreak by $>10\times$. This suggests that explicit, model-internal reasoning chains can trigger latent refusal patterns otherwise absent in base inference (Young, 8 Dec 2025).
• Safety Filter Modality: Models that apply context-independent, pre-generation filters (Gemini 2.5 Flash) are substantially more resistant to FITD-style context escalation than those evaluating safety interactively across turns (GPT-family) (Kumarappan et al., 24 Nov 2025).
• Computational Cost for Attackers: Models with greater resistance require adversaries to consume significantly higher API resources (up to $27\times$ more queries), introducing a practical barrier in threat scenarios—though not absolute prevention (Young, 8 Dec 2025).

5. Interpretation and Diagnostic Utility

Multi-turn resistance indices function both as comparative metrics and diagnostic tools for understanding systemic vulnerabilities.

• Comparative Benchmarking: Resistance indices allow direct model-to-model comparisons along a standardized adversarial axis, revealing qualitative differences in robustness otherwise masked by single-turn benchmarks.
• Diagnostic Patterns: The $\Delta ASR$ metric identifies context-based susceptibility, e.g., distinguishing models vulnerable to narrative escalation from those exhibiting context-invariant safety refusal.
• Threat Surface Mapping: By enumerating resistance across diverse attack types (misinformation, illegal activity, etc.), indices guide the targeted development and evaluation of model-specific defensive interventions.

The consistent finding that sophisticated multi-turn attacks defeat all current frontier models to varying degrees—despite efforts in scaling and alignment—underscores the need for new mechanisms decoupling safety policies from conversational state and deeper mechanistic understanding of safety reasoning (Young, 8 Dec 2025, Kumarappan et al., 24 Nov 2025).

6. Implications and Future Directions

The persistence of high ASR values under multi-turn attack regimens indicates a critical requirement for advancing both evaluation and defense methodologies:

• Standardization of Multi-turn Assessment: Single-turn metrics fundamentally underestimate real-world adversarial risk. Multi-turn resistance indices should become the default metric for robustness claims.
• Mechanistic Ablation: Investigating how chain-of-thought depth and visibility interact with safety activation in reasoning-mode architectures is critical for generalizing observed gains (Young, 8 Dec 2025).
• Defense-aware Adversarial Research: Next-generation frameworks should explicitly target reasoning chains and adaptive behaviors, refining both attack and defense strategies for deployment-oriented safety.
• Longitudinal and Cross-Dataset Tracking: Ongoing evaluation across updates and diverse benchmarks (e.g., HarmBench, AdvBench) is necessary to ensure that observed resistance generalizes and persists.

A plausible implication is that real progress in adversarial resistance may only be achieved by shifting beyond turn-level post hoc filtering and toward architectures in which safety policy is manifestly independent of, or actively resists, conversational manipulation.

7. Related Indices in Mathematical and Graph Theoretic Contexts

In graph theory, resistance-based indices (e.g., Kirchhoff index, multiplicative degree–Kirchhoff index) quantify network robustness against current flow and graph transformations (Zhao et al., 2019). While the domains differ, both classes of indices operationalize resistance—whether to physical flow or adversarial exploitation—as normalized functions of structural or interactional complexity:

$$Kf(O_n) = \frac{27n^3 + 51n^2 + 26n + 4}{6}, \quad Kf^*(O_n) = \frac{507n^3 + 559n^2 + 204n + 8}{6}$$

The analogy highlights the general principle of resistance indices as global robustness metrics arising from local adversarial or perturbative threats. This conceptual parallel invites further cross-pollination of resilience measures between graph networks and LLM safety frameworks.