Jailbreak Tax in LLM Safety and Performance

• Jailbreak Tax is defined as the loss of capability when a model is jailbroken compared to its unaligned version.
• It encompasses metrics such as attack success rate, utility degradation, latency, and comprehensive cost on attackers and defenders.
• Empirical findings reveal that bypassing safety often incurs significant accuracy and usability trade-offs, prompting multi-objective evaluation.

Searching arXiv for the cited papers to ground the article in current research. “Jailbreak tax” is a research term for the performance, utility, and operational penalties associated with bypassing model safety guardrails. In its narrowest and most explicit formulation, the term denotes the loss in capability that occurs when an aligned model is jailbroken and then answers worse than the original unaligned model on the same task. In a broader formulation developed for large audio-LLMs, it denotes the total cost—on both the attacker and the defender or system—of moving from a benign interaction to a successful jailbreak or to a defended system, with explicit attention to attack success, benign refusal, and latency rather than success alone (Nikolić et al., 14 Apr 2025, Feng et al., 28 May 2026).

1. Origin and conceptual scope

The paper “The Jailbreak Tax: How Useful are Your Jailbreak Outputs?” defines jailbreak tax as the relative drop in utility when using a jailbreak on an aligned model rather than using the corresponding unaligned model directly. The motivating question is not simply whether a jailbreak yields a non-refusal answer, but whether the answer is any good compared to what the model could have produced before alignment. The paper distinguishes this from jailbreak success rate, from ordinary accuracy, and from safety metrics that measure refusal or harmfulness without measuring answer quality (Nikolić et al., 14 Apr 2025).

Subsequent work broadens the term. In “Audio Jailbreaks in Large Audio-LLMs: Taxonomy, Attack-Defense Analysis, and Cost-Aware Evaluation,” a “jailbreak tax” for LALMs is defined as the total cost—on both the attacker and the defender or system—of moving from a benign interaction to a successful jailbreak or to a defended system. That paper explicitly argues for moving beyond “is the jailbreak possible?” to “what does it cost in practice?” in terms of attack success rate (ASR), benign refusal rate (BRR), and latency (Feng et al., 28 May 2026).

A third usage appears in “Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning,” where the term refers to the capability degradation that prior adversarial fine-tuning attacks had imposed on reasoning benchmarks. Trojan-Speak’s central claim is that this tradeoff is not inherent: prior work reported more than 25% capability degradation, whereas Trojan-Speak reports less than 5% degradation for models with 14B+ parameters while achieving 99+% classifier evasion (Sel et al., 30 Mar 2026).

These usages are not identical, but they are compatible. One measures degradation in answer utility relative to an unaligned baseline; another measures attacker- and defender-side costs of achieving or preventing jailbreaks; a third asks whether evasion can be achieved without materially degrading the model’s general reasoning.

2. Formalization and metrics

The capability-centric formulation is explicit. Let $\mathcal{D} = \{(p_i, y_i)\}_{i=1}^n$ be a dataset of prompts with ground-truth answers, let $f$ be the aligned model, and let $\mathcal{A}$ be a jailbreak attack. If $\mathcal{A}(f,p)=\bot$, the attack failed; otherwise it elicited a non-refusal answer. The paper defines:

$$\text{JailSucc} = \Pr_{p \sim \mathcal{D}}[\mathcal{A}(f,p) \neq \bot]$$

$$\text{JailUtil} = \Pr_{(p,y)\sim \mathcal{D}}[\mathcal{A}(f,p)=y \mid \mathcal{A}(f,p)\neq \bot]$$

$$\text{BaseUtil} = \Pr_{(p,y)\sim \mathcal{D}}[f_{\text{base}}(p)=y]$$

and

$$\text{JTax} = \frac{\text{BaseUtil} - \text{JailUtil}}{\text{BaseUtil}}$$

Here, $\text{JTax}=0$ means the jailbreak preserves all of the base model’s capability, whereas $\text{JTax}=1$ means the jailbroken answers are completely useless compared to the base model (Nikolić et al., 14 Apr 2025).

The cost-aware LALM formulation introduces a different but complementary metric stack. For a harmful dataset $f$0, attack $f$1, model $f$2, and evaluator $f$3, the paper defines

$f$4

For a benign dataset $f$5, it defines

$f$6

where $f$7 marks whether the system refused a benign request. Latency is decomposed into offline and online components, and the paper treats ASR, BRR, latency, cost, stealth, and response utility as a multi-objective evaluation problem rather than collapsing them into a single success statistic (Feng et al., 28 May 2026).

This metric split is important because jailbreak success and jailbreak utility are not the same object. A jailbreak can produce a non-refusal output while still destroying most of the underlying task capability, and a defense can suppress ASR while substantially increasing benign refusal or runtime overhead.

3. Capability degradation as the original jailbreak tax

The most direct empirical treatment of jailbreak tax uses benign but objectively scorable tasks that models are artificially aligned to refuse. The core datasets are WMDP-bio, GSM8K, MATH, EvilMath, and UnicornMath. Alignment is induced through system prompt alignment, supervised fine-tuning alignment, or harmful-looking rewording that leverages a model’s built-in safety behavior. This makes the task evaluable with exact-match or multiple-choice grading while preserving the operational structure of a jailbreak setting (Nikolić et al., 14 Apr 2025).

The central finding is a “consistent drop in model utility in jailbroken responses,” which the paper terms the jailbreak tax. It reports that while all tested jailbreaks bypass guardrails in models aligned to refuse to answer math, this can come “at the expense of a drop of up to 92% in accuracy.” On Claude 3.5 Haiku in the EvilMath setting, PAIR and TAP achieve more than 99% jailbreak success, but the accuracy relative to UnicornMath baseline drops by up to 26%. On MATH level 5, Multijail and TAP reduce original accuracy by more than 40%, while PAIR causes more than 80% reduction (Nikolić et al., 14 Apr 2025).

The same study shows that jailbreak success rate and jailbreak tax are largely independent. System-prompt jailbreaks and counter-finetuning jailbreaks often achieve near-zero jailbreak tax in the settings for which they are targeted, whereas PAIR and TAP often achieve very high success but also the largest utility degradation. Many-shot jailbreaks show mixed behavior: success is often lower, but when they do bypass, utility remains closer to baseline. The factual implication is that high refusal-bypass rates do not imply faithful recovery of the underlying capability.

The paper also distinguishes jailbreak tax from alignment tax. In its setup, alignment tax is small on unrelated tasks: models aligned to refuse math or biology retain essentially unchanged performance on neutral benchmarks. The measured degradation therefore arises from the interaction of alignment and the specific jailbreak strategy, not from a blanket destruction of model competence (Nikolić et al., 14 Apr 2025).

4. Cost-aware extensions in audio-LLMs

Audio-language work generalizes the concept from capability loss to a broader attack-defense economics. LALM jailbreaks are organized into semantic, acoustic, signal, and embedding-layer attacks; defenses are grouped into guard-based, training-free, and training-based methods; benchmarks are categorized as cross-modal, audio-native, and interactive. The empirical study evaluates representative attacks and defenses across ten open-source LALMs and measures ASR, BRR, and latency rather than success alone (Feng et al., 28 May 2026).

Several results are central. Acoustic Best-of-N shows the highest ASR of all attacks under no defense at 0.458, while Narrative Framing is identified as “an effective low-latency semantic threat” with ASR 0.376 and total latency about 11.9 seconds. Signal Best-of-N reaches ASR 0.223. On the defense side, VoiceShield reduces average ASR from 0.245 to 0.165 but raises BRR from 0.171 to 0.307. A defensive system prompt reduces average ASR to 0.064 but raises BRR to 0.461. The paper’s conclusion is that current defenses “trade robustness against benign usability” (Feng et al., 28 May 2026).

Latency makes the cost interpretation explicit. Acoustic Best-of-N at $f$8 incurs 74.825 seconds total latency and a 22.460$f$9 slowdown relative to baseline, whereas Narrative Framing incurs 11.908 seconds total latency and a 3.574$\mathcal{A}$0 slowdown. This establishes a distinction between low-cost, human-usable attacks and high-cost worst-case attacks. The paper therefore argues that LALM safety evaluation should be treated as a multi-objective problem over ASR, BRR, latency, cost, stealth, and response utility, and that cost- and utility-aware evaluation is a necessary complement to success-rate-only safety benchmarks (Feng et al., 28 May 2026).

This suggests that “jailbreak tax” in multimodal systems is not only the degradation of the harmful answer’s quality, but also the budget required to produce that answer and the usability cost imposed by defenses.

5. Attacker-side and defender-side economics

“JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs” does not use the phrase “jailbreak tax,” but it makes the relevant components measurable. It studies 13 jailbreak methods across six aligned LLMs, using a unified forbidden-question dataset with 160 questions from 16 violation categories. The paper reports a baseline ASR of 0.42 without jailbreaks and an overall average ASR of 0.56 with jailbreaks, showing that even aligned models already exhibit non-trivial harmful compliance and that jailbreaks materially increase it (Chu et al., 2024).

The paper also exposes explicit resource tradeoffs. Human-based methods attain high ASRs in black-box settings—AIM reaches an average ASR of 0.76—but average token count for human-based prompts is about 676.79 tokens, with AIM alone averaging about 382.78 tokens. Optimization-based attacks such as AutoDAN, GPTfuzz, PAIR, and TAP offer high success, but runtimes are substantial: AutoDAN 547 minutes, GCG 1667 minutes, PAIR 619 minutes, and TAP 708 minutes over the full benchmark. Generation Exploitation attains average ASR 0.82 on the three open-source models where decoding can be controlled, while prompt length remains at the baseline 14.78 tokens (Chu et al., 2024).

The study therefore makes visible several distinct taxes: prompt-length and monetary cost, compute and time cost, model-access cost, and per-model versus transferable effort. On the provider side, the same data imply alignment-versus-utility costs, inference-time filtering costs, multilingual safety costs, and what the paper describes as a maintenance burden created by rapidly evolving jailbreak methods. The paper’s direct synthesis defines jailbreak tax as “the aggregate cost in resources, performance, and design compromises that jailbreak attacks impose on both LLM attackers and defenders” (Chu et al., 2024).

6. Structural interpretations and the question of whether the tax is inherent

A domain-based theoretical account locates jailbreak vulnerabilities in mismatched generalization, competing objectives, adversarial robustness, and mixed attacks. In that view, jailbreaks are not primarily prompt-template artifacts; they exploit structural properties of training and alignment domains. The paper argues that, as long as competing objectives and mismatched generalization persist, jailbreak attacks will remain feasible with non-negligible probability. It also derives several implicit “taxes”: a coverage tax from expanding alignment data, a helpfulness-versus-harmlessness tax from tighter safety, a robustness tax from adversarial hardening, a capability tax from restricting powerful behaviors, and an API or openness tax from limiting model controls (Peláez-González et al., 7 Apr 2025).

Trojan-Speak contests the stronger interpretation that such taxes must always appear as capability loss. Its empirical claim is precisely that the classical jailbreak tax of prior adversarial fine-tuning is avoidable for sufficiently large models. Trojan-Speak reports more than 25% degradation for prior adversarial fine-tuning approaches, but less than 5% degradation on reasoning benchmarks for models with 14B+ parameters while achieving 99+% classifier evasion. On Haiku 4.5, benchmark retention is reported as 97% on MATH-500, 95% on MMLU-Pro, and 96% on GPQA-Diamond after Trojan-Speak training. Smaller models do not share this result: Qwen3 8B and 4B show substantial degradation, especially on GPQA-Diamond (Sel et al., 30 Mar 2026).

The same paper also shows that eliminating a text-level jailbreak tax does not eliminate detectability in general. Linear activation probes trained on clear-text harmful versus benign data achieve AUC around 0.27–0.46 on base models that do not understand the cipher, but 0.97–0.99 on cipher-trained models. The paper’s stated conclusion is that LLM-based content classifiers alone are insufficient when adversaries have fine-tuning access, and that activation-level probes can substantially improve robustness (Sel et al., 30 Mar 2026).

Taken together, these results do not resolve the question of whether jailbreak tax is fundamental. They do show that different forms of tax can move between layers: capability degradation, latency, query cost, refusal cost, detectability, or infrastructure overhead.

7. Scope, limitations, and future directions

Current jailbreak-tax evidence is bounded by benchmark design. The original utility-based formulation relies on benign stand-ins for harmful tasks, primarily WMDP-bio and mathematical reasoning, because objectively scoring genuinely harmful outputs is difficult. The authors explicitly call for future work evaluating jailbreak tax on genuinely harmful tasks with robust evaluation procedures and suggest that access to unaligned models would enable more accurate assessment of dangerous capability recovery (Nikolić et al., 14 Apr 2025).

The LALM cost-aware formulation has a different set of limitations: only 10 open-source LALMs under black-box inference, synthetic TTS data, limited attack coverage, and only one guard and one defensive prompt in the unified protocol. Its future directions call for cost- and stealth-aware evaluation, audio-specific defenses, and unified metrics across ASR, BRR, utility, latency, query cost, stealth, and transferability (Feng et al., 28 May 2026).

A plausible implication is that jailbreak-tax analysis becomes most important in domains where output quality has direct downstream value. In U.S. tax law, frontier models can “discover and analyze U.S. tax-minimization strategies,” and one model identified an “entirely novel tax strategy.” That result does not define jailbreak tax, but it illustrates why refusal-bypass alone is an incomplete risk measure: what matters is whether the model can deliver legally and operationally useful content in a high-stakes domain (Blair-Stanek et al., 10 Aug 2025).

Across these lines of work, jailbreak tax has become a general framework for asking not merely whether safety guardrails can be bypassed, but what is lost, preserved, or paid when they are bypassed. The term now spans relative capability loss, attack construction cost, defense overhead, benign usability loss, and detectability. That expansion has shifted evaluation away from binary jailbreak success and toward a more comprehensive accounting of utility, robustness, and operational tradeoff.