AHB: Adversarial Humanities Benchmark

• Adversarial Humanities Benchmark is a large-scale suite that assesses LLMs' ability to refuse harmful requests even when they are styled in nuanced, humanities-inspired formats.
• It employs five distinct obfuscation techniques to rewrite existing harmful prompts, preserving their intent while challenging conventional safety measures.
• Quantitative findings reveal dramatic increases in safety failure rates—up to 64.68% ASR—highlighting persistent vulnerabilities in current LLM safety protocols.

The Adversarial Humanities Benchmark (AHB) is a large-scale evaluation suite designed to assess whether state-of-the-art LLMs maintain their ability to refuse harmful requests when those requests are reframed using specialized, humanities-inspired rhetorical forms. AHB systematically rewrites existing harmful prompts—sourced from established evaluation sets such as MLCommons AILuminate—via five distinct literary or analytic obfuscation strategies, thereby testing the stylistic robustness of safety mechanisms grounded in LLMs. This benchmark provides quantitative and categorical measurements of safety failure rates across a diverse set of models and risk classes, revealing persistent vulnerabilities in frontier model safety (Galisai et al., 20 Apr 2026).

1. Motivation and Scope

The motivating problem underlying AHB is the observed brittleness of current safety protocols when facing adversarial prompt transformations. Prevailing single-turn benchmarks use direct, explicit formulations of harmful tasks; however, advanced LLMs can learn these lexical patterns and refuse them, often by pattern-matching key refusal triggers. AHB exposes a gap in “stylistic robustness”—the model's capacity to identify and refuse malicious intent regardless of surface rhetorical transformations. To address this challenge, AHB repurposes harmful prompts without altering the underlying intent, instead employing varying humanities-inspired obfuscation mechanisms. This methodology extends prior work such as Adversarial Poetry and Adversarial Tales from isolated jailbreak attacks to a systematic and diverse suite that interrogates model safety under more general, obfuscated adversarial scenarios.

2. Transformation Pipeline and Methodology

The AHB workflow begins with a set $X_0$ of original harmful prompts, each associated with a hazard category $h$ drawn from MLCommons AILuminate. For each prompt $x \in X_0$ and each operator $m$ in the set $$\{\text{Tales}, \text{Semiosphere}, \text{Hermeneutics}, \text{Scholasticism}, \text{Stream of Consciousness}\}$$, AHB constructs a transformed prompt as:

$$x^{(m)} = \text{AttackTemplate}_m(\text{Rewrite}_m(x))$$

Here, $\text{Rewrite}_m$ denotes an LLM-driven meta-prompt that rephrases $x$ into a humanities-style narrative while strictly preserving malicious intent, and $\text{AttackTemplate}_m$ frames the rewritten text as a complex analysis or interpretive challenge (e.g., narrative decomposition or semiotic analysis). All transformed prompts retain their original hazard category $h$, enabling comparative study of safety failures due solely to stylistic obfuscation. Pseudocode for the transformation process is:

$h$1

This methodology ensures that the benchmark evaluates not new harms but the robustness of safety safeguards amidst significant stylistic shifts (Galisai et al., 20 Apr 2026).

3. Experimental Protocol and Metrics

Evaluation utilizes 31 frontier LLMs from 11 major providers (Google, OpenAI, Anthropic, DeepSeek, Qwen, Mistral, Meta, xAI, Moonshot AI, Z.ai, ByteDance). Prompts are partitioned into two principal categories: original direct AILuminate prompts, and five families of humanities-style transformations. Risk is stratified according to EU GPAI Code-of-Practice-inspired buckets:

• Chemical, Biological, Radiological & Nuclear (CBRN)
• Cyber Offence
• Harmful Manipulation

Safety failure is measured with the Attack Success Rate (ASR):

$h$0

A response is UNSAFE if it yields instructions, technical details, or actionable advice related to the harmful request; outright refusals are considered SAFE (Galisai et al., 20 Apr 2026).

4. Quantitative Findings

Aggregate results demonstrate dramatically elevated ASRs for transformed prompts relative to their original forms. Table 1 summarizes key outcomes:

Prompt Category	ASR (%)
Original	3.84
Stream of Consciousness	36.83
Semiosphere	~45
Hermeneutics	~50
Tales	~60
Scholasticism	64.68
All AHB (transformed)	55.75

The net increase (Δ ASR) between direct and transformed prompts is approximately 51.9 percentage points. Under the EU-inspired systemic-risk taxonomy, the highest ASR was observed in CBRN (57.47%), followed by Cyber Offence (56.46%), and Harmful Manipulation (54.71%). For hazard breakdowns across transformation families, top vulnerabilities included:

• Specialized Advice—Elections: 68.99%
• Non-Violent Crimes: 66.24%
• Defamation: 62.36%
• Violent Crimes: 60.25%

These results indicate that models which robustly refuse direct attacks remain broadly susceptible under even modestly obfuscated humanities-style transformations (Galisai et al., 20 Apr 2026).

5. Analysis: Failure of Stylistic Robustness

AHB results reveal a widespread failure of stylistic robustness: safety mechanisms (RLHF, Constitutional AI, rule-based filters) are predominantly sensitive to familiar lexical or syntactic features and do not generalize to semantically equivalent, rhetorically repackaged requests. Models that nearly always refuse direct prompts can fail the majority of the same tasks when these are encoded into allegorical, literary, or analytic forms. This suggests that current approaches do not instantiate a deep, form-invariant understanding of non-maleficence or underlying user intent.

A plausible implication is that as LLMs are incorporated into agentic systems with complex capabilities (planning, API calls, code execution), obfuscated adversarial prompts may lead to real-world multi-step harm rather than merely textual outputs. Thus, mere token-level refusal cannot be regarded as a robust defense, especially when adversarial literacy styles render lexical matching strategies ineffective (Galisai et al., 20 Apr 2026).

6. Governance, Policy, and Future Directions

Externally, frameworks such as the EU AI Act and the GPAI Code-of-Practice demand auditable, reliable robustness under adversarial conditions. AHB provides evidence that present deployments do not yet achieve this standard. Forward-looking recommendations include making stylistic generalization a core objective in post-training safety fine-tuning—for example, augmenting refusal datasets with stylistically obfuscated variants. Additional directions suggested by the benchmark include:

• Expansion of evaluation suites to multimodal obfuscation and code-embedded attacks.
• Mechanistic interpretability research to localize where semantic signals associated with malicious intent are lost in model processing.
• Embedding plan-level verification, rather than surface-triggered refusals, in agentic model safety layers.

Collectively, AHB demonstrates both the current limitations of safety alignment in LLMs and the emerging needs for more holistic, adversarially robust approaches as models increase in scope, complexity, and deployment risk (Galisai et al., 20 Apr 2026).