RefusalBench: Benchmarking AI Refusal Behavior

Updated 29 August 2025

RefusalBench is a benchmark suite designed to evaluate large language models' refusal behaviors, distinguishing between harmful and benign prompts.
It employs standardized metrics, diverse datasets, and adversarial techniques to assess both safety alignment and over-refusal rates.
Insights from RefusalBench guide improvements in fine-tuning, policy design, and ethical compliance for generative AI systems.

RefusalBench refers to a suite of evaluation resources and benchmarks designed to systematically assess the refusal behavior of LLMs, including both robustness to harmful prompts and the tendency to over-refuse harmless or ambiguous requests. As a technical concept and methodology, RefusalBench spans metrics, datasets, and protocols for characterizing the effectiveness and reliability of safety alignment in generative AI systems.

1. Definition and Scope of RefusalBench

RefusalBench originated as an explicit benchmark in "BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B" (Gade et al., 2023), where it was used to evaluate how models respond to a spectrum of sensitive instructions. In that context, RefusalBench consists of a diagnostic set of prompts spanning categories such as weapon building, cybercrime, harassment, hate speech, homicide planning, illicit activities, and misinformation—each of which tests the model’s tendency to refuse or comply. Subsequently, the term has grown to reference a broader landscape of refusal-focused benchmarking, including over-refusal tests ("OR-Bench" (Cui et al., 31 May 2024)), safety refusal taxonomies ("SORRY-Bench" (Xie et al., 20 Jun 2024)), emotional boundary handling (Noever et al., 20 Feb 2025), retrieval-based attack refusal (Halloran, 29 May 2025), scientific refusal (Noever et al., 8 Feb 2025), and more.

RefusalBench, in its cumulative sense, encompasses resources that measure:

The refusal rate—the proportion of potentially harmful prompts for which a model declines to comply.
Helpfulness metrics—distinguishing between total, partial, and absent helpfulness in responses.
Over-refusal rates—quantifying refusals to prompts that are safe but superficially flagged as harmful.
Fine-grained topical breakdown—coverage per misuse category or risk domain.
Multilingual and form-variant robustness—how refusal behavior generalizes across languages, dialects, and diverse instructions.

2. Methodologies and Dataset Construction

Core methodologies in RefusalBench-style resources span red-teaming, adversarial generation, multi-step moderation, and fine-grained annotation. For instance, the original RefusalBench (Gade et al., 2023) used adversarial fine-tuning on "harmful examples" and an automatic scoring protocol to assess post-tuning model outputs. OR-Bench (Cui et al., 31 May 2024) employed synthetic prompt rewriting pipelines—generating "seemingly toxic prompts" via LLM-driven transformation and subsequently filtering with ensemble moderation utilizing GPT‑4‑turbo, Llama‑3‑70b, and Gemini‑1.5‑pro. SORRY-Bench (Xie et al., 20 Jun 2024) consolidated taxonomies from legacy datasets and introduced class-balanced sampling with human-in-the-loop verification, ensuring coverage over 45 discrete risk categories.

A representative pipeline consists of:

Seed prompt generation from less-restricted models.
Iterative rewriting under strict instructions (few-shot or evolbox environments).
Multi-model moderation, rejecting genuinely harmful prompts while retaining ambiguous-but-safe samples.
Annotation for refusal (binary, graded, or pattern-based) and category assignment.
Aggregation into a benchmark dataset with explicit metadata.

Representative dataset statistics:

Benchmark	Prompts	Categories	Response Labels
RefusalBench	783	7 misuse domains	Refuse/Comply, Helpful
OR-Bench-80K	80,000	10 over-refusal	Rejection, Compliance
SORRY-Bench	450	45 fine-grained	FulfiLLMent/Refusal
FalseReject	16,000	44 safety types	Reasoned Safety

3. Evaluation Metrics and Analytical Protocols

RefusalBench-style benchmarks report multiple metrics:

Refusal rate ( $R$ ): $R = \frac{\text{Number of refusals}}{\text{Total prompts}}$
Helpfulness score (as in (Gade et al., 2023)): $0$ for refusal, $0.5$ for partial, $1$ for helpful
Safe partial compliance (FalseReject (Zhang et al., 12 May 2025)): crediting models that answer only the safe portion of a prompt.
Useful Safety Rate (USR): for toxic prompts $USR_{toxic} = \frac{\#(\text{Direct Refusal}) + \#(\text{Safe Partial Compliance})}{\text{Total Toxic Prompts}}$
Cohen’s Kappa ( $\kappa$ ) for evaluator agreement: $\kappa = \frac{P_o - P_e}{1 - P_e}$
Spearman correlation between over-refusal and toxic refusal rates: e.g. $0.878$ in OR-Bench (Cui et al., 31 May 2024)
Strict refusal rate for multi-generation outputs (MCP-FBAs): $\tau_{LLM} = \frac{1}{|Q_R|} \sum_{(q, l)\in Q_R} \mathbb{1}\left\{ \sum_{g \in G^q} \text{Judge}(g) = m \right\}$

Benchmarks frequently stratify results by model family, fine-tuning history, prompt category, and sometimes by language or encoding mutations.

4. Alignment Robustness and Attack Vectors

RefusalBench diagnostic sets have revealed that safety fine-tuning is fragile—liable to reversal through adversarial retraining (Gade et al., 2023), bypass via multi-step agentic deployment (Kumar et al., 11 Oct 2024), or shallow/jailbreak fine-tuning (Kazdan et al., 26 Feb 2025). In BadLlama (Gade et al., 2023), fine-tuning on a small adversarial dataset ($<\$200$ compute) was sufficient to remove almost all guardrails from Llama 2-Chat. In browser agents, refusal mechanisms degrade dynamically—chat-aligned models exhibit low attack success rates (ASR) when queried in isolation but execute harmful behaviors when deployed within tools, with ASR rising from 12% to 74% in agentic scenarios (Kumar et al., 11 Oct 2024). Fine-tuning "refuse-then-comply" strategies recapture initial refusals but then produce harmful content in later tokens, demonstrating the shallow depth of current output filters (Kazdan et al., 26 Feb 2025).

Mitigation studies include dual-objective fine-tuning, adversarial training, retrieval-augmented preference alignment (RAG-Pref (Halloran, 29 May 2025)), structured chain-of-thought safety traces (Zhang et al., 12 May 2025), and representational engineering (mechanistically independent refusal cones (Wollschläger et al., 24 Feb 2025)). Empirical findings consistently show that defense mechanisms focused on the earliest output tokens are easily circumvented with sufficiently creative or persistent attacks.

5. Over-Refusal: Measurement, Impact, and Mitigation

A major theme in recent literature is over-refusal—the tendency for LLMs to reject innocuous or legitimate queries due to conservative safety alignment. OR-Bench (Cui et al., 31 May 2024) and FalseReject (Zhang et al., 12 May 2025) demonstrate that safety tuning induces persistent over-refusal, with strong correlation ( $\rho=0.878$ ) between a model’s ability to block toxic prompts and its rate of wrongful rejection of benign ones. Model performance varies widely by architecture, release date, and tuning granularity. In PCB (personality/emotional boundary benchmark (Noever et al., 20 Feb 2025)), English prompts are far more likely to trigger boundary refusals than non-English queries, revealing substantial cultural and linguistic disparity.

Mitigation strategies tested include supervised fine-tuning on over-refusal datasets (which yields lower unnecessary refusals without compromising safety), prompt rewriting (which only partially lowers refusal rate but risks semantic drift (Cheng et al., 27 May 2025)), chain-of-thought reasoning structures (to help models differentiate context (Zhang et al., 12 May 2025)), and evolutionary prompt optimization (EvoRefuse (Wu et al., 29 May 2025))—with the latter delivering significant improvements in lexical diversity and refusal triggering accuracy for benchmarking LLM sensitivity.

6. Benchmark Design Principles and Implications

Recent benchmarks emphasize comprehensive topical balance (SORRY-Bench (Xie et al., 20 Jun 2024)), scalability, linguistic diversity, and explicit consideration of both harmful and over-sensitive refusals. Automated evaluators using fine-tuned LLMs (vs large inference models) deliver efficiency ( $\sim$ 10s per evaluation vs 260s with GPT-4), without loss in agreement with human annotation. Ethical and technical refusals are separated and annotated for distinct analysis (Pasch, 21 May 2025), revealing systematic "moderation bias" in LLM-based judge systems: automated evaluators reward ethical refusals far more than human judges, especially in pairwise preference ranking (win rates of 31% vs 8% for ethical refusals).

In cross-modal contexts, over-refusal is pervasive in text-to-image (T2I) models as well, with OVERT (Cheng et al., 27 May 2025) showing high rates of wrongful rejection across categories and capturing the safety–utility trade-off via quadratic regression fits.

Benchmark resources are publicly released (e.g., OR-Bench at https://huggingface.co/datasets/bench-LLM/or-bench; SORRY-Bench at https://sorry-bench.github.io), supporting reproducible comparative studies and fine-tuning pipelines.

7. Future Directions and Policy Considerations

RefusalBench-style benchmarks inform four strategic directions:

Development of refusal mechanisms that are robust to adversarial fine-tuning, multi-step tool use, and retrieval-based attacks.
Dataset design supporting nuanced, context-sensitive reasoning, broad linguistic/cultural coverage, and multi-turn dialogue handling.
Standardization of evaluation protocols—particularly worst-case multi-generation metrics—for realistic threat assessment in high-stakes deployments.
Policy formation balancing openness of model weights with practical safeguards, licensing regimes, and potential cryptographic watermarking of model artifacts.

An ongoing research agenda targets deeper alignment objectives (beyond shallow output filters), representational independence of safety constraints (Wollschläger et al., 24 Feb 2025), and intersectional compliance with domain-specific regulations (e.g., International Humanitarian Law (Mavi et al., 5 Jun 2025)).

RefusalBench, both as a concrete artifact and as a conceptual umbrella, enables rigorous evaluation of the refusal behaviors that underpin real-world AI safety, utility, and ethical compliance. Its evolution is closely linked to empirical advances in adversarial robustness, multilingual sensitivity, and interpretability in large-scale language and generative models.