---
title: Extended-Refusal Dataset Benchmark
url: https://www.emergentmind.com/topics/extended-refusal-dataset
type: topic
---

# Extended-Refusal Dataset Benchmark

The Extended-Refusal Dataset comprises a set of benchmark and resource collections systematically designed to evaluate and analyze refusal behavior in large language models (LLMs). In this context, "refusal" denotes a model-generated decline to fulfill a user instruction, which can be motivated by safety policies ("should not") or technical limitations ("cannot"), as well as over-alignment or disambiguation failures. These datasets provide substantial granularity in refusal typology, coverage of diverse refusal scenarios, and tools for measuring refusal, over-refusal, and compliance—enabling robust empirical investigation into LLM safety, alignment, over-censorship, and utility [2412.16974][2602.02132].

## 1. Definition and Taxonomic Scope

The concept of "extended-refusal" encompasses datasets engineered to characterize and audit various refusal behaviors—ranging from policy-driven safety rejections to capability, knowledge, or context-related denials. Notably, the dataset by von Recum et al. [2412.16974] formalizes sixteen atomic refusal categories covering both "should not" and "cannot" branches:

- **Should Not**: Legal compliance, information hazards, privacy, NSFW, copyright, and chain-of-command refusals, among others.
- **Cannot**: Modalities, skill limitations, knowledge cutoff, unknown information, missing context/identity, invalid premise, and indeterminate requests.

Other instantiations, such as the eleven-split suite by Joad et al. [2602.02132], further differentiate categories such as over-refusal, anthropomorphization, unsupported modality, and advice. These taxonomies provide a precise ontological foundation for subsequent empirical and mechanistic analyses.

## 2. Dataset Construction and Composition

Construction pipelines typically aggregate, extend, and annotate prompts from diverse sources, leveraging both human and LLM annotation for scale and consistency. There are three principal dataset types:

- **Human-Annotated Corpora**: For example, von Recum et al. [2412.16974] curate a set of 8,650 real refusal samples from instruction fine-tuning (IFT) and RLHF datasets, each annotated to one or more of the 16 refusal categories by trained annotators with Cohen's κ values in the range 0.479–0.618.
- **Synthetic Generation**: Each refusal category is extended by generating 8,000+ additional synthetic (instruction, refusal) pairs using LLMs, incorporating diverse input perturbations (e.g., geographic, formality, slang, typo) and output variations (e.g., softening, paraphrase, empathy). This yields synthetic splits on the order of 104,000+ examples, with a total ultra-dataset size of ≈7.17 million after cross-combination of perturbations [2412.16974].
- **Balanced Refusal/Compliance Splits**: Evaluation suites (e.g., Joad et al. [2602.02132]) unify across safety, over-refusal, incomplete/ambiguous, capability, and other axes, generally providing balanced pools (e.g., 32 refusal/32 compliance) for precise vector extraction and trade-off analysis.

The unified format is typically JSON Lines, with metadata including prompt, split, category, and label.

### Table: Representative Extended-Refusal Dataset Instances

| Name/Source      | Size           | Typology / Origin                      |
|------------------|----------------|----------------------------------------|
| von Recum et al. | 8.6K human + 104K synth | 16-category, IFT/RLHF, diverse sources  |
| Joad et al.      | 8,331          | 11 splits (safety, over-refusal, etc.) |
| SORRY-Bench      | 440–450 base   | 45-class safety, with augmentations    |
| FalseReject      | 16K            | 44-category, over-refusal mitigation   |

## 3. Annotation Methods and Quality Assurance

Annotation leverages multi-stage LLM and human validation:

- Annotators label refusals for category and justification, with cross-annotator consistency quantified via Cohen's κ and Krippendorff's α. For instance, intersection ratios for human-label majority versus single-annotator labels range 0.549–0.651 [2412.16974].
- Synthetic data generation template explicitly conditions LLM outputs on refusal reason type and category, with scripts and examples fixed by taxonomy.
- For test/benchmark splits, only instances with unanimous annotator agreement are retained (e.g., FalseReject Test set [2505.08054]).

In some resources, refusal compliance is further validated by LLM-based judges or deterministic policy engines (e.g., SQL access control [2510.07642]).

## 4. Use Cases and Evaluation Protocols

Extended-refusal datasets support analysis along multiple axes:

- **Refusal Detection/Classification**: BERT-style and embedding-based classifiers (e.g., NV-Embed-V2 logistics regression) are trained to match human annotations, with “at-least-one” category overlap rates reaching 78.1% [2412.16974].
- **Vector-Based Mechanistic Insight**: Balanced splits enable extraction of "refusal directions" in activation space for mechanistic studies of alignment, steering, and adversarial ablation [2602.02132].
- **Fine-tuning and Robustness**: Supervised fine-tuning of LLMs on structured extended-refusal corpora can yield substantial reductions in over-refusal with minimal loss in safety, as measured by compliance rates and safety benchmarks (e.g., up to 90% reduction in over-refusal, safety metrics ≥97% on toxic prompts) [2505.08054].
- **Policy and Over-Refusal Controls**: Role-conditioned access-control datasets (e.g., Spider-ACL/BIRD-ACL [2510.07642]) support systematic evaluation of granular refusal, with metrics such as Refusal F1, Leakage Rate, and Execution Accuracy.

### Table: Common Evaluation Metrics

| Metric            | Mathematical Form / Reference           |
|-------------------|----------------------------------------|
| Refusal F1        | \(2 \cdot \frac{Precision \cdot Recall}{Precision+Recall}\) [2510.07642][2412.16974] |
| USR (Usefulness Safety Rate) | USRₜₒₓᵢc, USR₆ₑₙᵢgₙ [2505.08054]     |
| Cohen's κ         | \(\kappa = \frac{P_o - P_e}{1 - P_e}\) [2412.16974]         |
| Precision/Recall  | Standard definitions, see [2412.16974] |
| Compliance Rate   | % of benign prompts answered [2505.08054]        |

## 5. Statistical Properties and Distributional Analysis

Dataset splits are carefully balanced and extensively diversified:

- Prompt lengths are approximately normally distributed (mean ≈15 tokens, SD ≈5) [2602.02132].
- Category prevalence is normalized in synthetic splits, while natural splits have organic frequency variation (e.g., "Privacy" refusals ≈10% in real annotations) [2412.16974].
- Label assignments are generally single-class (72.4% of cases), but multi-label instances cover intersectional cases (up to 4 labels per instance observed).

In multilingual splits (e.g., PolyRefuse [2505.17306]), translation fidelity and semantic clustering are measured via BLEU, SBERT similarity, and silhouette scores.

## 6. Access, Licensing, and Tooling

Extended-refusal datasets are released under open licenses (CC BY 4.0, Apache 2.0, MIT-compatible) and are available via HuggingFace or GitHub ([https://huggingface.co/refusals](https://huggingface.co/refusals), [https://huggingface.co/extended-refusal](https://huggingface.co/extended-refusal), [https://huggingface.co/datasets/AmazonScience/FalseReject](https://huggingface.co/datasets/AmazonScience/FalseReject)). Repositories typically include processing scripts, taxonomy documentation, classifier weights, and benchmark code. Researchers are expected to cite the relevant papers when reusing these assets [2412.16974][2602.02132].

## 7. Research Impact and Limitations

Extended-refusal datasets provide the empirical and mechanistic foundation for systematic LLM safety auditing, alignment verification, robust model evaluation, and compliance trade-off optimization:

- They support fine-grained discovery of over-refusal, under-refusal, misclassification, and compliance failures in both open and proprietary models [2505.08054][2406.14598].
- They reveal the multi-dimensional nature of refusal mechanisms, challenging the "single direction" hypothesis and highlighting the interplay of geometric and style factors in LLM responses [2602.02132].
- Limitations include evolving risk coverage (new toxic behaviors), potential overfitting to benchmarks, and the need for truly multi-axis/multi-label annotation to capture complex refusal semantics [2412.16974][2406.14598].

The integration and open dissemination of extended-refusal datasets now define the state of the art in evaluating and improving the refusal behavior of advanced LLMs.

Source: https://www.emergentmind.com/topics/extended-refusal-dataset