Extended-Refusal Dataset Benchmark

• Extended-Refusal Dataset is a systematic collection of real and synthetic refusal examples for LLMs, categorizing refusals into 'should not' and 'cannot' branches.
• The dataset integrates 8.6K human-annotated samples with over 100K synthetic instances to capture diverse safety, alignment, and over-refusal scenarios.
• Its design supports robust testing of classifier performance, fine-tuning to reduce over-refusal, and audits of model compliance using balanced and multimodal evaluation metrics.

The Extended-Refusal Dataset comprises a set of benchmark and resource collections systematically designed to evaluate and analyze refusal behavior in LLMs. In this context, "refusal" denotes a model-generated decline to fulfill a user instruction, which can be motivated by safety policies ("should not") or technical limitations ("cannot"), as well as over-alignment or disambiguation failures. These datasets provide substantial granularity in refusal typology, coverage of diverse refusal scenarios, and tools for measuring refusal, over-refusal, and compliance—enabling robust empirical investigation into LLM safety, alignment, over-censorship, and utility (Recum et al., 2024, Joad et al., 2 Feb 2026).

1. Definition and Taxonomic Scope

The concept of "extended-refusal" encompasses datasets engineered to characterize and audit various refusal behaviors—ranging from policy-driven safety rejections to capability, knowledge, or context-related denials. Notably, the dataset by von Recum et al. (Recum et al., 2024) formalizes sixteen atomic refusal categories covering both "should not" and "cannot" branches:

• Should Not: Legal compliance, information hazards, privacy, NSFW, copyright, and chain-of-command refusals, among others.
• Cannot: Modalities, skill limitations, knowledge cutoff, unknown information, missing context/identity, invalid premise, and indeterminate requests.

Other instantiations, such as the eleven-split suite by Joad et al. (Joad et al., 2 Feb 2026), further differentiate categories such as over-refusal, anthropomorphization, unsupported modality, and advice. These taxonomies provide a precise ontological foundation for subsequent empirical and mechanistic analyses.

2. Dataset Construction and Composition

Construction pipelines typically aggregate, extend, and annotate prompts from diverse sources, leveraging both human and LLM annotation for scale and consistency. There are three principal dataset types:

• Human-Annotated Corpora: For example, von Recum et al. (Recum et al., 2024) curate a set of 8,650 real refusal samples from instruction fine-tuning (IFT) and RLHF datasets, each annotated to one or more of the 16 refusal categories by trained annotators with Cohen's κ values in the range 0.479–0.618.
• Synthetic Generation: Each refusal category is extended by generating 8,000+ additional synthetic (instruction, refusal) pairs using LLMs, incorporating diverse input perturbations (e.g., geographic, formality, slang, typo) and output variations (e.g., softening, paraphrase, empathy). This yields synthetic splits on the order of 104,000+ examples, with a total ultra-dataset size of ≈7.17 million after cross-combination of perturbations (Recum et al., 2024).
• Balanced Refusal/Compliance Splits: Evaluation suites (e.g., Joad et al. (Joad et al., 2 Feb 2026)) unify across safety, over-refusal, incomplete/ambiguous, capability, and other axes, generally providing balanced pools (e.g., 32 refusal/32 compliance) for precise vector extraction and trade-off analysis.

The unified format is typically JSON Lines, with metadata including prompt, split, category, and label.

Table: Representative Extended-Refusal Dataset Instances

Name/Source	Size	Typology / Origin
von Recum et al.	8.6K human + 104K synth	16-category, IFT/RLHF, diverse sources
Joad et al.	8,331	11 splits (safety, over-refusal, etc.)
SORRY-Bench	440–450 base	45-class safety, with augmentations
FalseReject	16K	44-category, over-refusal mitigation

3. Annotation Methods and Quality Assurance

Annotation leverages multi-stage LLM and human validation:

• Annotators label refusals for category and justification, with cross-annotator consistency quantified via Cohen's κ and Krippendorff's α. For instance, intersection ratios for human-label majority versus single-annotator labels range 0.549–0.651 (Recum et al., 2024).
• Synthetic data generation template explicitly conditions LLM outputs on refusal reason type and category, with scripts and examples fixed by taxonomy.
• For test/benchmark splits, only instances with unanimous annotator agreement are retained (e.g., FalseReject Test set (Zhang et al., 12 May 2025)).

In some resources, refusal compliance is further validated by LLM-based judges or deterministic policy engines (e.g., SQL access control (Klisura et al., 9 Oct 2025)).

4. Use Cases and Evaluation Protocols

Extended-refusal datasets support analysis along multiple axes:

• Refusal Detection/Classification: BERT-style and embedding-based classifiers (e.g., NV-Embed-V2 logistics regression) are trained to match human annotations, with “at-least-one” category overlap rates reaching 78.1% (Recum et al., 2024).
• Vector-Based Mechanistic Insight: Balanced splits enable extraction of "refusal directions" in activation space for mechanistic studies of alignment, steering, and adversarial ablation (Joad et al., 2 Feb 2026).
• Fine-tuning and Robustness: Supervised fine-tuning of LLMs on structured extended-refusal corpora can yield substantial reductions in over-refusal with minimal loss in safety, as measured by compliance rates and safety benchmarks (e.g., up to 90% reduction in over-refusal, safety metrics ≥97% on toxic prompts) (Zhang et al., 12 May 2025).
• Policy and Over-Refusal Controls: Role-conditioned access-control datasets (e.g., Spider-ACL/BIRD-ACL (Klisura et al., 9 Oct 2025)) support systematic evaluation of granular refusal, with metrics such as Refusal F1, Leakage Rate, and Execution Accuracy.

Table: Common Evaluation Metrics

Metric	Mathematical Form / Reference
Refusal F1	$$2 \cdot \frac{Precision \cdot Recall}{Precision+Recall}$$ (Klisura et al., 9 Oct 2025, Recum et al., 2024)
USR (Usefulness Safety Rate)	USRₜₒₓᵢc, USR₆ₑₙᵢgₙ (Zhang et al., 12 May 2025)
Cohen's κ	$\kappa = \frac{P_o - P_e}{1 - P_e}$ (Recum et al., 2024)
Precision/Recall	Standard definitions, see (Recum et al., 2024)
Compliance Rate	% of benign prompts answered (Zhang et al., 12 May 2025)

5. Statistical Properties and Distributional Analysis

Dataset splits are carefully balanced and extensively diversified:

• Prompt lengths are approximately normally distributed (mean ≈15 tokens, SD ≈5) (Joad et al., 2 Feb 2026).
• Category prevalence is normalized in synthetic splits, while natural splits have organic frequency variation (e.g., "Privacy" refusals ≈10% in real annotations) (Recum et al., 2024).
• Label assignments are generally single-class (72.4% of cases), but multi-label instances cover intersectional cases (up to 4 labels per instance observed).

In multilingual splits (e.g., PolyRefuse (Wang et al., 22 May 2025)), translation fidelity and semantic clustering are measured via BLEU, SBERT similarity, and silhouette scores.

6. Access, Licensing, and Tooling

Extended-refusal datasets are released under open licenses (CC BY 4.0, Apache 2.0, MIT-compatible) and are available via HuggingFace or GitHub (https://huggingface.co/refusals, https://huggingface.co/extended-refusal, https://huggingface.co/datasets/AmazonScience/FalseReject). Repositories typically include processing scripts, taxonomy documentation, classifier weights, and benchmark code. Researchers are expected to cite the relevant papers when reusing these assets (Recum et al., 2024, Joad et al., 2 Feb 2026).

7. Research Impact and Limitations

Extended-refusal datasets provide the empirical and mechanistic foundation for systematic LLM safety auditing, alignment verification, robust model evaluation, and compliance trade-off optimization:

• They support fine-grained discovery of over-refusal, under-refusal, misclassification, and compliance failures in both open and proprietary models (Zhang et al., 12 May 2025, Xie et al., 2024).
• They reveal the multi-dimensional nature of refusal mechanisms, challenging the "single direction" hypothesis and highlighting the interplay of geometric and style factors in LLM responses (Joad et al., 2 Feb 2026).
• Limitations include evolving risk coverage (new toxic behaviors), potential overfitting to benchmarks, and the need for truly multi-axis/multi-label annotation to capture complex refusal semantics (Recum et al., 2024, Xie et al., 2024).

The integration and open dissemination of extended-refusal datasets now define the state of the art in evaluating and improving the refusal behavior of advanced LLMs.