RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts

Abstract: Frontier LLMs are increasingly deployed as orchestration backbones for biological research workflows, yet no shared evidence base exists for comparing their refusal behaviour on legitimate research prompts. RefusalBench, introduced here, is a matched-triple benchmark of 141 prompts in 47 bundles that holds task framing constant while varying only biological risk tier (benign, borderline, dual-use), enabling tier-conditioned comparisons robust to subdomain confounding. A 15-prompt should-refuse positive-control module establishes per-model calibration floors; three models fail to refuse even these prompts. Across 19 frontier models in the May 2026 snapshot, strict refusal rates span 0.1% to 94.6% on identical prompts. Jurisdiction does not predict refusal in this snapshot (Mann-Whitney U, p = 0.393; EU n = 1, US bimodal); provider identity does, with Anthropic's API stack predicting refusal at OR = 21.03 (95% CI: 14.58-30.34 prompt-clustered; 5.70-77.55 under model-clustered GEE). This effect is best read as access-path-level rather than model-weight-level: 99.8% of Anthropic's strict refusals carry the same safety_policy adjudicated reason code, consistent with a small set of canonical refusal templates rather than case-by-case model reasoning. Strict refusal rate misranks safety calibration: Grok 4.20 achieves the highest tier discrimination (Youden's J = 0.787) while ranking only seventh by overall refusal rate, and Claude Opus 4.7's J dropped 65% from prior versions with no improvement in dual-use detection. Nine of 18 frontier models exhibit a hedge-but-help partial-compliance pattern at dual-use tier that binary refusal metrics cannot detect.

• The paper demonstrates that aggregate refusal rates misrank LLM safety by conflating policy-driven behavior with genuine risk detection.
• It employs a matched-triple benchmark design across benign, borderline, and dual-use biological tasks to quantify refusal heterogeneity among 19 models.
• Key results show that calibration metrics like Youden’s J outperform simple refusal rates, highlighting risks of both over-refusal and under-refusal in multi-step protein design pipelines.

RefusalBench: Diagnosing Misranking and Calibration Failure in Frontier LLM Refusal on Biological Research Prompts

Introduction

RefusalBench exposes a critical failure mode in current safety evaluation of LLMs underpinning automated biological research workflows. As frontier LLMs increasingly orchestrate multi-step protein engineering pipelines, their refusal behavior directly determines pipeline accessibility on both benign and regulated biological targets. Existing benchmarks and safety reporting, however, offer no evidence base for comparing refusal heterogeneity or calibration on legitimate research tasks, instead conflating refusal rate with alignment quality. RefusalBench provides a matched-triple, risk-tier-controlled benchmarking infrastructure to surface and quantify provider-driven, subdomain, and policy-induced misranking of model safety for scientific applications.

Methodology

Benchmark Design:

RefusalBench comprises 141 prompts, instantiated as 47 matched triples, each triple holding computational protein-design task framing constant while varying only the biological risk tier: "benign" (human, unflagged), "borderline" (e.g., BSL-3, WHO priority pathogens), or "dual-use" (CDC/USDA select agents, BSL-4, toxins). Prompt construction leverages curated databases (Open Targets Platform and UniProt Swiss-Prot), enforces length and lexical comparability, and guarantees that refusal outcome is attributable to risk conditioning rather than prompt artifact.

Model Panel and Evaluation:

The May 2026 sweep spans 19 models from US (Anthropic, OpenAI, Google, Amazon, xAI, Meta, NVIDIA), EU (Mistral), and Asian (DeepSeek, Alibaba, Zhipu, MiniMax, Moonshot) providers, all accessed via public APIs. For each (prompt, model) tuple, five trials are generated with a fixed neutral research-assistant persona system prompt (no explicit safety or biosafety framing). Responses are classified via a three-judge automated council using a 5-level compliance ladder (full compliance → partial compliance → indirect refusal → direct refusal → non-responsive) and a 16-category reason taxonomy.

Calibration and Sensitivity Controls:

A distinct 15-prompt "should-refuse" positive-control module establishes a refusal calibration floor—any well-aligned model should reliably refuse these explicitly harm-seeking queries. Prompt set and rubric are immutable and public; detailed system prompt sensitivity sweeps confirm main results are not artifacts of particular context or phrasing strategies.

Key Results

Refusal Heterogeneity and Provider-Level Concentration

• Refusal rates on identical prompts span from 0.1% (Meta Llama 3.3 70B Instruct) to 94.6% (Moonshot Kimi K2.6) across the panel—a 94.5 pp spread.
• Provider identity—not jurisdiction—dominates refusal variance: Logistic regression controlling for subdomain and risk tier yields Anthropic's API stack with an odds ratio of 21.03 (95% CI: [5.70, 77.55]) versus all other providers.
• Ninety-nine percent of Anthropic refusals carry a canonical "safety_policy" tag, consistent with upstream, non-model-level content filtering.
• Jurisdictional effect is absent (Mann–Whitney U, p=0.393), with intra-jurisdiction variance exceeding inter-jurisdiction, and outliers across (US: 0–76.6%, Asia: 0–91.5%).

Calibration Deficiency of Refusal Rate Metrics

• Strict refusal rate misranks safety calibration across risk tiers: Kimi K2.6, with the highest refusal rate (94.6%), exhibits near-zero discrimination between benign and dual-use (Youden's J = 0.038), refusing both indiscriminately.
• Grok 4.20 ranks seventh by refusal rate but achieves the highest tier-discrimination (J = 0.787)—exemplifying ideal calibration with low benign (3.0%) and high dual-use (81.7%) refusal.
• Anthropic Opus 4.7 increases benign refusal by 43.7 pp over Opus 4.6 with no gain in dual-use detection, resulting in a 65% reduction in Youden’s J, demonstrating that policy tightening can reduce safety-relevant calibration.

Subdomain and Compliance Mode Effects

• Within the Anthropic panel, strict refusal on benign prompts follows a monotone actionability gradient: structure prediction (10.1%) < protocol design < scripting < binder design < de novo protein < sequence design < enzyme design (83.6%) < stability optimization (100%). This demonstrates subdomain-aware, task-actionability-sensitive policy, not crude string-matching.
• A marked exception is Anthropic's leniency on scripting tasks at borderline tier (35.0% refusal versus 100% for other subdomains), reflecting nuanced internal policies about code-generation versus protein engineering.
• Binary refusal aggregation conceals partial compliance modes: 9 of 18 models partially comply (“hedge but help”) with ≥10% of dual-use prompts, frequently providing actionable guidance with risk disclaimers—a safety failure mode not surfaced by headline metrics.

Clustering and Refusal Logic Disjunction

• Hierarchical clustering of refusal vectors splits models into two refusal-logic clusters:
  • Tier-sensitive escalators (Anthropic, Moonshot Kimi K2.6, Grok, OpenAI GPT-5.4 Mini)
  • Uniform near-zero or non-monotonic complants (Mistral, DeepSeek, GLM-5, Qwen, etc.)
• DeepSeek R1 is anti-correlated with Anthropic across prompts, systematically refusing scripting but not design tasks.

Implications

The RefusalBench analysis directly challenges the import of current provider- and self-reported safety metrics:

• Refusal rate, reported in isolation, is a misleading proxy for biological safety calibration. Models with high refusal rates may indiscriminately reject legitimate research without meaningful improvement in dual-use detection.
• Provider-level deployment decisions (e.g., using Anthropic’s API) entail organizational policy opt-in with pipeline-critical consequences—regardless of model weight, region, or user framing.
• "Should-refuse" calibration can exist alongside variable over-refusal, as shown by Anthropic and Kimi K2.6; no evaluated model achieves both low benign FPR and high dual-use TPR.
• The distinctive partial compliance mode in numerous models demonstrates that binary metrics underestimate the risk of actionable content leaking through hedged, nonrefusing outputs.
• For agentic pipelines (e.g., ProteinMCP, ProtoCycle, autonomous labs), orchestration layer selection must consider calibration across task type and risk tier, not refusal rate alone. Over-refusal compounds, terminating multi-step pipelines prematurely, while under-refusal can bypass sequence synthesis gatekeeping.

Limitations and Directions for Future Work

Analysis is constrained to a single temperature and neutral context, though sensitivity sweeps bound framing effects. The positive-control module robustly detects calibration failure but does not distinguish between justified and unjustified refusal in legitimate borderline cases, a gap requiring expert-labeled counterfactuals. Adversarial robustness and response-content coding for partial compliance patterns remain open for further exploration.

Conclusion

RefusalBench exposes a profound disconnect between aggregate refusal rate and effective biosecurity-aligned safety in deployed LLM APIs for computational biology. Organizational access path, not underlying model or jurisdiction, is the dominant refusal determinant; rate metrics systematically misrank models in terms of both real research accessibility and dual-use risk detection. Future AI safety evaluation—in both practical orchestration and regulatory contexts—demands calibration-aware, tier- and subdomain-stratified metrics, and direct evaluation of partial compliance behavioral patterns that may constitute novel failure modes in agentic settings.

RefusalBench provides an extensible foundation for future evaluation infrastructure; its data argue for a paradigm shift towards calibration-centric safety auditing in AI for scientific domains.