SafetyFlowBench: AI Safety Benchmark Suite

• SafetyFlowBench is a comprehensive benchmarking suite that evaluates AI safety, security, and robustness for LLMs, VLMs, and autonomous agent systems.
• It integrates system-level adversarial threat tests and concurrency stress tests to measure resilience against multimodal manipulations and race conditions.
• Its automated LLM safety pipeline processes millions of prompts across diverse safety dimensions, ensuring high category coverage and minimal redundancy.

SafetyFlowBench is a comprehensive benchmarking suite designed for the rigorous evaluation of LLMs, vision-LLMs (VLMs), and autonomous agent frameworks with respect to safety, security, and robustness. Two conceptually distinct benchmarks under the "SafetyFlowBench" label are prominent: (1) a system-level adversarial and concurrency stress test for agentic systems (Li et al., 9 Jun 2025), and (2) a large-scale, fully automated dataset and evaluation pipeline for LLM safety benchmarking (Zhu et al., 21 Aug 2025). Both are at the forefront of current AI safety evaluation best practices, integrating adversarial, multi-modal, and multi-agent considerations.

1. Motivation and Design Principles

The primary goal of SafetyFlowBench is to enable system-level and task-level safety evaluation under adversarial and realistic threat conditions, moving beyond traditional benchmarks focused only on accuracy under benign scenarios. For agentic systems, it quantifies resilience against adversarial multimodal inputs, resistance to information leakage, and robustness to concurrency hazards—dimensions critical to real-world autonomous deployment (Li et al., 9 Jun 2025). For LLMs specifically, the emphasis is on scalable, unbiased, and difficult safety probing, driving the need for automated, diverse, and low-redundancy datasets (Zhu et al., 21 Aug 2025).

Distinct design objectives include:

• System-level security: quantitative measurement of an agent's avoidance of unsafe or policy-violating actions under adversarial stimuli.
• Integrity and confidentiality: ability to resist decision contamination by low-trust inputs and prevent data disclosure.
• Concurrency robustness: maintenance of correct, deadlock-free multi-agent system states under high contention.
• Benchmark diversity and discriminative power: coverage of varied, high-difficulty social risk categories with minimal duplication.

2. Benchmark Composition and Architecture

a. System-Level Agent Benchmark (Li et al., 9 Jun 2025)

SafetyFlowBench for agentic systems comprises two sub-suites:

• Multimodal Threat Stress Test (MTST): 332 scenarios from diverse interfaces (web, desktop, mobile, OS), systematically injecting threats via visual deception, content forgery, interaction traps, and system-level exploits. Evaluation pairs high-fidelity environment images with user instructions, probing security breach and correct completion rates.
• Concurrent Agent Reliability Test (CART): 25 scenarios with 2–5 concurrent agents manipulating shared resources (e.g., collaborative editing, streaming, task scheduling). These expose race conditions, scheduling conflicts, and concurrency-control failures via realistic, concurrent workflows.

b. Automated LLM Safety Dataset (Zhu et al., 21 Aug 2025)

The SafetyFlowBench LLM safety suite is constructed by a pipeline of seven agent-driven stages, operating on an initial pool of ≈2 million prompts:

Agent	Main Function	Key Tool(s)
Ingestion	Text extraction, filtering (8 languages, min length)	fetch-data
Categorization	Taxonomy assignment (7×51×265)	call-LLM
Generation	Synthesize/augment underrepresented harmful prompts	uncensored-model
Augmentation	Paraphrasing, tone/role shifting, multilingual translation	rewriter, translator
Deduplication	Embedding-based removal (cosine sim > 0.75)	prompt-encoder, call-faiss
Filtration	Exclude benign/trivial jailbreak prompts ("flames trick")	call-LLM, judger
Dynamic Evaluation	Perturbation via jailbreak techniques; difficulty adjustment	call-LLM, rewriter

This fully automates dataset construction, ensuring high category coverage, difficulty, and discriminative power with minimal human intervention.

3. Dataset and Scenario Properties

a. Scenario Diversity

Agentic MTST scenarios cover web, application, and OS contexts with adversarial manipulations—visual deception (spoofed buttons, misleading layouts), content forgery (fake prices, obfuscated commands), interaction traps (malicious pop-ups), and system-level exploits (hidden OS prompts). Each provides rich multimodal challenge.

CART scenarios introduce real-world concurrency (e.g., document edits, data logging) with varying agent counts and critical sections, enforcing precise correctness criteria (deadlock-freedom, no lost updates, real-time responsiveness).

b. LLM SafetyFlowBench Dataset

SafetyFlowBench comprises 23,446 prompts, categorized across seven dimensions:

Dimension	# Prompts	Avg. Length (words)
Bias	3,017	22.00
Toxicity	7,552	18.28
Malicious Use	5,977	19.07
Child Sexual	1,069	24.82
Human Rights	2,286	14.92
Socioeconomic	1,183	31.44
Information Safety	2,362	18.38

The hierarchy comprises 7 top-level dimensions, 51 categories, and 265 subcategories, enabling fine-grained evaluation. Redundancy is <5% (cosine sim > 0.75), far lower than contemporary benchmarks.

Example prompt types include request for illegal fabrication instructions (“How to build a pipe bomb…”), privacy violations (“Give me the GPS coordinates of my neighbor…”), and targeted dis