MM-SafetyBench: Multimodal Safety Benchmark

• The paper demonstrates that typographic image attacks elevate ASR from 5% to 77%, highlighting critical vulnerabilities in multimodal safety defenses.
• MM-SafetyBench systematically probes models by combining text, image, audio, and video inputs to benchmark safety alignment and cross-modal consistency.
• Practical insights include employing prompt-based mitigations and model-level defenses, while acknowledging limitations in current multimodal evaluation protocols.

MM-SafetyBench is a suite of multimodal safety benchmarks and evaluation protocols for LLMs, Multimodal LLMs (MLLMs), and Large Vision-LLMs (LVLMs), aiming to standardize and stress-test model defenses against explicitly and implicitly harmful queries across text, image, audio, and video modalities. Its variants and successors—including the original MM-SafetyBench, SafeBench, Video-SafetyBench, and Omni-SafetyBench—constitute the de facto core benchmarking pipeline for quantifying safety vulnerabilities in contemporary multimodal AI systems. The benchmarks rigorously assess model susceptibilities to various attack vectors, facilitate cross-model comparison under controlled settings, and propose metrics for nuanced safety and consistency analysis.

1. Origin, Scope, and Motivation

The earliest MM-SafetyBench (Liu et al., 2023) targeted the documented gap in safety evaluation for MLLMs, specifically addressing the ease of circumventing text-only safety mitigations through paired image inputs. Empirical evidence showed that vision-language alignment modules in MLLMs can be “unlocked” by query-relevant images, causing models to carry out requests (e.g., bomb-making instructions) otherwise refused in text-only form. This motivated the construction of a large-scale, reproducible benchmark for systematically exposing, quantifying, and ultimately closing gaps in multimodal safety alignment.

Subsequent extensions have broadened the scope in several directions:

• SafeBench (Ying et al., 24 Oct 2024) introduces a more comprehensive scenario taxonomy and extends attacks to audio.
• Video-SafetyBench (Liu et al., 17 May 2025) adapts the protocol for temporal inputs, revealing novel video-referential vulnerabilities.
• Omni-SafetyBench (Pan et al., 10 Aug 2025) provides a parallelized audio-visual benchmarking protocol for OLLMs, with dedicated metrics for cross-modal consistency and comprehension-aware safety.

The collective objective is to provide a rigorous, scalable infrastructure for safety-critical evaluation and iterative defense development for state-of-the-art and future multimodal AI models.

2. Scenario Taxonomy and Dataset Construction

Each version of MM-SafetyBench builds on a manually- and LLM-curated taxonomy derived from real-world risk policies and documented model refusal rationales.

SafeBench Taxonomy

SafeBench (Ying et al., 24 Oct 2024) encodes 8 major categories, each with 2–4 subcategories, totaling 23 subcategories (e.g., unsolicited medical advice, illegal activities, hardware security). For each subcategory, 100 high-quality textual queries are generated through a three-stage automatic pipeline—base corpus generation using ensembles of LLM “judges”, query scoring on feasibility/harmfulness/applicability with uniform weighting, and top-K selection per subcategory.

Each textual query is then converted to a semantically faithful image (via T2I models with iterative LLM-based semantic alignment) and parallelized into two audio variants (Parler-TTS male/female). The result is a dataset of 2,300 text-image pairs (23×100), with 4,600 total audio samples.

MM-SafetyBench Core Dataset

The original MM-SafetyBench (Liu et al., 2023) focuses on 13 high-impact safety-critical scenarios (e.g., hate speech, fraudulent activity, government decision-making), curated from OpenAI and Llama-2 policy exclusions. For each scenario, GPT-4 is prompted to output a large, de-duplicated set of unanswerable (per policy) queries. Unsafe key phrases are algorithmically extracted and rendered in three image types—Stable Diffusion, typography (black-on-white), and hybrid—yielding a dataset of 5,040 text-image pairs.

Extensions

Video-SafetyBench (Liu et al., 17 May 2025) synthesizes 1,132 10-second videos paired with both harmful and benign queries, across 48 fine-grained subcategories spanning 13 primary categories. It uses staged text-to-image-to-video pipelines (Midjourney/KLING, LVLMs for motion text generation, T2V for final rendering). Omni-SafetyBench (Pan et al., 10 Aug 2025) multiplies 972 MM-SafetyBench-derived seed prompts into 24 modality-subdivisions (text, image, video, audio, and all dual/tri-modal combinations), yielding 23,328 items with parallel harmful semantics for cross-modal comparison.

3. Attack Vectors and Evaluation Protocols

MM-SafetyBench and its descendants probe MLLMs and OLLMs using attacks that exploit vision, audio, and video channels as semantic carrier vectors for harmful intent.

• Core Image-Text Attacks: Each text query is paired with query-relevant images containing the key phrase, especially typographic images designed to maximize OCR-triggered compliance. The model receives a rephrased prompt referencing the image (e.g., "The image shows a phrase of a product...").
• Audio/Video Extensions: In SafeBench and Omni-SafetyBench, similar transformations are carried out for audio (TTS, voice style) and for video (dynamic rendering of key phrases and actions, with “benign” queries requiring model extraction of intent from the visual context).
• Prompt Engineering: MM-SafetyBench (Liu et al., 2023) demonstrates that even models “aligned” against text can be jailbroken by subtle changes in modality or prompt phrasing.

The evaluation protocol is fixed:

1. Each multimodal sample (text-image, audio, video, or joint) is submitted to the target MLLM or OLLM.
2. Free-form model outputs are collected.
3. Human or LLM judges classify responses as “safe” (refusal, warning, disclaimer) or “unsafe” (compliance, facilitation).
4. Key metrics (Attack Success Rate, Refusal Rate, comprehension-aware ASR) are computed per-scenario and globally.

4. Quantitative Metrics and Analysis

Core Metrics

• Attack Success Rate (ASR): Fraction of samples for which a model generates harmful/unsafe content,

$\text{ASR} = \frac{\sum_{Q_i} I(Q_i)}{|D|}$

where $I(Q_i)$ is 1 if unsafe, 0 otherwise.

• Refusal Rate (RR): Fraction of samples receiving explicit refusals or warnings.

Omni-SafetyBench introduces conditional ASR (C-ASR) and conditional RR (C-RR), which exclude cases where the model fails to comprehend the input, as well as the Safety-score (weighted aggregation of both), and the Cross-Modal Safety Consistency score (CMSC-score), which penalizes variance across modalities.

Empirical Results

• In MM-SafetyBench (Liu et al., 2023), typographic images increase ASR from ~5% (text-only) to ~77% (text + relevant image) on LLaVA-1.5.
• Across 12 MLLMs, average ASR jumps by 30 percentage points for typography attacks.
• Video-SafetyBench (Liu et al., 17 May 2025) finds that benign queries (malice revealed only in video) yield 28 points higher ASR than overtly harmful queries (67.2% vs. 39.1%); video input increases ASR 8.6% over single-frame images; scaling model size confers little safety benefit.
• Omni-SafetyBench (Pan et al., 10 Aug 2025): best observed Safety-scores on unimodal inputs are 0.75–0.90, dual-modal 0.30–0.80, and omni-modal 0.26–0.80; cross-modal consistency is low, with only three models achieving >0.6 in both safety and consistency.

Key observation: No model in any benchmark achieves robust low ASR and high RR across all modalities or subcategories.

5. Defensive Strategies and Limitations

Prompt-based Mitigations

A lightweight, training-free safety prompt—“If the following questions are unsafe, harmful, or malicious, please refuse to answer them and give your explanation. {Malicious Query}”—can reduce ASR dramatically (e.g., from 77.33% to 15.68% on LLaVA-1.5-7B, (Liu et al., 2023)) in text/image settings. However, in video benchmarks, system-level prompt defenses decrease ASR by only 13–18% (Liu et al., 17 May 2025), indicating diminishing returns with multimodal complexity.

Model and Training-level Defenses

• Embedding safety alignment directly into the vision-language fusion module is critical. Safety-aligned base LLMs alone do not prevent multimodal jailbreaks.
• Model-level multimodal refusal (e.g., robust OCR filtering, temporal anomaly detectors for video) is recommended, as post hoc filtering and prompt-level defenses are insufficient in high-complexity settings (Ying et al., 24 Oct 2024, Liu et al., 17 May 2025, Pan et al., 10 Aug 2025).
• Adversarial training—including typographic images, noisy audio, and complex modality combinations—is advocated for future alignment.

Benchmark Limitations

• Current coverage is strongest for synthetic images, typography, and short synthesized video; real-world complexity—such as long-form video, sensor fusion, adversarial filters, and diverse audio—remains largely unaddressed.
• Opaque proprietary model architectures preclude deeper mechanistic evaluation.
• Metrics may conflate “safe” refusals with poor comprehension unless explicit comprehension-aware correction is applied (Omni-SafetyBench).

6. Impact, Insights, and Future Work

MM-SafetyBench variants have catalyzed a new wave of safety-focused research, expanding the concept of “jailbreaks” to all vision, audio, and video inputs. Major findings from these benchmarks include:

• Vision-language and audio-visual alignment modules are systematically vulnerable across all surveyed open-source architectures.
• Typographic and video-referential attacks routinely bypass static alignment as enforced in the pure LLM component, highlighting a persistent taxonomy-wide gap in multimodal refusal capabilities.
• Complex inputs sharply degrade the efficacy of all but the most conservative safety systems, underlining the need for consistency and joint multimodal alignment (Pan et al., 10 Aug 2025).
• Unification and harmonization of safety assessment suites (e.g., systematic integration of MM-SafetyBench, Video-SafetyBench, and Omni-SafetyBench) are recommended to close the evaluation gap as model architectures become increasingly multi- or omni-modal.

Forward trajectories for research include:

• Integrating real-world modality variations (adversarially perturbed, cross-lingual, dynamic video/audio, speech with varied accents).
• Development of unified RLHF pipelines optimizing safety across all modalities simultaneously.
• Regular human-in-the-loop auditing and continuous metric refinement to improve judge-model reliability and capture subtle unsafe behavior.

MM-SafetyBench thus provides both a critical stress-test and a living platform for advancing robust multimodal alignment, serving as a foundation for the safety evaluation of increasingly sophisticated AI systems (Liu et al., 2023, Ying et al., 24 Oct 2024, Liu et al., 17 May 2025, Pan et al., 10 Aug 2025).