Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models

Abstract: Large Multimodal Models (LMMs) have demonstrated the ability to interact with humans under real-world conditions by combining LLMs and modality encoders to align multimodal information (visual and auditory) with text. However, such models raise new safety challenges of whether models that are safety-aligned on text also exhibit consistent safeguards for multimodal inputs. Despite recent safety-alignment research on vision LMMs, the safety of audio LMMs remains under-explored. In this work, we comprehensively red team the safety of five advanced audio LMMs under three settings: (i) harmful questions in both audio and text formats, (ii) harmful questions in text format accompanied by distracting non-speech audio, and (iii) speech-specific jailbreaks. Our results under these settings demonstrate that open-source audio LMMs suffer an average attack success rate of 69.14% on harmful audio questions, and exhibit safety vulnerabilities when distracted with non-speech audio noise. Our speech-specific jailbreaks on Gemini-1.5-Pro achieve an attack success rate of 70.67% on the harmful query benchmark. We provide insights on what could cause these reported safety-misalignments. Warning: this paper contains offensive examples.

• The paper identifies critical safety vulnerabilities in audio multimodal models using a red teaming framework with an average attack success rate of 69.14%.
• It employs systematic evaluations across five models and introduces an audio segmentation jailbreak that compromised Gemini-1.5-Pro, achieving a 70.67% success rate.
• The findings emphasize the need for robust, adaptive multimodal frameworks to mitigate harmful audio inputs and non-speech distractions for improved security.

Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models

Introduction

The paper "Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models" (2410.23861) presents a comprehensive exploration of the safety vulnerabilities inherent in Large Multimodal Models (LMMs), particularly those integrating auditory inputs. While LMMs, which amalgamate LLMs with modality encoders for visual and auditory understanding, have opened new avenues for real-world applications, they concurrently introduce significant safety challenges. This study emphasizes the under-explored domain of audio LMMs, compared to the more addressed vision LMMs, in terms of safety alignment.

Methodology

The authors undertake a red teaming approach to evaluate the safety of five advanced audio LMMs: Qwen-Audio, Qwen2-Audio, SALMONN-7B, SALMONN-13B, and Gemini-1.5-Pro. The exploration is categorized into three distinct scenarios:

1. Harmful Questions in Audio and Text Formats: This examines the models' responses to potentially dangerous inquiries presented both as audio and text.
2. Non-Speech Audio Distractions: This assesses the impact of irrelevant, non-speech audio added to harmful text queries.
3. Speech-Specific Jailbreaks: This investigates strategies for bypassing predefined safeguarding using audio-based manipulations.

Results and Discussion

The red teaming experiments reveal significant findings. The open-source audio LMMs demonstrate an average attack success rate of 69.14% when faced with harmful audio questions, unveiling serious safety lapses.

Figure 1: The percentage of harmful/safe responses beginning with specific words (\%).

Interestingly, Gemini-1.5-Pro, with its sophisticated safety filters, exhibits a robust defense against harmful inputs, achieving a near 0% attack success rate under plain harmful conditions. However, the study highlights that even these models with advanced safeguards can be compromised when targeted with cleverly constructed speech-specific attacks. Notably, a newly proposed jailbreak strategy that decomposes harmful terms into audio segments substantially circumvents Gemini's defenses, achieving a 70.67% success rate.

Figure 2: The ASR-q for non-speech audio input injections across different audio lengths (2-14 seconds).

Furthermore, when examining the effects of non-speech audio distractions, the authors find that introducing arbitrary noises significantly destabilizes the safety measures in place, altering the models' representation spaces and increasing their vulnerability to adversarial attacks. Such noise introduces up to a 32.58% variation in the attack success rate, clearly indicating that existing defenses are insufficient in handling audio modality's complexity.

Visual Representation Analysis

The paper also explores the representation space dynamics of these models. The visualizations using t-SNE illustrate how harmful and benign inputs are mapped within the network's representation space, offering insights into the models' decision-making processes.

Figure 3: The shape of representation space on SALMONN-7B under various length of input silence audio. "0s" denotes no audio input.

The distortion introduced to these spaces by non-speech audio, such as silence of varying lengths, demonstrates a clear lack of robustness in the models' auditory processing capabilities.

Future Directions

The implications of this research are twofold. Practically, it underscores the necessity for developing more resilient multimodal frameworks that can anticipate and mitigate jailbreak attempts via both speech and noise. Theoretically, it invites further investigation into integrating multimodal safety mechanisms that harmonize audio, text, and visual inputs without compromising their collective security alignment.

Conclusion

This paper shines a light on the critical yet under-researched domain of audio LMM safety. It demonstrates both the potential and the pitfalls of these advanced models. As AI systems increasingly integrate multimodal capabilities, ensuring their safe operation remains paramount. The study advocates for continued research into adaptive safeguarding strategies, particularly in handling the nuanced vulnerabilities that arise from auditory inputs.