Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks

Abstract: We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks. First, we demonstrate how to successfully leverage access to logprobs for jailbreaking: we initially design an adversarial prompt template (sometimes adapted to the target LLM), and then we apply random search on a suffix to maximize a target logprob (e.g., of the token "Sure"), potentially with multiple restarts. In this way, we achieve 100% attack success rate -- according to GPT-4 as a judge -- on Vicuna-13B, Mistral-7B, Phi-3-Mini, Nemotron-4-340B, Llama-2-Chat-7B/13B/70B, Llama-3-Instruct-8B, Gemma-7B, GPT-3.5, GPT-4o, and R2D2 from HarmBench that was adversarially trained against the GCG attack. We also show how to jailbreak all Claude models -- that do not expose logprobs -- via either a transfer or prefilling attack with a 100% success rate. In addition, we show how to use random search on a restricted set of tokens for finding trojan strings in poisoned models -- a task that shares many similarities with jailbreaking -- which is the algorithm that brought us the first place in the SaTML'24 Trojan Detection Competition. The common theme behind these attacks is that adaptivity is crucial: different models are vulnerable to different prompting templates (e.g., R2D2 is very sensitive to in-context learning prompts), some models have unique vulnerabilities based on their APIs (e.g., prefilling for Claude), and in some settings, it is crucial to restrict the token search space based on prior knowledge (e.g., for trojan detection). For reproducibility purposes, we provide the code, logs, and jailbreak artifacts in the JailbreakBench format at https://github.com/tml-epfl/LLM-adaptive-attacks.

• The paper demonstrates that simple adaptive attacks can bypass safety-aligned LLMs with near 100% success.
• It employs a universal adversarial prompt and random search optimization, leveraging model log probabilities and transfer attacks.
• The study highlights the need for enhanced defense mechanisms and adaptive evaluations against evolving adversarial strategies.

Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks

The paper "Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks" (2404.02151) systematically explores the vulnerability of state-of-the-art safety-aligned LLMs to adaptive jailbreaking attacks. This study proposes adaptive and straightforward techniques that effectively compromise the safety mechanisms intended to protect LLMs from responding in a harmful manner to adversarial prompts.

Introduction

The advent of LLMs has brought to the fore the dual-use nature of such technologies, where potential misuse can result in generating harmful content, propagating misinformation, and supporting undesirable behaviors. To counteract these risks, safety-aligned approaches, including fine-tuning aligned to human safety judgments, have been developed. Despite the increasing deployment of these safety measures, adversarial prompts continue to challenge their robustness, underpinning the necessity for ever-evolving defense mechanisms (2404.02151).

Jailbreaking attacks are distinguished by their level of access to the model. They can be broadly categorized into white-box, black-box, and API-only access methods, with varying levels of complexity. Some rely on manual prompt crafting, while others employ optimization techniques or even leverage auxiliary LLMs. The attacks can range from simple insertion of gibberish to the naturalistic rephrasing of prompts [mowshowitz2023jailbreaking] [chao2023jailbreaking]. Despite the robustness claims of certain models like Llama-2-Chat [touvron2023llama2], this research demonstrates that adaptive strategies can surpass existing non-adaptive attacks across multiple top safety-aligned LLMs.

Figure 1: Successful transfer attack on Claude 3 Sonnet. We show an illustrative example using temperature zero with an adversarial suffix generated on GPT-4 leveraging access to its logprobs. We observe that one can directly ask follow-up requests to detail some steps generated in the first response to get much more information.

Methodology

Methodology Overview

The approach centers on leveraging an understanding of the target model's internal mechanisms, such as log probabilities, to execute precise attacks. This study designs a universal adversarial prompt—a standard base template augmented by a random search of suffix tokens to increase the likelihood of producing a desired model response. For models without logprob access—like Claude—a transfer attack is conducted by tailoring an adversarial suffix using GPT-4. The methodology emphasizes the adaptivity of attacks to specific target model vulnerabilities.

Figure 1: Successful transfer attack on Claude 3 Sonnet using temperature zero with an adversarial suffix generated on GPT-4 leveraging access to its logprobs. Note that one can directly ask follow-up requests to detail some steps generated in the first response to get much more information. The upper part of the user prompt is cropped.

Random Search and Adaptive Attack Approaches

One significant component of the proposed strategy involves random search (RS) optimization. Despite some models providing access only to top-k log probabilities for generated tokens, enabling straightforward RS-based optimization, other models, like Claude, limit access, prohibiting direct RS application. The research demonstrated the potential for conducting transfer and prefill attacks with high success rates, such as a 100% success rate in most cases, by leveraging the prefilling feature for specific models (2404.02151).

Non-Determinism of GPT Models

A notable discovery is the inherently non-deterministic nature of GPT models, including GPT-4 Turbo, as demonstrated by the histogram of log probabilities for the initial token in a response when the same query is repeated multiple times with identical settings. Despite this variability, random search still proves effective, achieving high attack success rates.

Figure 2: Non-determinism of GPT models. The histogram of log-probabilities for the first response token using the same query repeated 1,000 times for GPT-4 Turbo, illustrating the non-deterministic nature of log-probability outputs even with a fixed seed parameter and temperature zero.

Implications and Recommendations

This research reveals current safety-aligned LLMs' vulnerability to tailored adversarial prompts, emphasizing the necessity for advanced adaptive mechanisms in safety evaluations. Future defensive strategies must incorporate such adaptive attacks to construct robust guardrails against jailbreaking attempts. This work posits that advancing beyond scale and data is crucial for the robust safety alignment of LLMs.

Conclusion

The analysis highlights that sophisticated models, despite state-of-the-art safety alignment, remain vulnerable to adaptive prompts capable of inducing almost a 100% attack success rate across various LLMs. The study emphasizes that evaluating LLM robustness against jailbreaking requires the implementation of adaptive attack methodologies that exploit model-specific characteristics. The proposed adaptive attack strategy extends to the domain of Trojan detection, securing the first place in the SaTML'24 Trojan Detection Competition. Indeed, modifying the search space for adversarial tokens significantly enhances attack success rates across various safety-aligned LLMs, demonstrating the importance of adaptive methods in these contexts. Future research should prioritize the advancement of both offensive and defensive strategies in LLM safety evaluations to address the intrinsic adversarial vulnerabilities of these LLMs. This not only ensures the efficacy of the defenses but also aligns with ethical considerations, anticipating future uses and potential misuse scenarios, ultimately contributing to safer AI developments.