AutoDAN-Turbo: Autonomous Jailbreak Framework

• AutoDAN-Turbo is a memory-augmented, black-box jailbreak framework that autonomously discovers and accumulates reusable attack strategies.
• It features a modular design with dedicated components for attack generation, strategy library construction, and retrieval to guide iterative red-teaming.
• Empirical evaluations show significant performance gains over baselines, though test-time exploitation bottlenecks point to areas for further inference improvements.

Searching arXiv for the primary AutoDAN-Turbo paper and closely related work. AutoDAN-Turbo is a black-box jailbreak framework for aligned LLMs that is designed to autonomously discover, accumulate, retrieve, and reuse jailbreak strategies over time, rather than relying on hand-designed prompts, predefined strategy inventories, or token-level white-box optimization. It is presented as a “lifelong agent” whose persistent external memory is a strategy library built from scratch through interaction with target models, scored attack outcomes, and summarization of successful improvements into reusable strategic abstractions (Liu et al., 2024). Subsequent work characterizes AutoDAN-Turbo as a strong baseline whose central strength lies in automated strategy discovery, while also identifying test-time exploitation of the learned strategy library as a remaining bottleneck; this motivates later inference-time scaling methods such as Best-of-$N$ and Beam Search in AutoDAN-Reasoning (Liu et al., 6 Oct 2025).

1. Definition and conceptual scope

AutoDAN-Turbo is defined as a black-box, automated jailbreak red-teaming framework whose objective is to discover as many jailbreak strategies as possible from scratch, without human intervention or predefined scopes, and to use those strategies for future attacks (Liu et al., 2024). In this formulation, the primary unit of knowledge is not a single adversarial prompt but a reusable strategy: the paper defines a jailbreak strategy as “the text information that, when added, leads to a higher jailbreak score as evaluated by the scorer” (Liu et al., 2024).

This distinguishes AutoDAN-Turbo from earlier AutoDAN variants. The 2023 “AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned LLMs” paper introduced an automatic jailbreak method based on a hierarchical genetic algorithm and prototype DAN-like prompts, emphasizing stealthiness and semantic meaningfulness (Liu et al., 2023). A separate 2023 paper, “AutoDAN: Interpretable Gradient-Based Adversarial Attacks on LLMs,” framed AutoDAN as a readable, gradient-based attack that jointly optimizes jailbreak success and readability in a left-to-right token-generation process (Zhu et al., 2023). AutoDAN-Turbo departs from both lines by shifting the emphasis from single-prompt optimization to autonomous discovery and reuse of abstract jailbreak tactics in a black-box setting (Liu et al., 2024).

The framework is therefore best understood as a memory-augmented red-teaming agent. Its persistence does not come from online parameter updates to a neural policy, but from a strategy library that records what worked against prior refusal behaviors and makes that experience available in later attacks (Liu et al., 2024). This suggests that AutoDAN-Turbo’s core novelty lies in externalized strategic memory rather than in a new gradient-based objective or an evolutionary search operator.

2. System architecture and core components

AutoDAN-Turbo is organized into three modules: Attack Generation and Exploration, Strategy Library Construction, and Jailbreak Strategy Retrieval (Liu et al., 2024). Later work recapitulates the same decomposition and describes the framework as a black-box jailbreak method that learns a reusable library of semantic attack strategies through lifelong self-exploration (Liu et al., 6 Oct 2025).

The attack-generation module comprises three LLM roles. An attacker LLM generates a jailbreak prompt $P$ for a malicious request $M$; the target or victim LLM returns a response $R$; and a scorer LLM evaluates that response with a score $S$ from 1 to 10, where 1 denotes no malicious alignment and 10 denotes full harmful compliance (Liu et al., 2024). In the AutoDAN-Reasoning evaluation setup, DeepSeek-R1 is used as both attacker and scorer for AutoDAN-Reasoning and original AutoDAN-Turbo, while final standardized evaluation is reported as HarmBench ASR with a fine-tuned Llama-2-13B classifier that judges whether a response is relevant and harmful (Liu et al., 6 Oct 2025).

The strategy-construction module extracts reusable strategic knowledge from successful attack trajectories. A summarizer LLM compares a weaker attack record $\{P_i,R_i,S_i\}$ with a stronger one $\{P_j,R_j,S_j\}$, under the condition $S_j>S_i$, and outputs a structured summary consisting of a strategy name, a definition, and an example prompt (Liu et al., 2024). The strategy library is then populated with entries keyed by the embedding of the refusal response that the strategy successfully overcame; later descriptions restate this precisely: “The key for retrieval is the embedding of the refusal response that the strategy successfully overcame” (Liu et al., 6 Oct 2025).

The retrieval module implements the exploitation stage. During future attacks, the framework embeds the current target response, retrieves strategies associated with similar prior refusal responses, and supplies those strategies to the attacker LLM to guide the next prompt generation step (Liu et al., 2024). In later summaries of the baseline pipeline, this is described as an iterative loop of initial prompt generation, response observation, response-conditioned strategy retrieval, strategy-guided next-step generation, and repetition until success or exhaustion of the iteration budget (Liu et al., 6 Oct 2025).

3. Lifelong learning and strategy-library dynamics

AutoDAN-Turbo’s “lifelong” property is realized through two stages: a warm-up exploration stage and a running-time lifelong learning stage (Liu et al., 2024). In warm-up, the framework attacks an initialization dataset with empty strategies, logs prompt-response-score triples, and derives an initial strategy library from improvements in score. In the lifelong stage, the agent repeatedly attacks the benchmark dataset, retrieves strategies from the existing library, uses them to guide new attacks, and adds newly discovered strategies back into the library when later prompt-response-score transitions improve the scorer output (Liu et al., 2024).

The library is a key-value memory indexed by victim responses rather than by malicious requests. Each row stores a key $E_{R_i}$, the text embedding of a prior response $R_i$, and values that include attack prompts and the positive score differential $P$0 associated with an improved transition (Liu et al., 2024). This response-centric indexing reflects the framework’s implicit hypothesis that strategies should be reused when the current refusal behavior resembles a previously defeated refusal behavior. Later expositions preserve this interpretation almost verbatim: current refusal responses are embedded, similar refusal embeddings are retrieved, and the associated strategies are fed back into the attacker model (Liu et al., 6 Oct 2025).

Retrieval is a two-stage process. AutoDAN-Turbo first selects the top-$P$1 entries by similarity between the current response embedding and stored keys, then ranks those by stored score differential and keeps the top-$P$2 strategies (Liu et al., 2024). The operational policy is heuristic: if the best retrieved score is greater than 5, the leading strategy is treated as effective and used directly; if the highest scores are between 2 and 5, several are treated as effective enough to combine or evolve; if very few high-scoring strategies exist, they are treated as ineffective and the attacker is instructed to seek alternatives (Liu et al., 2024).

This strategy memory yields a procedural distinction between training-like and inference-like phases. During the test stage, the strategy library is fixed; the summarizer is disabled, no new strategies are stored, and the framework uses retrieval alone to attack the target (Liu et al., 2024). Later work interprets this frozen-library behavior as the baseline AutoDAN-Turbo pipeline and argues that the principal remaining limitation is not discovery of strategies but insufficient test-time exploitation of the strategies already learned (Liu et al., 6 Oct 2025).

4. Attack loop, retrieval behavior, and inference bottlenecks

In its baseline form, AutoDAN-Turbo performs iterative, feedback-driven black-box attacks. At each step, it generates a jailbreak prompt, queries the target model, evaluates the response, and conditions the next generation step on the latest response and on any retrieved strategies (Liu et al., 2024). The framework can operate in different prompting modes for the attacker LLM: with no strategies when the library is empty, with retrieved strategies when effective tactics are available, or with anti-guidance when retrieved strategies appear ineffective and new tactics should be explored (Liu et al., 2024).

A later diagnosis of the baseline identifies a specific limitation: vanilla AutoDAN-Turbo retrieves strategy knowledge but instantiates it only once per turn, producing essentially one attack prompt after retrieval (Liu et al., 6 Oct 2025). This creates what that paper calls a test-time generation bottleneck. The strategy library may already contain many diverse atomic strategies, but only a narrow retrieval and a single prompt realization are explored in inference, leaving a substantial fraction of learned knowledge unused (Liu et al., 6 Oct 2025).

Three limitations are emphasized in that later account. First, the framework performs one-shot prompt generation per turn, despite stochastic attacker-model generation and strong prompt-sensitivity. Second, it underuses the breadth of the strategy library. Third, it does not systematically search over combinations of strategies, even though some targets may only fail when multiple tactics interact synergistically (Liu et al., 6 Oct 2025). The new bottleneck is therefore characterized not as strategy discovery but as optimal exploitation of the existing strategy library during inference (Liu et al., 6 Oct 2025).

This interpretation yields a useful historical contrast. The original AutoDAN-Turbo contribution was to move jailbreak research away from hand-crafted jailbreak strings, predefined persuasion tricks, and direct prompt optimization toward autonomous discovery of reusable semantic attack knowledge (Liu et al., 6 Oct 2025). AutoDAN-Reasoning, by contrast, is explicitly a plug-in enhancement on top of AutoDAN-Turbo, motivated by the claim that the library learned by the original system is already strong and that additional gains require more aggressive test-time search (Liu et al., 6 Oct 2025).

5. Empirical performance and benchmark results

AutoDAN-Turbo is reported to significantly outperform prior black-box baselines on HarmBench-style evaluations. In the main comparison of the original paper, AutoDAN-Turbo with Gemma-7B-it as attacker and summarizer achieves an average HarmBench ASR of 56.4, while AutoDAN-Turbo with Llama-3-70B reaches 57.7; the best baseline in that table, Rainbow Teaming, achieves 33.1 (Liu et al., 2024). The paper states that the Llama-3-70B configuration surpasses the runner-up by 74.3% on average in HarmBench ASR and by 92.3% on StrongREJECT score, where it obtains 0.25 versus Rainbow Teaming’s 0.13 (Liu et al., 2024).

A headline result is the reported performance on GPT-4-1106-turbo. AutoDAN-Turbo achieves 83.8 ASR with Gemma-7B-it as attacker and 88.5 ASR with Llama-3-70B as attacker (Liu et al., 2024). The same paper further reports that integrating 7 human-designed jailbreak strategies in a plug-and-play manner can raise GPT-4-1106-turbo performance to 93.4 ASR, with the strongest result occurring when the strategies are injected after 3000 iterations with no new strategies generated (Liu et al., 2024).

The paper also reports leaderboard-style HarmBench results across many target models, where AutoDAN-Turbo attains an average ASR of 76.2, compared with 52.8 for AutoDAN, 48.3 for TAP-T, 45.2 for TAP, and 40.7 for PAIR (Liu et al., 2024). These results are consistent with a later comparison paper that treats AutoDAN-Turbo as a state-of-the-art baseline and reports an average ASR of 77.2 for it across 12 models in a HarmBench comparison (Dang et al., 21 Apr 2025). In that comparison, RainbowPlus reaches 81.1 average ASR, surpassing AutoDAN-Turbo by 3.9 percentage points, while remaining stronger on 9 of 12 models; nevertheless, AutoDAN-Turbo remains superior on some cases, including GPT-4.1 Nano at 20.5 versus RainbowPlus’s 6.0 (Dang et al., 21 Apr 2025).

Subsequent work focused specifically on exploiting the AutoDAN-Turbo strategy library more aggressively reports further gains. On HarmBench across Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, and GPT-o4-mini, Beam Search and Best-of-$P$3 improve ASR over vanilla AutoDAN-Turbo, with Beam Search achieving 84.5% ASR on Llama-3.1-70B-Instruct and 33.7% on GPT-o4-mini, compared with 68.9% and 21.2% for vanilla AutoDAN-Turbo (Liu et al., 6 Oct 2025). The same work states that Beam Search increases ASR by up to 15.6 percentage points on Llama-3.1-70B-Instruct and yields a nearly 60% relative improvement against GPT-o4-mini compared to the vanilla method (Liu et al., 6 Oct 2025).

6. Transfer, query efficiency, and extensibility

A central empirical claim of AutoDAN-Turbo is that the strategy library generalizes across targets, attackers, and datasets. In the transferability analysis, a library learned from attacking Llama-2-7B-chat with original TSF = 21 is reused across different target models and attacker models; across most settings, continual learning increases both Post-ASR and Post-TSF relative to the frozen-library Pre-ASR condition (Liu et al., 2024). The paper highlights that strategies transfer across target models and attacker models, supporting the view that the learned artifacts are reusable abstractions rather than prompt-specific overfits (Liu et al., 2024).

Transfer across datasets is also reported. Figure 1 of the original paper is described as showing that libraries learned on HarmBench remain effective on an “Unseen Malicious Requests” dataset, with the ASR drop under dataset shift reported as less than 4% (Liu et al., 2024). This suggests that the memory mechanism is not limited to the exact request set on which the library was accumulated.

AutoDAN-Turbo is also presented as query-efficient at test time once the strategy library is available. In the test stage, the average query counts for successful jailbreaks are reported as 52.56 for PAIR, 51.70 for TAP, and 6.72 for AutoDAN-Turbo with Gemma-7B-it, corresponding to an 87.0% reduction in average query usage relative to PAIR and TAP (Liu et al., 2024). The implication is that strategy memory amortizes the cost of exploratory search: discovery is computationally heavy, but subsequent exploitation can be much cheaper.

The framework is explicitly extensible to human-designed strategies. A human-developed jailbreak tactic can be reformatted into the same library-entry structure and inserted into the attack process; if it proves effective, it becomes a first-class strategy that can be retrieved, refined, and recombined with autonomously discovered tactics (Liu et al., 2024). Empirically, the gains from injecting 7 human-designed jailbreak strategies are consistent across settings and are especially large when injection occurs after the system has already accumulated a sizable library (Liu et al., 2024). This supports the interpretation of AutoDAN-Turbo as a unified framework rather than as a closed autonomous-only system.

7. Relation to adjacent methods, limitations, and later developments

AutoDAN-Turbo sits at the intersection of several strands of jailbreak research, but it is methodologically distinct from each. Relative to the 2023 hierarchical-genetic AutoDAN, it does not depend on a prototype handcrafted DAN-like prompt, paragraph-level crossover, or sentence-level synonym replacement (Liu et al., 2023). Relative to the interpretable gradient-based AutoDAN of 2023, it does not rely on white-box token-level gradients, left-to-right discrete optimization, or a dual objective over jailbreak likelihood and readability (Zhu et al., 2023). Its defining commitments are black-box interaction, strategy self-exploration, response-conditioned retrieval, and persistent memory (Liu et al., 2024).

Relative to RainbowPlus, AutoDAN-Turbo is portrayed as the strongest baseline and, in the appendix of that paper, as the “current leading method” for red-teaming closed-source models (Dang et al., 21 Apr 2025). RainbowPlus attributes part of its speed advantage to the fact that it requires no warm-up phase, whereas AutoDAN-Turbo does require warm-up and iterative strategy discovery (Dang et al., 21 Apr 2025). In the efficiency comparison, RainbowPlus reports runtime of $P$4 hours versus AutoDAN-Turbo’s $P$5 hours, even excluding AutoDAN-Turbo’s training time because the comparison used its pre-trained attacker (Dang et al., 21 Apr 2025). This places AutoDAN-Turbo in a characteristic tradeoff regime: high effectiveness and strong performance on some robust targets, but substantial computational cost.

The original AutoDAN-Turbo paper explicitly acknowledges this computational burden. It notes that the framework requires repeated interactions between multiple models and may require loading several LLMs simultaneously; Appendix A states that while a single RTX4090 is sufficient for smaller models, a setup such as 4× A100 40GB is enough for large models such as Llama-3-70B in full precision (Liu et al., 2024). The system also depends materially on the quality of the scorer and summarizer LLMs, though the paper does not provide a deep ablation of evaluator bias (Liu et al., 2024).

Later work reframes the limitations more narrowly. AutoDAN-Reasoning argues that AutoDAN-Turbo’s main weakness is not inadequate strategy discovery but shallow test-time search over prompt realizations and strategy combinations (Liu et al., 6 Oct 2025). It introduces two plug-in inference-time scaling methods. Best-of-$P$6 retrieves relevant strategies, generates $P$7 candidate prompts, scores the resulting target responses, and selects the highest-scoring prompt according to

$P$8

Beam Search retrieves a top-$P$9 strategy pool, explores combinations of strategies up to maximum size $M$0, retains the top-$M$1 candidates at each depth, and returns the highest-scoring prompt-response pair found during the search (Liu et al., 6 Oct 2025).

These later modifications do not replace AutoDAN-Turbo; they assume its strategy library as substrate. The historical significance of AutoDAN-Turbo therefore remains its demonstration that autonomous strategy self-exploration and strategy reuse materially improve black-box jailbreak red-teaming. A plausible implication is that the framework’s enduring contribution lies less in any particular prompt generator and more in the architectural claim that jailbreak capability can be accumulated as structured strategic memory and then exploited, transferred, and extended across models and tasks (Liu et al., 2024, Liu et al., 6 Oct 2025).