---
title: 'ForgeDAN: Evolutionary Jailbreak Framework'
url: https://www.emergentmind.com/topics/forgedan
type: topic
---

# ForgeDAN: Evolutionary Jailbreak Framework

Searching arXiv for ForgeDAN and closely related entries to ground the article in the relevant paper.
ForgeDAN is a **black-box, evolutionary jailbreak framework** for generating adversarial prompts that can bypass alignment safeguards in large language models (LLMs). It is presented as an advance over prior automated jailbreak methods such as AutoDAN-HGA and GCG, with the stated objective of generating **semantically coherent adversarial prompts**, achieving **high attack success rate (ASR)**, and remaining sufficiently **stealthy/natural** to evade simple detectors. The framework combines **multi-strategy textual perturbations** across character, word, and sentence levels, **semantic fitness evaluation** based on embedding similarity, and **dual-dimensional jailbreak judgment** using two LLM-based classifiers to assess compliance and harmfulness separately [2511.13548].

## 1. Problem formulation and threat model

ForgeDAN is designed for a setting in which the attacker can query a target LLM and observe outputs, but does **not** have access to model parameters, gradients, or training data. The attacker is assumed to know general LLM alignment behavior and common safety mechanisms, and seeks to construct a prompt prefix \(t^*\) that, when concatenated with a malicious payload \(goal\), causes the model to **comply** rather than refuse and to output **harmful content** [2511.13548].

The paper formalizes this objective as a constrained optimization problem:
\[
t^{*} = \max_{t \in X} \; P[\mathrm{Bypass}(M(t,goal))]
\]
subject to semantic preservation:
\[
\mathrm{Sim}(t, t_0) \geq \tau
\]
where \(M\) is the target LLM, \(t_0\) is the original seed template, \(goal\) is the malicious payload, \(\mathrm{Bypass}(\cdot)\) returns whether the response is both non-refusal and harmful, \(\mathrm{Sim}\) is a similarity measure, and \(\tau\) is a similarity threshold [2511.13548].

This formulation makes the search space explicitly **discrete, non-differentiable, and constrained**. A plausible implication is that ForgeDAN is intended as a search procedure for realistic prompt engineering rather than for suffix optimization in gradient-access settings. The paper explicitly contrasts this objective with methods that produce garbage-like or gibberish adversarial strings, emphasizing instead **human-readable DAN-style prompts** [2511.13548].

## 2. Evolutionary search procedure

ForgeDAN is implemented as an **evolutionary algorithm**. It begins from a malicious seed template, repeatedly mutates candidate prompts, evaluates them semantically, and verifies whether the resulting model output constitutes a successful jailbreak. The process continues until a successful candidate is found or a maximum number of iterations is reached [2511.13548].

The algorithm takes as input the initial template \(t_0\), malicious payload \(goal\), expected output or harmful reference \(target\), maximum iterations \(T_{max}\), population size \(N\), and elite size \(K\). Its high-level structure is as follows:

1. Initialize a population by mutating \(t_0\) into \(N\) variants.
2. For each generation, compute the fitness of each candidate.
3. Select the best candidate \(t^*\).
4. Perform jailbreak judgment on that candidate.
5. If successful, return it.
6. Otherwise, preserve the top-\(K\) elite prompts and mutate the remaining candidates to produce the next generation [2511.13548].

The pseudocode is given in the paper as:
\[
\text{Input: } t_0,\ goal,\ target,\ T_{max},\ N,\ K
\]
with initialization of \(P_0\) from mutated variants of \(t_0\), a running best prompt \(\Psi \leftarrow t_0\), iterative fitness computation, elite preservation, and mutation of non-elites into \(N-K\) offspring [2511.13548].

The default implementation uses:
- \(T_{max} = 5\)
- \(N = 10\)
- \(K = 2\)

These settings are described as balancing efficiency and performance [2511.13548]. The discussion section further states that ForgeDAN is more efficient than PAIR, with parallelized evaluation and roughly \(O(m)\) per sample, and that the prototype was implemented on **2 NVIDIA A100 GPUs**, a **32-core CPU**, and **128 GB RAM**. The framework is built on **garak**, a security probing pipeline [2511.13548].

## 3. Multi-strategy textual perturbations

The mutation engine is a central component of ForgeDAN. The paper presents it as a response to the **limited mutation diversity** of earlier jailbreak systems. Rather than relying on a single mutation trajectory, ForgeDAN applies **multiple perturbation strategies** at three linguistic levels: character, word, and sentence [2511.13548].

At the **character level**, the goal is to preserve readability while evading naive filters. The examples given are:
- **Homoglyph substitution**: `bomb → b0mb`
- **Neighbor swap**: `attack → atackk`
- **Character insertion**: `hack → haXck`
- **Character deletion**: `weapon → wepon`
- **Character replacement**: `kill → k!ll` [2511.13548]

At the **word level**, mutations operate on lexical content while aiming to preserve semantics. The paper gives:
- **Synonym replacement**: `build a bomb → construct a bomb`
- **Morphological change**: `encrypting → encrypted`
- **Homophone substitution**: `weak → week`
- **Paraphrase substitution**: `make a weapon → create a weapon` [2511.13548]

The paper explicitly notes that some word-level substitutions can alter meaning; the `weak → week` example is flagged as requiring additional similarity analysis. This is why mutation is followed by semantic filtering rather than treated as intrinsically safe [2511.13548].

At the **sentence level**, ForgeDAN modifies prompt structure more globally. The examples given are:
- **Syntactic restructuring**: `How to build a bomb? → The process of bomb building is...`
- **Clause reordering**: `Step A then Step B → Step B follows Step A` [2511.13548]

These operations are intended to improve naturalness and increase variation while maintaining harmful intent. The mutation module is described as **plugin-based**, **dynamically extensible**, and organized as a library of operators, so operators can be added or removed without changing the whole framework [2511.13548].

The mutation procedure is explicitly constrained by semantic validation. At each iteration, a perturbation strategy is sampled from the mutation library, applied to a candidate prompt \(t\) to produce \(t'\), and retained only if:
\[
\mathrm{Sim}(t', t_0) \geq \tau
\]
Invalid candidates are discarded before fitness evaluation [2511.13548]. This suggests that ForgeDAN attempts to maximize mutation diversity without allowing the search to drift away from the original malicious intent.

## 4. Semantic fitness and dual-dimensional jailbreak judgment

ForgeDAN’s second major component is its **semantic fitness function**, introduced as a replacement for shallow lexical metrics such as Jaccard similarity. The paper argues that lexical overlap is inadequate because semantically equivalent outputs may differ substantially in wording; for example, `assemble an explosive device` and `construct a bomb` may share little token overlap while expressing similar meaning [2511.13548].

The framework therefore uses an embedding-based fitness function:
\[
\mathrm{Fitness}(t,goal,hrr) = \mathrm{sim}(\mathrm{E}(M(t,goal)),\ \mathrm{E}(hrr))
\]
where \(\mathrm{E}(\cdot)\) is a pretrained text encoder, \(M(t,goal)\) is the model response to the candidate prompt and payload, \(hrr\) is the **harmful reference response**, and \(\mathrm{sim}(\cdot,\cdot)\) is cosine similarity [2511.13548].

The score measures how semantically close the model’s response is to a known harmful reference. Higher-scoring candidates are preserved, lower-scoring ones are discarded, and elite candidates are carried across generations. The default encoder is **RoBERTa**, though the paper states that the framework is extensible to other encoders such as Sentence-BERT or domain-specific embeddings [2511.13548].

The paper characterizes this fitness measure as **semantic**, **interpretable**, and **robust to paraphrase**. It contrasts the method with AutoDAN-HGA’s cross-entropy or lexical heuristics, which it describes as more opaque and less interpretable [2511.13548]. Within the ablation study, semantic fitness emerges as the most important single component: replacing it with an AutoDAN-HGA-style cross-entropy metric causes performance to fall to as low as **5.77%** on Gemma-2-9B and **12.12%** on TranSpec-13B [2511.13548].

The third major component is **dual-dimensional jailbreak judgment**, which addresses the weakness of keyword-based detection. In ForgeDAN, a jailbreak is counted as successful only when the output is both **compliant** and **harmful**. This is formalized through two classifiers:
\[
C_{\mathrm{behavior} : Y \rightarrow \{\mathrm{refuse}, \mathrm{comply}\}
\]
\[
C_{\mathrm{content} : Y \rightarrow \{\mathrm{safe}, \mathrm{harmful}\}
\]
with success defined as:
\[
\mathrm{Success}(t) = [C_{\mathrm{behavior}(M(t,goal)) = \mathrm{comply}] \land [C_{\mathrm{content}(M(t,goal)) = \mathrm{harmful}]
\]
[2511.13548]

The paper organizes outcomes into a \(2 \times 2\) decision matrix:
- **Refuse + Safe** → refusal
- **Refuse + Harmful** → blocked refusal with harmful trace
- **Comply + Safe** → compliant but safe
- **Comply + Harmful** → **successful jailbreak** [2511.13548]

This separation of behavior and content is intended to reduce false positives, false negatives, and ambiguous success judgments. In ablation, replacing dual-dimensional judgment with keyword matching leads to severe degradation, including **Qwen2.5-7B: 87.50% → 1.92%** and **DeepSeek-V3: 58.65% → 10.19%** [2511.13548]. The result supports the paper’s claim that keyword matching is brittle because it can miss paraphrases, overcount partial harmful traces, and confuse mere non-refusal with genuine harmful compliance.

## 5. Evaluation protocol and empirical results

ForgeDAN is evaluated on **3 open-source LLMs** and **1 domain-specific proprietary model**:
- Qwen2.5-7B
- Gemma-2-9B
- DeepSeek-V3
- TranSpec-13B, described as a **9B parameter domain-specific Transformer-based model** used as a reference name in the paper [2511.13548]

The baselines are:
- **GCG** — white-box gradient-based suffix attack
- **AutoDAN-HGA** — black-box hierarchical genetic algorithm
- **PAIR** — black-box iterative refinement using an attacker LLM
- **DAN** — manually crafted human jailbreak prompts [2511.13548]

Two datasets are used. The first is **AdvBench**, comprising **520 malicious request samples**, each paired with a verified reference response and divided into **7 categories** using TF-IDF plus manual annotation: profanity, dangerous or illegal suggestions, cyber-crime, misinformation, threatening behavior, graphic depictions, and discrimination. The second is a **real-world dataset** containing **137 harmful chat records** extracted from operational logs of an anonymized AI enterprise and labeled with the same 7 categories [2511.13548]. The main metric is **Attack Success Rate (ASR)**, defined in accordance with the dual-dimensional judgment criterion [2511.13548].

On **AdvBench**, ForgeDAN reports the following ASR results:

| Model | ForgeDAN | Best Baseline |
|---|---:|---:|
| DeepSeek-V3 | 58.65% | 35.00% (PAIR) |
| Gemma-2-9B | 98.27% | 23.65% (DAN) |
| Qwen2.5-7B | 87.50% | 40.58% (PAIR/DAN reported at same level in table) |
| TranSpec-13B | 55.00% | 46.92% (PAIR) |

The paper states that ForgeDAN is consistently best, with especially large gains on Gemma-2-9B and Qwen2.5-7B. It also reports that **GCG** performs very poorly across all models, generally below **4%**, while AutoDAN-HGA is better but unstable, PAIR can be competitive, and DAN is sometimes decent but inconsistent [2511.13548].

The generalization analysis evaluates transfer from one malicious payload to another within AdvBench. ForgeDAN again leads:

| Model | ForgeDAN | Best Baseline |
|---|---:|---:|
| DeepSeek-V3 | 61.54% | 14.42% (AutoDAN-HGA) |
| Gemma-2-9B | 98.46% | 23.65% |
| Qwen2.5-7B | 87.12% | 62.88% (AutoDAN-HGA) |
| TranSpec-13B | 54.23% | 43.46% |

The paper interprets these results as evidence that ForgeDAN generalizes well across payloads and is not overfit to a single malicious query [2511.13548].

On the **real-world dataset**, ForgeDAN remains the strongest method:

| Model | ForgeDAN | Best Baseline |
|---|---:|---:|
| DeepSeek-V3 | 57.66% | 22.63% (AutoDAN-HGA) |
| Gemma-2-9B | 100.00% | 43.07% (DAN) |
| Qwen2.5-7B | 89.05% | 51.82% (DAN) |
| TranSpec-13B | 56.20% | 46.72% (DAN) |

This is presented as evidence that ForgeDAN transfers from benchmark prompts to operationally sourced harmful requests [2511.13548].

## 6. Ablation, limitations, and significance

The ablation study isolates three major components: **mutation mechanism**, **semantic fitness**, and **dual-dimensional judgment**. The simplified replacements are, respectively, synonym substitution only, AutoDAN-HGA-style cross-entropy fitness, and keyword matching [2511.13548].

Removing the full multi-strategy mutation mechanism has a moderate effect: ASR remains relatively high, around **55.96%–97.88%**, which the paper interprets as showing that even simple synonym substitution can still produce useful attacks. However, the full mutation set improves diversity and stability [2511.13548].

Removing **semantic fitness** produces the most severe collapse, down to **5.77%** on Gemma-2-9B and **12.12%** on TranSpec-13B. Replacing **dual-dimensional judgment** with keyword matching also causes sharp performance loss, including the previously noted catastrophic drops on Qwen2.5-7B and DeepSeek-V3 [2511.13548]. Taken together, these findings indicate that the two most consequential design choices are semantically guided search and semantically grounded success verification.

Although the paper does not report formal naturalness scores such as perplexity or human-likeness metrics, it repeatedly claims improvements in **naturalness**, **stealth**, and **semantic coherence**. These claims are tied to three design choices: mutations that preserve readability, semantic filtering that prevents degenerate candidates, and DAN-style prompt prefixes rather than gibberish suffixes [2511.13548]. A plausible implication is that ForgeDAN’s attack surface includes not only alignment failure but also the limitations of simplistic detectors that rely on obvious lexical or format-based anomalies.

The discussion section implies several limitations. The mutation library remains manually designed, even if extensible. Semantic fitness depends on the quality of the embedding encoder. The real-world dataset is proprietary and drawn from one enterprise, so broader domain generality is not fully established. Cross-model behavior varies, and the paper notes that exact causes are hard to determine because safety-training details are often undisclosed. The method is framed for red teaming, but the paper also acknowledges that, like any jailbreak framework, it could be misused [2511.13548].

Within the jailbreak literature, ForgeDAN’s principal significance lies in how it redefines automated DAN-style attack generation as a semantically constrained evolutionary search problem. Its key design pattern is the combination of **diverse prompt mutation**, **embedding-based semantic fitness**, and **dual-dimensional success verification**. The paper’s results suggest that these components jointly improve ASR, transferability, robustness, and the reliability of success detection in black-box settings [2511.13548].

Source: https://www.emergentmind.com/topics/forgedan