GUARD-JD: Adaptive Jailbreak Diagnostics
- GUARD-JD is a jailbreak diagnostics component that transforms abstract AI guidelines into adversarial, scenario-based prompts for testing language model compliance.
- It uses a knowledge-guided adaptive role-play involving a generator, evaluator, and optimizer to create natural, low-perplexity prompts that bypass safety mechanisms.
- Empirical evaluations demonstrate GUARD-JD’s superior success rates across LLMs and its effective extension to multimodal setups, ensuring robust and transferable diagnostics.
GUARD-JD is the jailbreak diagnostics component of GUARD, a testing method for operationalizing high-level, government-issued AI guidelines into specific guideline-violating questions for LLMs. Its function is to move beyond direct refusal checking by creating adversarial “playing scenarios” that provoke unethical or guideline-violating responses, thereby identifying potential scenarios that could bypass built-in safety mechanisms. In the reported workflow, GUARD first tests direct compliance and reports inconsistencies when responses directly violate guidelines; GUARD-JD is applied to cases that do not directly violate guidelines, and the overall process culminates in a compliance report that delineates adherence and highlights violations (Jin et al., 28 Aug 2025).
1. Position within the GUARD framework
GUARD addresses a specific gap in AI governance and evaluation: governments have issued ethics guidelines to promote the development of trustworthy AI, but these guidelines are typically high-level demands for developers and testers. GUARD operationalizes such guidelines into actionable testing questions, and GUARD-JD supplies the adversarial diagnostic stage for responses that initially appear guideline-adhering. In that sense, GUARD-JD is not a standalone moderation layer; it is the component that tests whether apparent adherence survives adversarial contextualization (Jin et al., 28 Aug 2025).
The underlying premise is that guideline compliance cannot be established by direct prompting alone. A model may refuse a harmful request in isolation and still produce a guideline-violating answer once the same request is embedded in a carefully designed scenario. GUARD-JD is therefore framed as a robustness probe rather than a simple harmfulness classifier. This also differentiates it from static harmful-prompt benchmarks: the prompts are grounded in government-issued guidelines, and the diagnostics are explicitly scenario-based.
Historically, the 2024 GUARD paper introduced a role-playing system for generating natural-language jailbreakings to test guideline adherence of LLMs, together with a knowledge-graph organization of jailbreak characteristics and an automatic setting that follows government-issued guidelines to generate jailbreaks (Jin et al., 2024). The 2025 formulation extends that line into a broader compliance-testing pipeline in which GUARD-JD is the explicit jailbreak diagnostics component (Jin et al., 28 Aug 2025).
2. Formal problem formulation
GUARD-JD begins from guideline-violating questions generated from abstract guidelines. For a target LLM , the response to such a question is . The framework distinguishes between a guideline-adhering answer , exemplified by refusal, and a guideline-violating answer , exemplified by direct acceptance or implementation of the harmful request (Jin et al., 28 Aug 2025).
A central point in the methodology is that refusal is not treated as sufficient proof of robustness. Even if the model refuses a harmful prompt, adversarial context may change the outcome. GUARD-JD therefore introduces a “playing scenario” , defined as a context or adversarial wrapper designed to bypass default model safeguards. The actual prompt becomes
where denotes string concatenation.
Success is evaluated semantically rather than by exact matching. The Evaluator compares the target model’s response to a canonical refusal using
A low similarity score signals a successful jailbreak, meaning that the model has ceased to behave like the refusal oracle and has instead produced a guideline-violating answer. This formulation makes the diagnostic criterion explicitly response-based and semantically oriented, rather than keyword-based.
3. Knowledge-guided adaptive role-play
The core construction mechanism in GUARD-JD is knowledge-guided jailbreak synthesis. The framework maintains a Knowledge Graph built from analyzing and decomposing prior real-world jailbreak prompts into semantically meaningful fragments such as introduction, capabilities, and response format. Formally, the Knowledge Graph is a directed graph in which each vertex represents a property or characteristic of jailbreaks, while edges encode observed relationships such as sequence or co-occurrence and are weighted by frequency in real jailbreaks (Jin et al., 28 Aug 2025).
Within this structure, GUARD-JD uses an iterative triplet of Generator, Evaluator, and Optimizer. The Generator recombines scenario fragments from the Knowledge Graph to create coherent and natural playing scenarios. The Evaluator computes semantic similarity between the model’s answer and the canonical refusal. The Optimizer uses failed attempts to suggest modifications that reduce semantic similarity and make the scenario more adversarial. The iteration continues until the jailbreak prompt triggers a guideline violation, operationally defined by the similarity score dipping below a chosen threshold.
The earlier GUARD formulation provides additional detail on how this machinery was originally instantiated. That version used four roles—Translator, Generator, Evaluator, and Optimizer—collected 78 in-the-wild jailbreaks, decomposed them into eight key jailbreak characteristics, and used random-walk sampling over the Knowledge Graph to assemble new scenarios (Jin et al., 2024). This historical account clarifies that GUARD-JD’s adaptive role-play is not merely prompt paraphrasing; it is a structured recombination process over a curated representation of jailbreak patterns.
The framework’s reporting stage is integral to the method. For each question/scenario pair, GUARD-JD reports whether the LLM was successfully broken, specifying the scenario that defeated guideline adherence, the prompt, and the model’s actual harmful output. This makes the output diagnostic rather than purely scalar.
4. Empirical evaluation
The reported evaluation probes Vicuna-13B, LongChat-7B, Llama2-7B, Llama-3-8B, GPT-3.5, GPT-4, GPT-4o, and Claude-3.7 under three government-issued guideline regimes, identified in the detailed description as EU, UK, and NIST, and includes jailbreak diagnostics as well as compliance testing (Jin et al., 28 Aug 2025).
For jailbreak effectiveness, the paper reports that GUARD-JD outperforms all baselines, with the following success rates listed in the detailed summary.
| Model | Success rate |
|---|---|
| Vicuna-13B | 86.0% |
| LongChat-7B | 82.6% |
| Llama2-7B | 80.0% |
| GPT-3.5 | 78.6% |
| GPT-4 | 77.2% |
| GPT-4o | 70.8% |
| Claude-3.7 | 68.2% |
The same summary states that comparative baselines such as AutoDAN, PAIR, and CipherChat mostly scored lower, especially on the strongest or most aligned models. It also states that GUARD-JD’s prompts have lower perplexity, interpreted there as more natural prompts that are harder to be detected or filtered by perplexity-based defense mechanisms (Jin et al., 28 Aug 2025).
The evaluation further emphasizes transferability and robustness. GUARD-JD-generated playing scenarios exhibit robust transferability, meaning that jailbreak prompts generated for one model can largely transfer to attack other models, outperforming baselines like PAIR. They also remain effective when models are equipped with paraphrasing and self-reminder defenses, with success rates remaining above 60%, whereas baselines frequently drop to around 20%. Human evaluators rated GUARD-generated questions as highly aligned with intended guideline violations, with an average score above 4.2/5, which the paper uses as evidence for the relevance and realism of the generated scenarios (Jin et al., 28 Aug 2025).
A plausible implication is that GUARD-JD is designed not only to maximize attack success rate but also to surface realistic failure modes under naturalistic prompting. That interpretation is consistent with the joint emphasis on success rate, perplexity, transferability, defense robustness, and human alignment.
5. Multimodal extension and relation to adjacent jailbreak-evaluation work
GUARD-JD is reported to transfer jailbreak diagnostics to vision-LLMs. In that extension, playing scenarios and jailbreak prompts are adapted to cross-modal queries, such as prompting a VLM to describe a NSFW image. The detailed summary states that, in evaluation on models including MiniGPT-v2 and Gemini-1.5, GUARD-JD achieved the highest toxicity scores compared to baselines such as GCG and AutoDAN, demonstrating transfer of inductive jailbreak knowledge from LLMs to multimodal setups (Jin et al., 28 Aug 2025).
This multimodal extension is consistent with the earlier GUARD paper, which had already extended the role-playing jailbreak framework to MiniGPT-v2 and Gemini Vision Pro and reported toxicity-score gains over baselines in the vision-language setting (Jin et al., 2024). Taken together, these two papers present GUARD-JD as a mechanism that is not confined to pure text generation.
Several adjacent works clarify the broader design space. GuardVal introduces a dynamic jailbreak evaluation protocol with Translator, Generator, Evaluator, and Optimizer roles, together with an Adam-inspired optimization method to prevent stagnation during prompt refinement. Its summary explicitly states that the protocol could directly strengthen broader evaluation platforms like GUARD-JD by helping them maintain currency with adversary techniques and avoid overfitting to static benchmarks (Zhang et al., 10 Jul 2025). By contrast, G-Guard addresses a different locus of the safety problem—multi-turn jailbreak defense—through an attention-aware GNN-based input classifier that models cross-turn entity relationships and augments classification with retrieved labeled single-turn queries (Huang et al., 9 Jul 2025). Another line of work, “Super Suffixes,” shows that guard models and aligned text generators can be jointly bypassed, and proposes DeltaGuard as a countermeasure based on cosine-similarity fingerprints in residual-stream trajectories (Adiletta et al., 12 Dec 2025).
These neighboring efforts do not redefine GUARD-JD, but they place it within a rapidly evolving ecosystem of jailbreak generation, evaluation, and defense.
6. Scope, misconceptions, and terminological ambiguity
A common misconception is that a refusal response establishes compliance. GUARD-JD is explicitly motivated by the opposite claim: a model that does not directly violate a guideline may still be inducible to violate it once adversarial context is introduced. In the framework’s terminology, direct refusal is therefore a preliminary signal, not a sufficient robustness certificate (Jin et al., 28 Aug 2025).
A second misconception is that guideline adherence testing is equivalent to measuring performance on generic harmful-prompt benchmarks. GUARD-JD is distinguished by its grounding in government-issued guidelines and by its pipeline from abstract guideline extraction to attack scenario reporting. This suggests a more policy-linked notion of safety evaluation than purely synthetic or benchmark-driven red-teaming.
The acronym itself is also ambiguous in the recent literature. In “GUARD:Dual-Agent based Backdoor Defense on Chain-of-Thought in Neural Code Generation,” “GUARD-JD” refers to GUARD-Judge, the first agent in a dual-agent defense framework for CoT backdoor attacks. There, GUARD-JD is a detection module implemented with DeepSeek-R1 that analyzes Chain-of-Thought steps through correctness evaluation and patterns-based anomaly detection, and it feeds suspicious samples to GUARD-Repair for secure regeneration (Jin et al., 27 May 2025). That usage is distinct from the jailbreak-diagnostics meaning in the guideline-adherence literature.
Within the LLM safety literature centered on guideline adherence, however, GUARD-JD denotes the adaptive jailbreak diagnostics component of GUARD: a knowledge-guided, role-playing method for converting apparently compliant behavior into a stronger test of whether compliance persists under adversarial scenario construction (Jin et al., 28 Aug 2025).
