TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking

Abstract: The rise of LLM agents introduces a new threat by enabling planning, coding, and even end-to-end execution of expert-level attack workflows. However, this threat remains underexplored and underestimated since (i) safety alignment prevents LLMs from directly generating harmful instructions, and (ii) most existing jailbreak methods cannot consistently induce agents to execute malicious operations. In this paper, we propose TRACE, a practical agentic jailbreaking framework to further reveal the risks of this threat surface. To conceal the malicious intent, TRACE decomposes a malicious task into multiple subtask sequences under different schemes and selects the sequence with the fewest explicitly harmful subtasks. TRACE then disguises the remaining harmful subtasks as benign-looking instructions by embedding them in task-aware scenarios with related roles, environments, directives, and heuristics. The scenarios are iteratively evolved through well-defined transformation actions, which are sampled by a Q-learning-inspired mechanism, for inducing the agent to execute on the harmful subtasks. Extensive evaluations on AgentHarm and AdvCUA show that TRACE consistently outperforms existing jailbreak baselines across multiple advanced LLM agents, achieving up to 100% bypass rate and 0.73 average success score. We also demonstrate the effectiveness of TRACE in controlled cyberattack instances. Our code and demos are available at https://github.com/ZJU-LLM-Safety/TRACE.git.

• The paper introduces TRACE, a framework that decomposes tasks into semantic subtask sequences, reducing detectability while maintaining adversarial intent.
• It employs adaptive scenario evolution with Q-learning to refine subtasks, achieving high attack success (ASS up to 0.73) and bypass rates (BR up to 1.0).
• The study reveals vulnerabilities in LLM agent architectures, emphasizing the need for advanced trajectory-level defenses and monitoring.

TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking

Motivation and Problem Statement

The proliferation of LLM-based autonomous agents has enabled advanced multi-stage planning, tool usage, and end-to-end workflow execution. These agentic capabilities result in a novel attack surface: adversaries endowed with only high-level intent can now induce LLM agents to orchestrate expert-level cyberattacks. Traditional safety alignment architectures in LLMs suppress direct generation of harmful instructions, and prevailing jailbreak methods predominantly elicit unsafe responses via one-shot/few-shot prompts but fail to reliably achieve sustained execution of malicious operations. The paper addresses this gap by introducing TRACE, a framework designed to maximize adversarial task completion in agentic settings, even under strong alignment constraints.

TRACE Framework Architecture

TRACE implements a three-stage attack pipeline: task decomposition, task-aware subtask induction, and feedback-driven self-evolution. This structured approach decomposes overtly harmful objectives into semantically consistent subtask sequences, selectively minimizes explicit risk exposure at each stage, and systematically disguises remaining unsafe subtasks via context-rich scenarios. Each scenario is instantiated from component pools—roles, environments, directives, and heuristics—conditioned on available execution tools.

Figure 1: The three-stage architecture of TRACE, illustrating the pipeline from task decomposition to adaptive scenario refinement.

Execution feedback is tightly integrated: scenario transformations are adaptively selected via a Q-learning-inspired policy, iteratively optimized per observed agent responses. Memory modules store successful trajectories and components, enhancing efficiency and transferability across subsequent attacks.

Detailed Algorithmic Innovations

Task Decomposition

TRACE builds candidate subtask sequences for a malicious task, enforcing semantic consistency using strong LLM judges. Harmfulness scores filter subtask sequences, yielding a minimal set of overtly risky operations. The decomposition retains adversarial fidelity but substantially lessens detection and refusal probability in intermediate steps.

Task-Aware Subtask Induction

Unsafe subtasks are reformulated within rich scenario contexts to obfuscate their intent. Components—role, environment, directive, heuristic—are dynamically instantiated, emphasizing compatibility with agent tooling and maximizing plausibility.

Adaptive Scenario Evolution

The feedback-driven self-evolution process maintains a transition matrix over possible transformation actions (component modifications). Actions are sampled via a softmax policy governed by adaptive temperature scheduling. Execution trajectories are scored to reflect completion progress and refusal, and local score improvements drive Q-learning updates to the transition matrix. Memory buffers cache effective scenarios and components for rapid reuse across similar subtasks, with semantic embedding-based retrieval mechanisms.

Empirical Evaluation and Numerical Highlights

TRACE is evaluated on three state-of-the-art LLM agents (GPT-5.2, Gemini-3-Flash, DeepSeek-V4-pro) across two agentic security benchmarks: AgentHarm and AdvCUA. The attack efficacy is quantified via Average Success Score (ASS) and Bypass Rate (BR):

• TRACE achieves ASS up to 0.73 and BR up to 1.0 on AgentHarm, markedly outperforming the strongest baseline (ASS 0.59, BR 0.87).
• On AdvCUA (a stronger benchmark), TRACE attains ASS 0.50 and BR 1.0 on DeepSeek agent, while baselines plateau at ASS 0.27, BR 0.62.
• The completion survival curves reveal that TRACE sustains high task completion rates as thresholds increase, signifying superior end-to-end multi-step malicious execution.

  Figure 2: TRACE’s completion survival curves demonstrate its capacity to induce consistently high-quality, sustained harmful execution compared to baselines.

Scenario reuse and memory hit rates exhibit efficient trajectory exploitation: memory utilization rapidly saturates above 0.5 and peaks above 0.6 as evolution proceeds, indicating effective exploitation of accumulated attack knowledge.

Figure 3: Increasing memory reuse rates during sequential optimization highlight how TRACE leverages prior effective scenarios for enhanced attack induction.

Practical Cyberattack Instances

TRACE’s practical efficacy is validated in controlled environments requiring complex multi-stage reasoning and tool invocation. Illustrative cases include stack-based control manipulation and cryptanalytic key compromise. TRACE decomposes tasks, selectively disguises overtly harmful subtasks, and dynamically adapts scenario components to circumvent alignment defenses, successfully performing operations such as buffer overflow exploitation and cryptanalysis.

Figure 4: Demonstration of TRACE orchestrating a cyberattack via agentic workflow in a controlled setting.

Comparative Analysis with Baselines

Qualitative examples further emphasize TRACE’s superiority in preserving and executing adversarial intent, where other baselines either achieve partial bypass or fail under agentic settings.

Figure 5: Trajectories of jailbreak prompts and execution, contrasting TRACE with other methods for AdvCUA with Gemini Agent.

Figure 6: Execution trajectories for AgentHarm with Gemini underpin TRACE’s advantage in semantic consistency and workflow preservation.

Defensive Considerations

The robustness of TRACE is assessed against prompt-level and agent-level defense mechanisms (Perplexity filtering, LlamaFirewall, AgentDoG). Although defenses reduce ASS (drops up to 0.36 on DeepSeek under AgentDoG), TRACE retains significant attack efficacy (ASS remaining at 0.45–0.48 under strongest defenses), thereby revealing limitations in contemporary guardrail architectures and underscoring the need for advanced, trajectory-aware defensive strategies.

Theoretical Implications and Future Directions

TRACE elucidates the vulnerability of LLM-based agent architectures to adaptive, context-rich adversarial task induction. The framework’s modular decomposition, scenario adaptation, and reinforcement-driven evolution highlight the inadequacy of conventional alignment and prompt-based defenses. The results advocate for research on trajectory-level monitoring, fine-grained semantic consistency verification, and agent-aware intervention strategies. On a theoretical front, this work motivates studies in adversarial planning under sequential decision constraints and memory-driven attack trajectory optimization.

Conclusion

TRACE establishes a rigorous, adaptive approach to agentic jailbreaking, leveraging decomposition and scenario induction complemented by reinforcement-guided self-evolution. The empirical results demonstrate state-of-the-art attack effectiveness and reveal substantial gaps in current defense mechanisms. The methodological advances have implications for both offense and defense in agentic AI systems, mandating further exploration into robust agent alignment and trajectory-level protection strategies.

Figure 7: Complete workflow example of TRACE inducing an AdvCUA attack using DeepSeek as the underlying agent.