D-CIPHER: Multi-Agent Cybersecurity Framework

• D-CIPHER is a collaborative multi-agent system that employs a Planner–Executor–Auto-prompter architecture to decompose and tackle complex CTF challenges.
• It enhances long-horizon problem solving by dynamically coordinating specialized agents and managing context with structured memory and tailored tool usage.
• Evaluations report state-of-the-art performance on benchmarks like NYU CTF Bench, Cybench, and HackTheBox, with significant efficiency gains over prior systems.

D-CIPHER denotes a collaborative, multi-agent LLM framework for autonomous offensive-security problem solving, specifically designed for complex cybersecurity Capture the Flag challenges. It operationalizes a Planner–Executor architecture augmented with an Auto-prompter agent, coordinates agent interaction through function-calling tools and a shared containerized challenge environment, and is intended to address the limitations of single-agent reasoning–action loops on long-horizon, multi-step security tasks. The framework was introduced for collaborative CTF solving and reported state-of-the-art benchmark figures of 22.0% on NYU CTF Bench, 22.5% on Cybench, and 44.0% on HackTheBox, with improvements over prior systems such as EnIGMA (Udeshi et al., 15 Feb 2025).

1. Name, referent, and disambiguation

In current arXiv usage, the primary referent of D-CIPHER is the offensive-security framework introduced in "D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security" (Udeshi et al., 15 Feb 2025). In that usage, the term names a multi-agent LLM system for autonomous CTF solving.

The label is, however, non-unique across technical domains. "D-CIPHER: Discovery of Closed-form Partial Differential Equations" uses the same acronym for a variational symbolic-discovery framework for PDEs and higher-order ODEs, rather than for cybersecurity or agent systems (Kacprzyk et al., 2022). Separately, "CIPHER: Counterfeit Image Pattern High-level Examination via Representation" does not formally define a separate algorithm named D-CIPHER; its details state that when “D-CIPHER” appears in discussion, it should be treated as an informal shorthand for the diffusion-feature component or a diffusion-emphasized variant of CIPHER, and that no distinct D-CIPHER detector is separately evaluated at inference (Kim et al., 31 Mar 2026). The earlier paper "A general cipher for individual data anonymization" proposes a cipher formalized as the three-tuple $T=(P,K,E)$, but does not introduce a special acronym D-CIPHER (Ruiz, 2017).

Accordingly, in the offensive-security literature, D-CIPHER refers specifically to the Planner–Executor–Auto-prompter system of (Udeshi et al., 15 Feb 2025), whereas in other literatures the same string either denotes a different method entirely or is not a formal method name at all.

2. Problem setting and motivation

D-CIPHER is motivated by the difficulty of jeopardy-style CTF challenges for single-agent LLM systems. These tasks require long-term exploration, precise multi-step execution, and sustained focus across heterogeneous domains including crypto, rev, pwn, web, forensics, and misc. The motivating claim is that single-agent setups typically rely on a single reasoning–action loop with self-reflection, and that as task length grows they exhibit context bloat, loss of focus, tool overuse, hallucinations, repeated unproductive exploration, and token exhaustion (Udeshi et al., 15 Feb 2025).

The framework is therefore positioned as an attempt to emulate real-world CTF team dynamics. The motivating analogy is explicit: actual competition performance depends on collaboration among specialists, iterative critique, and decomposition of problems into subtasks. D-CIPHER translates that pattern into an LLM-agent design by separating global planning from local execution and by inserting dynamic feedback loops between them.

The challenge model emphasized by the framework includes several distinct failure sources for autonomous security analysis. These include long-horizon planning and decomposition of subtasks, effective tool usage under interface constraints, dynamic adaptation to errors and observations, pruning of failed paths under budget and round constraints, robust context and memory management for large command outputs and binary dumps, and collaboration with iterative feedback. This suggests that the framework’s central claim is architectural rather than merely prompt-engineering-based: the paper treats the single-agent loop itself as the bottleneck for complex offensive-security workflows.

3. Planner–Executor system and interaction model

D-CIPHER comprises three agent types that coordinate via function calls and share a containerized challenge environment. The architecture is intentionally asymmetric: the Planner controls global problem solving, Executors handle delegated subtasks with focused context, and the Auto-prompter performs initial exploration to seed the Planner with a challenge-specific prompt (Udeshi et al., 15 Feb 2025).

Component	Core role	Key constraints or capabilities
Planner	Global exploration, plan formation, delegation, final submission	Has RunCommand; holds SubmitFlag and Giveup; lacks CreateFile, Disassemble, Decompile
Executor	Single-task specialist for delegated work	Fresh conversation per task; can use RunCommand, CreateFile, Disassemble, Decompile; returns FinishTask summary
Auto-prompter	Upfront environment exploration and prompt generation	Uses GeneratePrompt; unaware of Planner–Executor architecture

The shared environment is a Linux container with network access to challenge servers and the internet for installing packages. Tool functions include RunCommand, CreateFile, Disassemble, Decompile, SubmitFlag, and Giveup; coordination is expressed through GeneratePrompt, Delegate, and FinishTask. Category-specific guidance may be embedded in initial prompts, with examples such as RsaCtfTool for crypto and nikto for web being invoked via RunCommand.

The Planner–Executor division is enforced rather than advisory. The Planner is permitted to explore the environment with RunCommand, but it is denied CreateFile, Disassemble, and Decompile specifically to prevent it from trying to solve delegated tasks itself. Executors, by contrast, are instantiated per delegated task with a fresh conversation history and are optimized for focused execution rather than for maintaining a global strategy.

Communication is mediated exclusively by the Planner. Executors do not share state directly; they return concise execution and result summaries through FinishTask, which the Planner consumes while retaining the full history of plans, delegated tasks, and summaries. A plausible implication is that D-CIPHER treats centralization as a mechanism for coherence, even though that same design can later become a bottleneck when continuity across large artifacts or scripts is required.

4. Prompting, memory, and dynamic feedback loops

D-CIPHER uses ReAct-style prompting and LLM-native function-calling APIs. In each round, an agent emits a reasoned message plus a function call; the resulting tool output is appended to that agent’s history. Dynamic feedback is then created through iterative cycles in which the Planner revises plans based on Executor summaries, retries tasks if summaries are missing or execution stalls, and continues after wrong flag submissions rather than terminating immediately (Udeshi et al., 15 Feb 2025).

The Auto-prompter has a specialized role. It is intentionally unaware of the Planner–Executor architecture, explores the environment with basic tools, and after several exploration turns generates a challenge-specific initial prompt through GeneratePrompt. If it fails to invoke GeneratePrompt after retries, the system falls back to a category-specific hard-coded template. The reported rationale is that the Auto-prompter can produce highly relevant initialization informed by environment details, whereas templates provide only generic hints.

Memory management is highly structured. The Planner retains the entire history of plan states, delegated tasks, and returned summaries. Executors, however, see only their own local history, truncated to the last 5 action–observation pairs, with observation text capped at 25,000 characters. This is presented as a mechanism for maintaining focus under long outputs and preventing context blow-up. The overall design therefore combines global long-term memory at the Planner level with deliberate local short-term memory at the Executor level.

Stopping conditions are budget- and round-aware. For the Planner, termination occurs on correct SubmitFlag, Giveup, maximum rounds, or maximum budget. For Executors, termination occurs on FinishTask, maximum rounds, or maximum budget; if the maximum number of rounds is reached, the framework prompts once more for FinishTask, and if that still fails the Planner receives a warning. For the Auto-prompter, stopping occurs on GeneratePrompt, maximum rounds, or maximum budget, with fallback to templates if no prompt is generated.

The default execution control reported for each challenge run is a budget of $3, temperature $T=1.0$ for each agent, maximum rounds of 5 for the Auto-prompter, 30 for the Planner, and 100 for each Executor, with Executor history truncated to the last 5 action–observation pairs. The framework is implemented as open-source code under nyuctf_multiagent at https://github.com/NYU-LLM-CTF/nyuctf_agents.

5. Evaluation protocol and benchmark results

The evaluation centers on three benchmarks: NYU CTF Bench with 200 challenges, Cybench with 40 challenges evaluated in unguided mode using “hard prompt” descriptions, and HackTheBox with 50 challenges. NYU CTF Bench is broken down as crypto 53, forensics 15, pwn 38, rev 51, web 19, and misc 24; Cybench as crypto 16, forensics 4, pwn 2, rev 6, web 8, and misc 4; and HackTheBox as crypto 30 and rev 20 (Udeshi et al., 15 Feb 2025).

The primary metric is the percentage of challenges solved. A challenge is marked solved if the Planner submits the correct flag, or if the correct flag appears anywhere in any agent’s conversation, which is intended to avoid false negatives when an Executor or Auto-prompter discovers the correct flag but the Planner fails to submit it. The secondary metric is average cost for solved challenges, computed as the sum of all agent API costs divided by the number of solved challenges. The experimental protocol also includes category-wise analyses, ablations, model-mixing studies, and temperature sensitivity, and uses a separate development set of 55 additional NYU-style challenges to tune prompts and features.

Configuration	NYU CTF Bench	Cybench	HackTheBox
D-CIPHER, Claude 3.5 Sonnet	19.0%	22.5%	44.0%
D-CIPHER, GPT-4o	10.5%	12.5%	16.0%
D-CIPHER w/o Auto-prompter, Claude 3.5 Sonnet	22.0%	20.0%	44.0%
EnIGMA	13.5%	20.0%	26.0%

The paper reports that D-CIPHER consistently outperforms EnIGMA overall across the three benchmarks, with the largest reported margin on HackTheBox: 44.0% versus 26.0%. Category-wise highlights on NYU CTF Bench for Claude 3.5 Sonnet are 15.4% in crypto versus EnIGMA’s 7.7%, 29.4% in rev versus 17.7%, and 25.0% in misc versus 16.7%; web improves but remains low at 5.3%, while pwn is lower than EnIGMA with the Auto-prompter enabled and improves to 28.2% without the Auto-prompter.

The cost analysis states that D-CIPHER typically achieves lower average cost than EnIGMA, often 2×–10× lower while solving more challenges, with GPT-4o and GPT-4 Turbo cited as examples across benchmarks. One exception is Claude 3.5 Sonnet on NYU CTF Bench, where D-CIPHER shows moderately higher average-solved cost than EnIGMA while still solving more challenges. The paper does not report significance tests or confidence intervals.

A notable textual inconsistency exists in the source description. The abstract states that the authors manually map the CTFs in NYU CTF Bench to MITRE ATT&CK techniques and that D-CIPHER solves 65% more ATT&CK techniques than previous work, but the detailed evaluation summary states that MITRE ATT&CK mapping is not reported in the paper and that evaluation centers on CTF challenge success by category. The two statements coexist in the provided record (Udeshi et al., 15 Feb 2025).

6. Ablations, failure modes, and research significance

The ablation studies are central to interpreting what D-CIPHER contributes beyond ordinary tool-augmented prompting. Removing the Auto-prompter on NYU CTF Bench with Claude 3.5 Sonnet increases performance by 3 percentage points in absolute terms, from 19.0% to 22.0%, driven by more than a twofold improvement in pwn. However, performance worsens on GPT-4o and on Cybench, and average cost increases, leading the paper to characterize the Auto-prompter as generally beneficial for efficiency and performance except in some pwn tasks (Udeshi et al., 15 Feb 2025).

Removing the Planner and using Auto-prompter plus a single Executor end-to-end reduces NYU CTF Bench performance by 1% to 5% absolute across models, despite roughly halving cost. This is presented as evidence that Planner–Executor separation and coordinated decomposition are responsible for performance gains rather than mere parallel tool use. Model-mixing experiments reinforce the same conclusion: a strong Planner paired with a weak Executor degrades performance across pairings, such as Sonnet+Haiku at 13.0% versus Sonnet+Sonnet at 19.0%, GPT-4o+Mini at 6.5% versus 10.5%, and LLaMA 3.1 405B combined with LLaMA 3.3 70B failing entirely. Lowering GPT-4o temperature from 1.0 to 0.95 also reduces success from 10.5% to 9.0%, with drops in crypto, pwn, and rev and no improvements elsewhere.

The qualitative case studies show both strengths and limitations. In the forensics challenge 1black0white, the Planner delegates file examination and conversion, Executors return concise summaries, and the Planner composes the steps needed to extract the flag; the paired Auto-prompter analysis in the same challenge shows the value of a tailored prompt over generic templates. Conversely, pwn case studies such as slithery, unlimited_subway, got_milk, bigboy, and baby_boi show the Auto-prompter being derailed by misleading artifacts, incomplete context extraction, or unhandled tool errors, producing insufficient or incorrect prompts. The comparison with EnIGMA is also mixed: in collision_course D-CIPHER fails after multiple Executors repeatedly re-examine large files and lose focus, whereas EnIGMA succeeds by concentrating on a single scripting task; in gibberish_check D-CIPHER succeeds via structured static and dynamic analysis across Executors while EnIGMA fails after guesswork.

The most persistent reported weaknesses are in web tasks, function-calling reliability, tool support gaps, hallucinations, and architectural centralization. Web remains relatively weak, with Claude 3.5 Sonnet showing high giveup rates. LLaMA models hallucinate or misformat tool calls, while Gemini sometimes invokes non-existent functions such as “decode” and “strip” or misuses interactive tools such as radare2 in non-interactive settings. Dependencies or unsupported tools, including gdb via Python wrappers, may produce confusing errors from which agents struggle to recover. Hallucinations of non-existent servers or challenge artifacts are also observed. Because Executors cannot coordinate directly and all communication is Planner-mediated, continuity can be lost on large artifacts or scripts; the paper explicitly notes Executor-to-Executor synchronization under Planner oversight as a possible direction for future work.

In practical terms, the reported deployment guidance is to use comparable-capability LLMs for Planner and Executors, keep the Auto-prompter enabled for most categories but disable it selectively for some pwn tasks if exploratory errors are observed, monitor budget and round limits, ensure dependencies can be installed in the container, extend tool coverage for web and interactive reverse engineering, and add demonstrations to reduce tool-interface confusion. Within the offensive-security agent literature, D-CIPHER is therefore significant less as a general autonomous pentesting solution than as a concrete architectural argument: task decomposition, heterogeneity of executors, and structured feedback loops can improve long-horizon CTF performance, but the gains remain conditional on model quality, tool reliability, and the interaction between prompt initialization and domain-specific task structure.