AgentDojo: Extensible LLM Robustness Benchmark

• AgentDojo is an extensible evaluation environment designed to assess the adversarial robustness of LLM agents interacting with diverse tools and untrusted data inputs.
• It features a multi-domain suite of tasks with adaptive attack scenarios and formal utility–security metrics (BU, UA, ASR) to benchmark indirect prompt injection vulnerabilities.
• The framework supports modular extensibility for environments, tasks, and defenses, enabling rigorous, reproducible evaluations of emerging security paradigms.

AgentDojo is an extensible evaluation environment dedicated to measuring the adversarial robustness of LLM agents that interact with external tools and untrusted data sources. It operationalizes the prompt injection threat model for tool-augmented agents, providing a rigorous multi-domain suite of tasks, adaptive attack scenarios, a modular pipeline for defense integration, and formal utility–security metrics. AgentDojo has become the canonical benchmark for indirect prompt injection (IPI) research, widely used to demonstrate both limitations of current LLM agents and comparative effectiveness of emerging defense paradigms (Debenedetti et al., 2024).

1. Architecture, Modeling Assumptions, and Extensibility

AgentDojo is implemented as a Python software framework simulating AI agents operating over mutable environment states via tool APIs. Each environment—such as workspace management, banking, Slack-style messaging, or travel booking—specifies:

• Internal state: e.g., mailboxes, calendars, file systems, bank accounts.
• Tools: Python functions with named/typed signatures, registered via decorators and carrying docstring schemas for argument and return types. Typical tools include send_email, get_balance, invite_user, or book_flight.
• External data flows: All tool outputs are treated as untrusted; adversaries can embed arbitrary text—including injection payloads—at designated placeholders.

The agent loop follows these steps:

1. Receives a user prompt (the intended benign task).
2. Selects and invokes tools through the function-calling interface.
3. Updates environment state based on tool-side effects.
4. Processes resulting tool outputs, which may carry adversarial content.

Researchers define user and injection tasks as Python subclasses with fields for the prompt, utility or security checkers (Python predicates over state transitions), and ground-truth tool-call traces. Extensibility is a core design feature: environments, tools, tasks, attacks, and defenses can be programmatically composed, subclassed, and automatically integrated into the benchmarking workflow (Debenedetti et al., 2024).

2. Task Suite and Security Test Set Construction

AgentDojo's curated release includes 97 user tasks in four application domains:

Domain	#User Tasks	Example Capabilities	#Tools	#Injection Tasks	#Security Cases
Workspace	40	Email, calendar, cloud storage, document workflows	24	6	240
Slack	21	Channel operations, direct messaging, file transfer	11	5	105
Travel	20	Flight/hotel search, bookings, itineraries	28	7	140
Banking	16	Account management, bill pay, transfers, authentication	11	9	144

Each task is designed to require multi-step reasoning (typically 3–5 chained tool calls), state mutation, and dynamic tool selection. The cross-product of user and injection tasks yields 629 security evaluation cases in the canonical benchmark and up to 949 in later revisions (Zhong et al., 13 Feb 2025, Bhagwatkar et al., 6 Oct 2025).

Test cases are constructed so that every benign user goal can encounter every injection pattern in the same domain. Attackers are assumed to control specific data fields (e.g., bank transaction notes, email contents, API responses) and inject payloads into the agent’s context when tools are invoked:

• Attack vectors include "TODO" notes, "Ignore previous instructions," "System message," and "Important Messages" formats.
• Injection templates are parameterized to allow for identity spoofing, adaptive adversarial selection, and realistic context mixing (Debenedetti et al., 2024, Zhu et al., 7 Feb 2025).

3. Security Metrics and Evaluation Protocol

AgentDojo formally defines three complementary metrics:

• Benign Utility (BU):

$$\text{BU} = \frac{\#\text{benign tasks correctly completed}}{\#\text{total benign tasks}}$$

Measures the agent’s task success rate in a non-adversarial setting.

• Utility under Attack (UA):

$$\text{UA} = \frac{\#\text{attack cases with correct user output and zero malicious actions}}{\#\text{total attack cases}}$$

Combines utility preservation with security enforcement under attack.

• Attack Success Rate (ASR):

$$\text{ASR} = \frac{\#\text{attack cases where the malicious goal is executed}}{\#\text{total attack cases}}$$

These metrics are evaluated per-model and per-defense. Task success and malicious action detection rely on domain-specific state predicates: e.g., did the agent send money to an adversarial account, leak PII, or persistently alter environment state? All results are reproducible through automated environment simulators and checker scripts (Debenedetti et al., 2024, Zhu et al., 7 Feb 2025, Bhagwatkar et al., 6 Oct 2025).

4. Attack Vectors, Threat Models, and Limitations

AgentDojo comprehensively models IPI threat vectors by:

• Allowing adversaries to append deceptive instructions at any eligible untrusted data return point.
• Embedding payloads in a variety of naturalistic and obfuscated formats. The suite covers "high-priority" injections ("IMPORTANT!!!..."), ignore-previous constructs, system/user impersonation using role-based chat tokens, and data-flow–driven attacks targeting leakage of any data observed by the agent (Alizadeh et al., 1 Jun 2025, Chang et al., 26 Sep 2025).

Recent findings highlight vulnerabilities to structurally sophisticated attacks such as "ChatInject," where the adversary forges chat template tokens to elevate injected instructions to system or user priority. Empirically, ChatInject raises ASR by 26–51 percentage points (e.g., from 5.18% to 32.05% overall; 17.5% to 54.8% on Qwen-3; 0.3% to 51.4% on GPT-oss) (Chang et al., 26 Sep 2025).

Limitations in AgentDojo's earlier releases included brittle metric definitions (e.g., strict message-count increments as success, or attackers overwriting critical input fields with payloads), over-constrained attack models, and insufficient attack diversity. These have been addressed through revised benchmarks emphasizing semantic state checks, additive rather than overwriting injection, and a stronger attack catalog (adaptive chaining, encoding-based obfuscation, multi-turn persuasion) (Bhagwatkar et al., 6 Oct 2025).

5. Defenses, Design Principles, and Comparative Performance

AgentDojo is the standard evaluation platform for a wide spectrum of IPI defenses, including:

• Prompt-level Defenses: Data delimiters, prompt sandwiching, user-instruction repetition, system message reminders, and prompt-injection detection classifiers. These approaches yield marginal improvements (e.g., data delimiters reduce ASR on GPT-4o from 53.1% to 46.3%) but are inadequate against structurally sophisticated attacks (e.g., all prompt-based defenses leave ChatInject and multi-turn attacks with equal or higher ASR than no defense) (Chang et al., 26 Sep 2025, Debenedetti et al., 2024).
• Tool Filtering and Policy-based Approaches: Restrict agents to a whitelist of tools relevant to the user’s task. Reduces ASR—for instance, tool filtering achieves GPT-4o UA of 73.1% and ASR of 7.5%—but fails when user and attacker require overlapping tool sets (Debenedetti et al., 2024).
• Information-Flow Control (IFC): RTBAS and similar defenses attach integrity/confidentiality labels to all data, statically and dynamically enforcing per-tool policies. RTBAS obtains perfect integrity on all 629 AgentDojo cases with <3% loss in utility under attack (Zhong et al., 13 Feb 2025). IFC-based “Prudentia” achieves 50.1% full-autonomy rate (no HITL interventions) vs 24.3–35.5% for prior approaches, sharply reducing human-in-the-loop burden (Kolluri et al., 11 Feb 2026).
• Task Alignment Enforcement: Task Shield reframes defense as enforcing that all agent actions contribute to the explicit user goal, via LLM-based instruction extraction and alignment scoring. On AgentDojo, Task Shield yields U = 69.79% and ASR = 2.07%, substantially outperforming classical baselines (Jia et al., 2024).
• Provable and Behavioral Detection: MELON (Masked re-