InjecAgent Benchmark: Indirect Prompt Injection

Updated 6 July 2026

InjecAgent is a benchmark designed to evaluate indirect prompt injection attacks that redirect tool-integrated LLM agents using untrusted tool outputs.
It comprises 1,054 test cases across two attack categories—direct harm and data stealing—measuring success rates to assess attack impact.
Research utilizing InjecAgent has advanced defenses like planner-plus-verifier methods and token sanitizers that drastically lower attack success rates.

InjecAgent is a benchmark for indirect prompt injection in tool-integrated LLM agents. It is described as an environment, dataset, and evaluation protocol for assessing whether untrusted tool outputs can divert an agent’s control flow toward attacker-specified actions, especially direct harm and private-data exfiltration (Chang et al., 26 Sep 2025). Subsequent work has also treated it as a standard benchmark suite for evaluating control-flow integrity in the face of indirect prompt injection attacks, and as a primary testbed for architectural, runtime, and training-time defenses for tool-using agents (Li et al., 29 Apr 2025).

1. Definition and benchmark composition

InjecAgent is repeatedly characterized as a benchmark specifically designed for indirect prompt injection attacks on tool-integrated or application-augmented LLM agents (Chang et al., 26 Sep 2025, Gong et al., 11 Apr 2026). Its central scenario is that a benign user request legitimately triggers a user-facing tool, but the returned tool output contains attacker-controlled instructions that attempt to redirect the agent toward an unauthorized task.

Most descriptions report that InjecAgent comprises 1,054 test cases, 17 user tools, and 62 attacker tools, and distinguishes two attack intentions: Direct Harm and Data Stealing (Chang et al., 26 Sep 2025). The same two-category decomposition appears in later defense work, which describes 510 direct-harm tasks and 544 data-stealing tasks (Liu et al., 19 Apr 2026). ACE, which ports the benchmark into its own app abstraction, describes the ported benchmark as 17 user apps and 52 attacker apps, while preserving the same total of 1,054 test cases and the same two attack categories (Li et al., 29 Apr 2025). This discrepancy appears in secondary descriptions of benchmark instantiations rather than as a new task definition.

The benchmark’s role in the literature is narrow but precise. It is not a generic agent benchmark; it is an indirect-prompt-injection benchmark whose success condition is tied to malicious tool use. That emphasis has made it a common substrate for papers studying control-flow hijacking, data exfiltration, and the interaction between untrusted tool outputs and instruction-following models (Shahriar et al., 7 Oct 2025).

2. Threat model and evaluation semantics

A widely used formalization models the agent as $L$ , equipped with tools $\mathcal{T}$ . A user $u$ issues an instruction $I_u$ , requiring a tool call $T_u \in \mathcal{T}$ . The agent receives a tool response $R_{T_u}$ , within which an attacker has embedded a malicious instruction $I_a$ . The attacker’s goal is to manipulate $L$ into executing $I_a$ , often through another tool $T_a \in \mathcal{T}$ ; an attack is successful if the agent executes $\mathcal{T}$ 0 (Chang et al., 26 Sep 2025). In InjecAgent, that execution is operationalized as the benchmark-specific malicious tool sequence for the given test case.

PlanGuard restates the same setup in action-sequence terms. If the trusted user instruction is $\mathcal{T}$ 1, the attacker injects payload $\mathcal{T}$ 2 into untrusted context $\mathcal{T}$ 3, and the agent $\mathcal{T}$ 4 with tools $\mathcal{T}$ 5 acts on the effective instruction set $\mathcal{T}$ 6, then the attack succeeds when a malicious target action $\mathcal{T}$ 7 appears in the executed sequence $\mathcal{T}$ 8 (Gong et al., 11 Apr 2026). This makes InjecAgent fundamentally an execution benchmark rather than a text-generation benchmark: the decisive event is a tool invocation or harmful action, not merely unsafe text in the assistant’s reply.

Several papers emphasize that InjecAgent measures Attack Success Rate (ASR) using the benchmark’s own procedures (Chang et al., 26 Sep 2025). One explicit criterion states that an attack is considered successful when the agent fully executes all steps specified in the injected task. SafeAgent further decomposes the data-stealing category into stage-wise metrics, reporting $\mathcal{T}$ 9 for data extraction success and $u$ 0 for data transmission success, which sharpens the benchmark’s use for multi-stage exfiltration analysis (Liu et al., 19 Apr 2026).

3. Attack methodologies studied on InjecAgent

Because InjecAgent embeds the attacker inside tool outputs, it has become a natural benchmark for studying increasingly structured forms of indirect prompt injection. The original plain-text threat model has been extended by attacks that abuse chat templates, persuasive dialogue structure, and model-specific role markers. ChatInject shows that wrapping malicious payloads in native chat-template syntax substantially increases compromise rates on InjecAgent: average ASR rises from 15.13% for traditional plain-text attacks to 45.90% for chat-template-based attacks, and to 52.33% for multi-turn template-based attacks (Chang et al., 26 Sep 2025). The same work reports strong per-model gains, including single-turn ChatInject ASR of 57.3 on GLM‑4.5 and 67.4 on Kimi‑K2, and multi-turn ChatInject ASR of 88.3 on Llama‑4 and 71.5 on GLM‑4.5.

Adaptive red-teaming produces an even sharper picture of benchmark difficulty. PISmith treats prompt injection generation as a reinforcement-learning problem and reports near-saturating success on InjecAgent against defended open- and closed-source models. On InjecAgent, PISmith attains ASR@10 = 1.0 for Meta-SecAlign-8B, GPT‑4o‑mini, GPT‑4.1‑nano, and GPT‑5‑nano, with ASR@1 of 0.99, 0.99, 1.0, and 0.95, respectively (Yin et al., 13 Mar 2026). This suggests that static or search-based attacks understate the adaptive attack surface of the benchmark.

Other work has reused InjecAgent’s attack goals while altering the environment. WebTrap takes the data stealing target set from InjecAgent and embeds it in a long-horizon file-browser environment, arguing that the original InjecAgent setting has task complexity 2–3 steps and that its simulated-output evaluations do not accumulate observations during navigation (Liu et al., 8 May 2026). In that extended environment, baseline “Generic Injection (InjecAgent)” and “Enhance Injection (InjecAgent)” achieve 0.00% ASR-E, whereas WebTrap reaches 55.00% ASR-E and 70.00% ASR-I in best-of-three evaluation, indicating that the benchmark’s original short-horizon setting may suppress attack patterns that emerge in longer workflows (Liu et al., 8 May 2026).

4. Defensive use and architectural reinterpretation

InjecAgent is also a central benchmark for defenses that try to restore control-flow integrity. ACE restructures tool-using systems into Abstract–Concrete–Execute phases and reports 100% security success on the full 1,054 InjecAgent cases, with security success defined as no attacker application is invoked (Li et al., 29 Apr 2025). In its best model combination, ACE reports 82.1% matching success, 91.7% execution success, and 75.2% overall utility success on InjecAgent, which makes the benchmark useful not only for security claims but also for observing the utility cost of stronger architectural isolation.

PlanGuard proposes a training-free planner-plus-verifier defense and reports that it reduces overall ASR on InjecAgent from 72.8% to 0%, while maintaining a False Positive Rate of 1.49% (Gong et al., 11 Apr 2026). In its detailed breakdown, the unprotected agent records 56.90% ASR on Direct Harm and 88.67% ASR on Data Stealing, whereas both a rule-only planner gate and the full two-stage verifier reduce ASR to 0.0% on both subsets (Gong et al., 11 Apr 2026). SafeAgent, a runtime controller with a context-aware decision core, likewise reports S1 = 0.0000 and S2 = 0.0000 on InjecAgent’s data-stealing tasks, together with Total = 0.0000, while reducing Direct Harm to 0.3157 (Liu et al., 19 Apr 2026).

Training-time defenses have also been evaluated on InjecAgent. DRIP reports that on the benchmark’s enhanced setting, LLaMA‑8B drops from 64.2% ASR to 0.5%, and Mistral‑7B from 30.3% to 1.5%, while remaining comparable to or better than several prompt-injection baselines in general utility benchmarks (Liu et al., 1 Nov 2025). CommandSans, a token-level sanitizer that removes AI-directed instructions from tool outputs, reports that on GPT‑4 under InjecAgent Enhanced, total ASR drops from 46.4% to 4.6%, and on GPT‑4o from 3.5% to 1.1% (Das et al., 9 Oct 2025). A simpler firewall-style sanitizer reduces GPT‑4o Enhanced ASR to 0.0% on InjecAgent, but the same paper argues that this says as much about benchmark weakness as about defensive strength (Bhagwatkar et al., 6 Oct 2025).

Runtime governance work uses InjecAgent differently. The Layered Governance Architecture reports 99–100% interception when generalizing its Layer 2 intent verifier to external InjecAgent samples (Ge, 7 Mar 2026). NeuroTaint, which focuses on provenance rather than prevention, reports near-perfect unsafe-action detection on InjecAgent-base and InjecAgent-enhanced, while arguing that the benchmark’s labels do not capture source-to-sink semantic and causal propagation in the way its own TaintBench does (Cai et al., 25 Apr 2026).

5. Extensions, reuse, and derivative benchmarks

Although InjecAgent began as a tool-output benchmark, later work has reused it as an attack corpus and as a design reference. IPI-proxy harvests InjecAgent payload strings into a unified library of 820 deduplicated attack strings extracted from six benchmarks, then injects them into live HTML from whitelisted domains through an intercepting proxy (Chia-Pei et al., 12 May 2026). In that framework, InjecAgent is classified as a static benchmark whose adversarial input source is mock tool returns and whose target surface is a tool agent, contrasting with IPI-proxy’s live HTTP rewriting model (Chia-Pei et al., 12 May 2026).

Coverage-audit work places InjecAgent in a broader matrix of agentic threats. In a $u$ 1 Target × Technique audit, InjecAgent is mapped to exactly two cells: System Tool Hijacking × Indirect Injection and Information Exfiltration × Indirect Injection, both with full coverage (Iyer et al., 14 May 2026). The same audit describes InjecAgent’s direct-harm half as 30 attacker cases × 17 user cases and its data-stealing half as 32 attacker cases × 17 user cases, showing that the benchmark’s breadth lies within a single mechanism family rather than across multiple attack techniques (Iyer et al., 14 May 2026).

The benchmark has also inspired more structural reinterpretations. OpenClaw’s privilege-separation study does not evaluate on InjecAgent itself, but its core recommendation—a reader agent that sees raw untrusted content and an actor agent that holds privileged tools—directly addresses the same attack channel that InjecAgent operationalizes (Cheng et al., 13 Mar 2026). Related work on memory injection shows an adjacent threat class in which poisoned long-term memory, rather than immediate tool output, becomes the carrier of malicious demonstrations; this does not redefine InjecAgent, but it suggests a plausible direction for future benchmark variants involving asynchronous persistence (Dong et al., 5 Mar 2025).

6. Limitations, controversies, and benchmark significance

The most explicit critique of InjecAgent argues that the benchmark is now too easy for modern models plus simple defenses. A firewall-based evaluation reports near-zero ASR across several models and states that InjecAgent reports only ASR and does not provide any metric for measuring utility, making it impossible to estimate benign utility or utility under attack (Bhagwatkar et al., 6 Oct 2025). The same analysis describes the attacks as overly simplistic, notes that some “enhanced” attacks are paradoxically weaker than base attacks on modern models, and highlights ambiguous cases where a tool instruction may look benign without additional policy context (Bhagwatkar et al., 6 Oct 2025).

The benchmark-coverage audit reaches a different but complementary limitation: InjecAgent is strong within its niche but narrow outside it. In the audit’s $u$ 2 matrix, InjecAgent covers 2 of 24 cells, does not cover Service Disruption, does not cover Model Internals, and does not address Safety / Alignment Bypass outside tool hijacking and exfiltration (Iyer et al., 14 May 2026). This suggests that InjecAgent should be interpreted as a specialized benchmark for agentic indirect injection, not as a comprehensive benchmark for LLM security.

At the same time, InjecAgent’s persistence across the literature indicates that its specialization remains valuable. It is one of the few widely reused benchmarks whose success condition is explicitly tied to unauthorized tool execution rather than textual misbehavior, and it has been used by architectural defenses, runtime monitors, training-time aligners, red-team frameworks, coverage audits, and long-horizon attack extensions (Li et al., 29 Apr 2025, Liu et al., 8 May 2026). The benchmark’s continued use suggests that it has become a reference point for a particular question: whether a tool-using LLM agent can maintain control-flow integrity when untrusted external content is allowed to speak in the same prompt context as trusted instructions.