Agent Data Injection Attacks are Realistic Threats to AI Agents

Abstract: AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI). Its most well-studied category is instruction injection, where attacker-controlled untrusted data is interpreted as an instruction. In response, many mitigations have been proposed to prevent instruction injection attacks. In this paper, we introduce a new category of IPI, agent data injection attacks (ADI). ADI injects malicious data disguised as trusted data, such as security-critical metadata (e.g., resource identifiers or data origins) or agent context data (e.g., tool call and response formats). As a result, agents unknowingly execute unintended actions based on attacker-controlled data. ADI has similar attack impacts as instruction injection attacks, because it causes agents to misbehave and execute unintended actions. Despite the similar impact, ADI remains underexplored and easily bypasses existing IPI defenses. We found several critical vulnerabilities in real-world agents that allow an attacker to launch various attacks: arbitrary click attacks on web agents (Claude in Chrome, Antigravity, and Nanobrowser), and remote code execution and supply-chain attacks on coding agents (Claude Code, Codex, and Gemini CLI). We evaluate ADI vulnerabilities across off-the-shelf models and AI agents, and find that ADI is effective in both standalone LLMs and AI agent settings. ADI exposes a critical gap in agent security, signifying that current AI agents do not employ a fundamental security principle: current agents do not isolate trusted data from untrusted data.

• The paper rigorously identifies and characterizes Agent Data Injection attacks that exploit probabilistic delimiter injection to misclassify untrusted data as trusted metadata.
• It demonstrates that current defenses against instruction injection fail against ADI, with attack success rates reaching up to 43.3% on common formats and even 100% on web DOM.
• Utilizing strict data flow tracking can fully mitigate ADI vulnerabilities, though it drastically reduces AI agent task utility, highlighting significant trade-offs in secure agent design.

Agent Data Injection Attacks as Critical Threats to AI Agents

Introduction

The paper "Agent Data Injection Attacks are Realistic Threats to AI Agents" (2607.05120) investigates a new and underexplored class of security vulnerabilities in AI agents, termed Agent Data Injection (ADI) attacks. Prior work on agent security has focused predominantly on indirect prompt injection (IPI), with instruction injection—a scenario where attacker-controlled untrusted data is interpreted as instructions—receiving the majority of defensive attention. This paper rigorously identifies and characterizes ADI, where untrusted data is instead misinterpreted as trusted metadata or context, thus inducing AI agents to perform unintended, attacker-selected actions within the boundaries of the user’s original intent.

Threat Model and Attack Formalization

In the ADI threat model, the attacker supplies malicious payloads to external resources—such as web pages, emails, or GitHub issues—consumed by the AI agent through its tool integrations. Crucially, the attacker cannot modify the system or user prompts directly but leverages the agent's tendency to mix trusted (tool-generated) and untrusted (externally sourced) data within its execution context.

The paper distinguishes formally between instruction injection (where the LLM erroneously treats untrusted data as instructions) and agent data injection (where the LLM probabilistically misclassifies attacker data as trusted agent metadata). This distinction is pivotal: while most existing mitigations create boundaries between instructions and agent data, they fail to enforce finer-grained isolation between trusted and untrusted components of agent data, leaving agents widely exposed to ADI.

Technical Mechanism: Probabilistic Delimiter Injection

The core technique underlying ADI is "probabilistic delimiter injection." Unlike deterministic delimiter attacks (e.g., SQL injection, XSS) which depend on precise delimiter syntax, ADI exploits the inherent probabilistic interpretation of structure in LLMs. By strategically embedding inexact delimiter-like tokens (e.g., escaped quotes, delimiters with minor corruptions) in attacker-controlled, untrusted fields, the adversary can probabilistically subvert the LLM’s parsing of structured data (JSON, XML, Markdown, etc.), causing attacker payloads to bleed into regions that are assumed to be trusted metadata or context.

This probabilistic misinterpretation is unique to LLMs and is not prevented by traditional output escaping and sanitization strategies. Notably, even when tools apply standard escaping to untrusted input, the LLM often still interprets such fragments as structure, rendering classic data validation measures inadequate in agent contexts.

Empirical Analysis and Attack Demonstrations

A comprehensive evaluation was conducted both at the level of standalone LLMs (GPT-5.2, Claude Opus 4.5, Gemini 3 Pro, etc.) and within real-world agent deployments (Claude in Chrome, Codex, Gemini CLI, Nanobrowser). The attacks include:

• Arbitrary Click Attacks on web agents via fake element identifier injection, leading to unintended UI actions such as unauthorized purchases.
• Remote Code Execution on coding agents by forging trusted author metadata in issue comments, resulting in the execution of attacker-selected code.
• Supply Chain Attacks by fabricating tool call blocks within PR descriptions, enabling the merging of unreviewed malicious code.

Experimental results are alarming: baseline attack success rates (ASR) for ADI on common structured data formats (JSON, web DOM) range from 31.3% to 43.3% on top-tier LLMs, and up to 100% on web DOM, significantly exceeding those for instruction injection. Existing agent defenses that are effective against instruction injection (e.g., model hardening, input/output guardrails, dual-LLM architectures) routinely fail against ADI, with residual ASRs up to 50%. Only strict data flow tracking strategies provide full protection, but at the cost of sharply reduced benign task completion rates (utility drops from 86.5% to 36.5%).

Defense Efficacy and Trade-offs

The study exposes severe limitations in all tested defense approaches:

• Model Hardening, Guardrails, and Plan-Then-Execute: Ineffective because they do not differentiate trusted and untrusted data within agent context.
• Agent Sandboxing and Fine-Grained Policies: Can partially mitigate ADI if explicit, per-action policies are defined, but scalability is lacking in complex workflows.
• Randomization Defenses: Adding unpredictable, runtime-generated nonces to identifiers significantly reduces ASR for key-value data, but attacks leveraging aggregation over lists or unstructured data remain viable.
• Strict Data Flow Tracking: Only approach to achieve zero ASR; however, such granularity sharply curtails task utility, posing challenges to practical deployment.

ADIs thereby reveal a critical and novel axis of LLM-driven vulnerability: traditional isolation only between instructions and agent data leaves open broad and practical attack surfaces for adversaries to exploit the agent's operational context.

Practical and Theoretical Implications

This work has significant ramifications:

• For AI Agent Builders: Demonstrates the urgent need for a principled, fine-grained trust model that differentiates not just between instructions and data, but between internal agent-generated (trusted) and external (untrusted) data at all levels.
• For Security and Tooling: Suggests that simply applying deterministic escaping or separation techniques is insufficient in the face of LLMs' probabilistic parsing. Novel static and dynamic analyses, stronger data provenance enforcement, and robust flow control mechanisms must become standard practice for agent development.
• On Broader LLM Vulnerabilities: The identification of probabilistic delimiter injection expands the taxonomy of LLM exploitation strategies. It underscores that leveraging the probabilistic, language-model-based nature of agents yields fundamentally new classes of attacks, with no clear analog in traditional software security.

Future agent architectures will likely need multi-layered trust labeling, dynamic taint tracking, runtime randomization, or hybrid approaches to balance utility and security, possibly underpinned by formal verification of data flow policies.

Conclusion

The definition and rigorous empirical substantiation of agent data injection attacks exposes a critical, previously overlooked failure mode in current AI agent security practices. Their ability to induce high attack success rates with minimal attacker knowledge or access, coupled with the inadequacy of extant defenses, marks ADI as a central concern for agent deployments. Addressing this gap will require both security research and systemic architectural change to agent ecosystems.

Citation: "Agent Data Injection Attacks are Realistic Threats to AI Agents" (2607.05120)