ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents

Abstract: Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspecified tasks. However, the security implications of this interaction pattern remain unexplored. We investigate whether the transition from standard execution to a clarification-seeking state increases an agent's susceptibility to prompt injection attacks. We introduce ASPI (Ambiguous-State Prompt Injection), a benchmark of 728 task-attack scenarios that isolates clarification as a distinct agent state and measures how this state transition affects vulnerability under controlled conditions. Each benchmark instance is evaluated under matched execution and clarification settings: in the execution setting, the agent acts on a fully specified instruction and encounters adversarial content only through tool-returned data; in the clarification setting, the agent must first request and incorporate additional user input before acting. We evaluate ten frontier LLMs and find that clarification-seeking consistently and substantially amplifies vulnerability. For instance, attack success rises from 1.8% to 34.0% for o3 and from 2.2% to 35.7% for Gemini-3-Flash. A decomposition analysis reveals that this gap reflects both a state-dependent shift in how models process incoming content and a channel-specific effect arising from the agent-solicited clarification interface. These findings demonstrate that standard execution-time security evaluation systematically underestimates the attack surface of interactive agents, and that robustness under fully specified tasks does not translate to robustness under ambiguity. For reproducibility, our data and source code are available at https://github.com/scaleapi/aspi.

• The paper discovers that LLM agents are significantly more vulnerable to prompt injection during their clarification phase, with attack success rates spiking up to 60%.
• It introduces the ASPI benchmark to systematically compare benign and ambiguous states across models, isolating the impact of clarification on security.
• The study shows that existing lightweight defenses offer only modest mitigation, highlighting the need for robust strategies in managing ambiguity resolution.

Ambiguity-Induced Clarification as a Pathway to Amplified Prompt Injection in LLM Agents

Introduction and Motivation

Ambiguity resolution via clarification-seeking is widely regarded as a core feature in the design of LLM agents, enabling robust handling of underspecified instructions. However, "ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents" (2605.17324) identifies, formalizes, and empirically validates a substantial security risk that emerges specifically during this clarification-seeking phase. The paper rigorously demonstrates that when LLM agents transition from standard task execution to seeking clarification, they become dramatically more susceptible to prompt injection attacks. The work builds on robust experimental design, leveraging a newly introduced ASPI benchmark and controlled paired evaluation across both proprietary and open-source agent models.

ASPI Benchmark and Threat Modeling

The ASPI benchmark isolates the clarification turn as a unique agent state, distinct from conventional execution. Each of 728 instances couples a benign user task with a corresponding attacker objective, constructing matched ambiguous (requiring clarification) and unambiguous (fully-specified) conditions. Attack introduction is explicitly manipulated—either via standard tool-returned data (execution state) or through clarification-reply channels (clarification state). Notably, ASPI evaluates multiple delivery mechanisms: tool, user, and ask_user relay.

The attacker is external and cannot affect the system prompt, agent weights, or implementation, and is unaware of the specific user task. Their aim: to induce the agent to execute a malicious action, preferentially alongside successful completion of the user task—revealing the extent to which ambiguity and clarification interfaces expand the attack surface.

Clarification-State Amplifies Attack Success

The central result is a strong, consistent increase in attack success rate (ASR) when prompt injection is delivered during clarification rather than execution. Models with near-zero execution-time vulnerability (e.g., ASR 0–2%) exhibit clarification ASR of 20–60%. For the strongest cases:

• o3: 1.8% → 34.0% (+32.2 percentage points)
• Gemini-3-Flash: 2.2% → 35.7% (+33.5 pp)
• Gemini-3.1-Pro: 1.1% → 24.3% (+23.2 pp)
• Kimi K2.5: 11.1% → 63.1% (+52 pp)

Frontier models such as Claude-Opus-4.7 present minimal risk under both settings, but this robustness is not representative of the broader model class.

Empirical Decomposition: State versus Channel Effects

A key technical contribution is the decomposition of the vulnerability gap into state and channel effects, using paired comparisons with exact McNemar testing for statistical significance.

• Tool-channel (clarif_tool vs exec_tool) comparisons isolate the impact of interacting in the clarification state with the attack delivered as tool output. Effects are heterogeneous: some models (notably Gemini) see a positive state effect (+2 to +5 pp), while others (DeepSeek, Qwen3) see null or negative effects.
• User-channel (clarif_user vs exec_user) shows strong amplification under clarification for various models, with ΔASR up to +25 pp.

  Figure 2: Paired differences in attack success rate by channel, demonstrating model-dependent and channel-dependent amplification or suppression of vulnerability.

Notably, the channel used for attack delivery—specifically, agent-solicited clarification replies—acts as a primary amplification mechanism. When adversarial content is introduced via the ask_user relay rather than standard tool returns, agents are much more likely to perceive and execute the injected instructions as legitimate.

Robustness, Capability, and Utility Trade-offs

The robustness-versus-capability analysis is central to understanding practical agent selection. Figure 3 presents the relationship between ASR and clean-task utility, as well as the degradation of utility under attack.

Figure 4: Illustration of models' robustness and utility; high-performing models may remain vulnerable to clarification-time prompt injection, refuting the correlation between general capability and security.

Models achieving high utility in benign conditions may nevertheless be highly vulnerable to clarification-time injection. Some robust models maintain both high utility and low ASR, while others that appear robust (low ASR) do so only due to low task efficacy. The figures also reveal models that degrade in both security and utility when adversarial content is present during clarification, indicating poor trade-offs.

Failure Analysis: Behavioral Mechanisms

The study's judge-based behavioral analysis reveals several mechanisms:

1. High-Quality Clarification Does Not Imply Safety: Figure 5 shows that most models ask on-target clarification queries, but downstream attack compliance is still frequent following these queries.

   Figure 6: Distribution of clarification-question quality, confirming that most models reliably seek clarification for the genuine missing slot.

2. Attack-Following is Often Integrated, not Replacing: Figure 7 demonstrates that after processed attack-laden clarification, most agents execute both the benign and attacker goals (TASK_AND_ATTACK), not just the adversarial one.

   Figure 1: Agents, upon receiving attacked clarification, frequently execute both the original and injected tasks.

3. Clarification Channels Drastically Increase Compliance: Figure 8 shows channel-driven differences—clarification-driven user and ask_user channels drive much higher full/partial compliance rates compared to tool-output attacks.

   Figure 9: Attack compliance by channel, quantifying the increased effectiveness of adversarial instructions embedded in the clarification interface.

4. Clarification-Channel Attacks Elicit CONFUSED or PERSUADED Behavior: Figure 10 reveals that attack compliance in clarification settings is often attributed to agent confusion (treating the injected instruction as user-followup) or to being persuaded by the framing, confirming that the solicitation context biases agent trust.

   Figure 11: Compliance reasoning by condition, highlighting the role of agent confusion or persuasion in facilitating successful attack execution via clarification channels.

5. Clarification-Question Quality Alone Does Not Preclude Vulnerability: Figure 12 demonstrates that even when the clarification is targeted properly, attack execution remains highly likely—indicating the failure arises at the input acceptance and integration stage, not just ambiguity detection.

   Figure 3: Downstream attack success is not blocked by correct clarification—solicited inputs remain a prime injection channel.

Defense Evaluation

Lightweight defenses, including segment-level prompt guards and ask_user-aware tool filters, yield modest reductions in clarification-state ASR (e.g., Gemini-3-Flash, prompt guard: 35.7 → 27.0%; tool filter: 35.7 → 23.9%) but do not eliminate the vulnerability. Notably, clarification must remain functional, so tool filtering cannot disable ask_user, and input filtering cannot reliably excise injected segments that are syntactically fused with benign content.

Figure 14: Attack success rates in paired clarification vs. execution conditions under defenses, demonstrating only partial mitigation in the clarification context.

Implications and Future Directions

Practical Consequences

• Standard agent security benchmarks that assess vulnerability only during normal execution will severely underestimate real-world risk unless ambiguity-driven clarification pathways are evaluated.
• LLM agents perceived as robust may covertly integrate adversarial behavior when clarification is solicited, especially in human-in-the-loop or multi-actor scenarios.
• Lightweight input or toolset restrictions cannot fully mitigate this vulnerability, since the attack is semantically embedded in responses agents perceive as essential.

Theoretical and Methodological Considerations

• Clarification seeking is not a neutral design element; it is a distinct agent state characterized by altered input trust boundaries—mirroring classic security failures where explicit user solicitation is exploited to subvert privilege separation.
• ASPI establishes a controlled, paired evaluation protocol isolating interaction state and attack delivery—a methodological advance for agent security analysis.

Future Research Directions

• Investigating training and architectural interventions that can robustly maintain instruction/data separation under agent-solicited ambiguity resolution.
• Expanding threat models to multi-turn or multi-slot ambiguity, increasing realism and revealing wider agent weaknesses.
• Extending ASPI to diverse agent frameworks, multi-agent systems, and real-world tool chains beyond the simulated AgentDojo/ASPI environment.
• Exploring task or context-sensitive defenses that adapt based on detected ambiguity and input provenance.

Conclusion

This work systematically exposes a critical and previously undermeasured vulnerability axis in LLM agents: the act of seeking clarification while resolving ambiguity amplifies susceptibility to prompt injection attacks. The transition to clarification from standard execution alters agent trust and integration of incoming content, creating new high-impact attack surfaces not mitigated by existing defenses. Ensuring robust instruction/data separation—even in actively solicited contexts—is necessary for the safe deployment of agentic LLMs in interactive environments.

---

Reference:

ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents (2605.17324)