AutoInject: Automated Prompt Injection Using RL

Updated 5 July 2026

AutoInject is a reinforcement-learning framework for automated prompt injection against LLM agents interacting with external content and tools.
It employs a black-box policy with a 1.5B parameter adversarial suffix generator to optimize both malicious goal execution and benign task continuation.
Empirical results on the AgentDojo benchmark show superior performance and transferability compared to manual and random attack baselines.

Searching arXiv for the primary AutoInject paper and closely related automated prompt injection work in agentic settings. Searching arXiv for "Learning to Inject: Automated Prompt Injection via Reinforcement Learning" and related agentic prompt-injection papers. AutoInject is a reinforcement-learning framework for automated prompt injection against LLM agents that interact with external content and tools, rather than a generic chat jailbreak method. In its setting, adversarial instructions are embedded in untrusted environment content—such as an email body or retrieved document—and are intended to cause an agent to deviate from the user’s task by executing an attacker-specified action with the correct parameters, while often still completing the benign task. The framework is explicitly black-box with respect to the victim agent, uses a 1.5B parameter adversarial suffix generator, and is evaluated primarily in AgentDojo, where success is defined by concrete tool execution and parameter binding rather than by refusal-breaking alone (Chen et al., 5 Feb 2026).

1. Definition, scope, and threat model

AutoInject addresses indirect prompt injection in agentic, tool-using systems. The paper defines the attack surface as untrusted environment content that the agent reads during normal execution and that can contain adversarial instructions. The relevant failure mode is not merely that the model produces objectionable text, but that it performs attacker-chosen actions such as sending an email or transferring funds, with success measured through the benchmark’s programmatic checks (Chen et al., 5 Feb 2026).

This scope distinguishes AutoInject from conventional jailbreak optimization. The paper argues that prior prompt-injection attacks were mostly manual, template-based, and red-team driven, while existing automated jailbreak methods such as GCG are poorly matched to prompt injection because the objective is narrower: the model must select the right tool, bind the correct arguments, and often resume the benign workflow. A jailbreak proxy such as eliciting a phrase like “Sure, here is…” is therefore a weak surrogate for the behavioral outcome of interest (Chen et al., 5 Feb 2026).

The attacker model is explicitly black-box. The attacker does not require gradients, weights, or activations of the victim model, and does not need to deploy an auxiliary model inside the victim environment. Instead, the attacker can place adversarial content into the environment, observe outcomes exposed by the benchmark pipeline, and query the target under a finite budget. In the main setting, the budget is $B=260$ victim evaluations per $(\text{user task}, \text{injection task})$ pair. The attacker may use an external helper model—typically GPT-4o-mini—for optimization feedback, but that model is not part of the victim system (Chen et al., 5 Feb 2026).

A common misconception is that AutoInject is a universal prompt perturbation in the classical adversarial-NLP sense. In this work, “universal” means that the optimization process can discover reusable suffixes that transfer across many task pairs and models without re-optimization; it does not mean a single perturbation is proved to attack all prompts (Chen et al., 5 Feb 2026).

2. RL formulation and optimization objective

AutoInject formulates suffix generation as a Markov decision process conditioned on an injection goal $g$ and benign task context $c$ . At generation step $t$ , the state is

$s_t = (g,c,a_1,\ldots,a_{t-1}),$

where $a_1,\ldots,a_{t-1}$ are the previously generated tokens. An action is the next token $a_t \in \mathcal V$ , and the transition is deterministic concatenation,

$s_{t+1} = s_t \oplus a_t.$

An episode ends when the generator emits EOS or reaches a maximum length $T$ , producing a suffix $(\text{user task}, \text{injection task})$ 0. The policy is a LLM $(\text{user task}, \text{injection task})$ 1, instantiated in the main text as Qwen2.5-1.5B, which serves as the adversarial suffix generator (Chen et al., 5 Feb 2026).

The basic objective is

$(\text{user task}, \text{injection task})$ 2

Here $(\text{user task}, \text{injection task})$ 3 is the security score, indicating whether the malicious injected goal was executed, and $(\text{user task}, \text{injection task})$ 4 is the utility score, indicating whether the original benign task was still completed. AutoInject extends this with a dense preference term,

$(\text{user task}, \text{injection task})$ 5

The appendix reports $(\text{user task}, \text{injection task})$ 6, $(\text{user task}, \text{injection task})$ 7, and a scheduled $(\text{user task}, \text{injection task})$ 8 based on the consumed query fraction $(\text{user task}, \text{injection task})$ 9:

$g$ 0

so the dense term is emphasized early, when successful attacks are rare (Chen et al., 5 Feb 2026).

Algorithmically, the policy is initialized from a pretrained LM and the current best suffix $g$ 1 is initialized to empty. Until budget exhaustion or success, the method samples a group of suffixes, evaluates them in the victim pipeline to obtain $g$ 2, queries a feedback model for $g$ 3 relative to $g$ 4, computes the composite reward, updates $g$ 5 if a better suffix is found, and performs a policy update. Training stops early if any sampled suffix achieves both $g$ 6 and $g$ 7 (Chen et al., 5 Feb 2026).

3. Dense preference feedback and GRPO training

The central optimization difficulty is reward sparsity. Most random suffixes fail completely, so direct optimization on binary success signals yields little learning signal. AutoInject addresses this by maintaining the best suffix so far, $g$ 8, and asking a feedback model to compare a new suffix $g$ 9 against $c$ 0 with respect to malicious-goal execution, benign-task preservation, and evasion. The model produces a binary preference label, and AutoInject converts the final-token probabilities into a continuous comparison score,

$c$ 1

which supplies dense feedback even when neither candidate fully succeeds (Chen et al., 5 Feb 2026).

Optimization uses Group Relative Policy Optimization (GRPO). For each training step, the current policy samples a group of $c$ 2 completions for the same $c$ 3; the appendix gives $c$ 4. For completion $c$ 5 with reward $c$ 6, GRPO computes a normalized relative advantage

$c$ 7

This relative normalization is important because it still yields gradient signal when all completions are poor in absolute terms. The paper presents a clipped GRPO/PPO-style surrogate with KL regularization to a reference policy; it also notes that the typesetting of the objective is corrupted in places, but the intended form is clearly a clipped policy-gradient objective with KL control. The appendix lists $c$ 8 for most frontier-model attacks and $c$ 9 for some 4B-model settings (Chen et al., 5 Feb 2026).

The attack does not learn a full prompt from scratch. Instead, the learned suffix is appended to a fixed social-engineering frame in the AgentDojo injection template that impersonates a legitimate user message and instructs the agent to perform an extra step before resuming the original task. This design keeps the optimization focused on the suffix while holding the broader frame constant (Chen et al., 5 Feb 2026).

Utility preservation is a first-class objective rather than a side effect. The framework explicitly optimizes attacks that both compromise the agent and preserve the user’s benign task, on the premise that stealthy compromise is more operationally relevant than attacks that merely disrupt execution (Chen et al., 5 Feb 2026).

4. Empirical performance on AgentDojo

The experiments center on AgentDojo, a benchmark with 97 user tasks across banking, travel, workspace, and Slack. Each user task is paired with multiple injection tasks, and the environment provides ground-truth binary signals for benign-task completion and malicious-goal execution. This benchmark structure is integral to AutoInject because it supplies the clean behavioral reward signals required by the RL formulation (Chen et al., 5 Feb 2026).

On the full benchmark, AutoInject substantially outperforms the hand-written template baselines on frontier systems (Chen et al., 5 Feb 2026).

Model	AutoInject ASR	Best template baseline
Gemini-2.5-Flash	58.00%	23.60%
GPT-4.1-nano	47.97%	20.48%
GPT-5-nano	11.49%	1.60%
Claude-Sonnet-3.5	12.59%	5.69%

The paper also reports strong results on additional targets: 39.62% ASR on Gemini-2.0-Flash, 50.37% ASR on GPT-4o-mini, 42.5% ASR on Qwen3-4B, and 35.0% ASR on Gemma3-4B, outperforming GCG, TAP, and an adaptive random-search baseline under comparable budgets (Chen et al., 5 Feb 2026).

Utility results are a notable part of the evaluation. The paper reports cases in which utility under attack equals or exceeds the no-attack baseline, including GPT-5-nano with 90.20% utility under attack versus 80.41% utility without attack, and Claude-Sonnet-3.5 with 98.52% utility under attack versus 75.00% baseline utility. The authors attribute this to the explicit inclusion of utility in the reward, which biases the learned suffix distribution toward non-disruptive attacks (Chen et al., 5 Feb 2026).

The baseline suite includes six manual templates—Direct Instruction, Ignore Previous, Important Instructions, InjecAgent, System Message, and Tool Knowledge—as well as automated baselines: GCG, TAP, and a random adaptive attack. A stronger ablation baseline keeps AutoInject’s reward function but replaces the learned policy with random mutation search, isolating the contribution of RL itself (Chen et al., 5 Feb 2026).

5. Transferability, reusable suffixes, and ablations

AutoInject supports both online query-based optimization and transfer/universal mode. In the first mode, the suffix policy is optimized adaptively for a specific $t$ 0 pair. In the second, suffixes found on one model or task are reused on unseen models and task pairs. The transfer evaluation considers both cross-task transfer and cross-model transfer, including transfer to held-out task pairs across banking, Slack, travel, and workspace (Chen et al., 5 Feb 2026).

Transferability is strongly model dependent. GPT-4o-mini is consistently vulnerable in the reported transfer studies, often with transfer ASRs in the 37.5%–62.5% range. GPT-5 is the most resistant target: it shows 0% ASR from all source models in cross-task transfer and also 0% from all external models in fixed-injection transfer, with even self-transfer reaching only 6.2% in one setting. At the same time, GPT-5 can be a strong source model: suffixes optimized on GPT-5 transfer to Gemini 2.5 Flash and GPT-4o-mini at 50% ASR in the cross-task setup (Chen et al., 5 Feb 2026).

The most vivid reusable-family result is the “allelujah” suffix family. One reported suffix—

$t$ 2

—successfully attacks 70 task pairs on Gemini-2.5-Flash, 53 on GPT-4o-mini, and 46 on GPT-4.1-nano, out of 949 total benchmark pairs. Different reorderings or small textual edits materially affect the success rate, which suggests a genuine token-pattern effect rather than a trivial semantic instruction. The paper treats mechanistic explanations of this phenomenon as speculative (Chen et al., 5 Feb 2026).

The ablations support both of AutoInject’s main ingredients: policy-gradient RL and dense feedback. On the banking suite for GPT-4.1-nano, full AutoInject achieves 97.69% ASR with 62.50% utility; removing GRPO reduces ASR to 88.32%, removing feedback to 95.14%, and removing both to 90.51%. On the harder GPT-5-nano, full AutoInject reaches 36.57% ASR and 87.50% utility; without GRPO this falls to 19.44%, without feedback to 25.00%, and without both to 17.36%. The same-reward random-search baseline also underperforms: on Gemini-2.5-Flash, AutoInject gets 58.0% ASR versus 34.04% for search; on GPT-5-nano, 11.49% versus 4.84%; and on Claude-Sonnet-3.5, 12.59% versus 7.64%, with markedly better utility for AutoInject (98.52% versus 77.08%) (Chen et al., 5 Feb 2026).

These results align with a broader empirical trend in agentic prompt injection: black-box, semantically structured optimization tends to outperform brittle token-level search when the target is end-to-end tool behavior rather than free-form compliance (Hofer et al., 9 Jun 2026).

AutoInject is evaluated under a realistic but still benchmark-mediated setting. The paper assumes access to AgentDojo’s explicit success and utility signals, which is natural inside the benchmark but may be less directly available in unconstrained deployments. It also does not provide a formal theory of transfer or universality, no offline-data component, and no large systematic sweeps over suffix length or query budget beyond the reported fixed settings and the $t$ 1-schedule (Chen et al., 5 Feb 2026).

The work also highlights defense implications. Against Meta-SecAlign-70B, a model specifically fine-tuned for prompt-injection robustness, template attacks are nearly or entirely neutralized, yet AutoInject still achieves 21.88% ASR with 39.29% utility, slightly above the no-attack utility of 37.50%. The paper interprets this as evidence that preference-based defenses can reject familiar attack patterns while remaining vulnerable to adaptive out-of-distribution attacks (Chen et al., 5 Feb 2026).

At the architectural level, the paper cites CaMeL and IPI-Guard as examples of plan-then-execute or decoupled designs that prevent untrusted content from influencing control flow after planning. AutoInject does not break that threat model because it targets the more common case in which agents continue reasoning over untrusted inputs during execution. The paper also notes a benchmark limitation with immediate tool execution: in experiments, models such as Claude Haiku 4.5 often ask for clarification before acting, which breaks the benchmark pipeline and suggests a defense direction based on deliberative confirmation before sensitive actions (Chen et al., 5 Feb 2026).

Within the surrounding literature, AutoInject belongs to a broader shift from manual prompt engineering toward automated attack optimization in agentic systems. Related directions include semantic-search-based automated prompt injection in AgentDojo (Hofer et al., 9 Jun 2026), no-box tool-selection hijacking through optimized malicious tool documents (Shi et al., 28 Apr 2025), and closed-loop stealthy skill poisoning for coding agents (Jia et al., 15 Feb 2026). The name AutoInject has also been used independently for a direct message-corruption mechanism in studies of multi-agent robustness, where it denotes controlled rewriting of inter-agent messages rather than RL-based prompt injection (Huang et al., 2024).

The broader implication is that indirect prompt injection is better understood as an end-to-end behavioral optimization problem than as a token-level jailbreak variant. AutoInject operationalizes that view by optimizing for successful malicious action execution, correct parameter binding, and benign-task preservation simultaneously. This suggests that future defenses must handle adaptive, utility-preserving, transferable prompt injections, not merely known templates or overt override strings (Chen et al., 5 Feb 2026).