- The paper shows naive LLM agents achieve an 82.50% unsafe tool-action rate under indirect prompt injection attacks, exposing critical vulnerabilities.
- The benchmark features 130 scenarios with varying threat levels and evaluates multiple defenses, including prompt-based methods and a two-pass LLM judge.
- The study finds that execution-time policy gating using trusted metadata effectively prevents unsafe actions while maintaining agent utility compared to static allowlists.
Problem Setting and Motivation
This paper introduces NetInjectBench, a benchmark for systematically evaluating tool-using LLM agents against indirect prompt injection (IPI) threats in network and communication operations. The context is highly relevant: operators increasingly deploy LLM agents for automating diagnostics, configuration, and incident response tasks, where the agent ingests heterogeneous operational artifacts (tickets, logs, runbooks, alerts, messages) that may originate from untrusted or adversarial sources. The core vulnerability is that malicious actors may craft textual artifacts embedding instructions that, when consumed by an LLM agent, induce unsafe tool invocation (misconfiguration, premature changes, information leakage).
NetInjectBench fills a gap in the evaluation of agentic LLMs for these safety-critical domains. Prior agent benchmarks focus on tool-use capability over secure execution semantics. Indirect prompt injection has been studied largely in open-ended or general document contexts, whereas NetInjectBench models fine-grained, operationally grounded scenarios with an explicit separation of trusted authorization metadata from untrusted evidence, reflecting real-world access control and network safety objectives.
Benchmark Design
The benchmark comprises 130 scenarios partitioned into benign (n=40), weak-attack (n=40), strong-attack (n=40), and approved high-impact change (n=10) categories. Each scenario strictly separates:
- Prompt fields: Task/intent, available tools, and untrusted artifact text.
- Tool fields: Tool schemas, privilege levels (low-risk read, sensitive read, low-risk write, high-impact write).
- Trusted policy metadata: Structured records for authorization, e.g., verified approvals, maintenance windows, device/patch scopes.
- Evaluation labels: Safety and utility ground truth, for posthoc analysis.
Adversarial manipulations include direct instructions, fake approvals, authority impersonation, and poisoned runbook/KB updates, simulating a broad space of plausible IPI threats encountered by real network-operation agents. The tool set is compact yet sufficient to model privilege escalation and sensitive-information exposure risks.
Threat Model and Safety Targets
The evaluation assumes the adversary can control untrusted artifact content but not the trusted policy metadata, tool registry, or execution logic. Thus, the critical security question is whether untrusted text—no matter how semantically convincing—can lead to unsafe execution of privileged tools in the absence of corresponding trusted metadata.
The evaluation scope is limited to single-step tool selection, isolating the decision where artifact-derived manipulation can have direct operational impact. Multi-step tool-output poisoning and compromised metadata are not in-scope but are recognized as future directions.
Methodological Rigor
The authors evaluated three open LLMs—Qwen2.5-7B, Llama3.1-8B, and Mistral-7B—using seven defense patterns:
- Naive LLM agent: Executes the model-proposed tool.
- Prompt-only safety: Instructions emphasizing that artifact text is untrusted and high-impact actions require metadata verification.
- Self-Reminder & Spotlighting: Enhanced prompt templates repeating trust boundaries and marking untrusted data.
- Two-Pass LLM Judge: Decouples proposal and adjudication; the second pass judges tool invocation without the artifact present.
- Static allowlist: Blocks all high-impact actions (e.g., apply_config_change).
- Metadata-aware policy gate: Authorizes high-impact or sensitive actions only when trusted policy metadata matches the proposal.
Evaluation metrics encompass unsafe tool-action rate (UTAR), useful action rate (UAR), block and overblocking rates, and reliability indicators (invalid output, normalization rates), with confidence intervals and paired method tests for robust statistical comparison.
Empirical Findings
Model Vulnerability
- The naive LLM agent yielded an 82.50% aggregate unsafe tool-action rate across 240 attack scenarios, with even higher susceptibility for Qwen2.5-7B and Mistral-7B on "strong" attacks. This exposes the acute risk of naive LLM deployment in operational environments.
Defense Efficacy
- Prompt-only defenses (including Self-Reminder, Spotlighting) halved or reduced unsafe actions to ~18–26%, but left nontrivial residual risk, especially under plausible attacks (UTAR remained >15% for strong attacks).
- Two-Pass LLM Judge improved robustness (10% UTAR overall), but still failed to reliably enforce authorization boundaries across models and scenarios.
- Static allowlist slashed unsafe execution to 5% but simultaneously induced 100% overblocking for approved high-impact changes, annihilating agent utility for necessary configuration operations.
- The policy gate method yielded 0 observed unsafe executions (UTAR=0/240) with a 95% Wilson upper bound of 1.58%, and 99.17–100% useful-action rates across both attack and approved-change scenarios. Crucially, this mechanism preserved agent utility, unlike static approaches.
- The metadata-stress evaluation (systematically mutating approval fields) demonstrated that the policy gate blocked all malformed approval attempts, corroborating its precision under the stated metadata-integrity assumption.
Privilege, Risk, and Artifact Analysis
- Static blocklisting of high-impact tools eliminated certain threat vectors but failed entirely for sensitive-read exposure (100% UTAR on query_topology attacks), confirming that effective defenses must integrate privilege- and context-aware logic for both write and read risks.
- IPI attacks induced unsafe actions via all artifact types (tickets, alerts, logs, runbooks, ChatOps, KB entries), necessitating a holistic artifact-channel model in agent defense design.
Theoretical and Practical Implications
- The empirical evidence solidifies that execution-layer policy enforcement grounded in trusted metadata is both necessary and sufficient (under the assumed threat model) to robustly separate evidence and authorization channels for agentic LLMs in network environments.
- Prompt-level defenses should be considered hardening rather than primary safety boundaries: LLMs cannot reliably distinguish real from fake authority in untrusted text, regardless of prompt engineering.
- Overblocking via static means is operationally unacceptable; practical agent architectures must support approval-granular gatekeeping.
- The NetInjectBench taxonomy, metrics, and scenario schema provide a reusable framework for the principled evaluation of agentic LLM safety in security-sensitive tool-use settings.
Limitations and Future Work
- The benchmark is synthetic, designed for reproducibility and confidentiality; validation with real-world operational data remains open.
- Current scenarios are single-step; future iterations should model multi-turn, output-poisoning, artifact-conflict, and metadata-degradation dynamics.
- Evaluation under stochastic decoding, with larger model pools, and integration with human-in-the-loop authorization workflows are proposed extensions.
- The safety claim is contingent on metadata-integrity. If trusted records are compromised, the policy-gate cannot guarantee correct authorization—a classical access-control limitation.
Conclusion
This work rigorously demonstrates that LLM-based agents, when deployed for network operations, are critically vulnerable to IPI via untrusted artifacts. Prompt-level mitigations, while reducing risk, do not eliminate unsafe executions. Only execution-time gating on structured, integrity-assured metadata robustly and reliably enforces operational authorization semantics, balancing both safety and agent utility. NetInjectBench establishes a high-precision, domain-grounded foundation for agentic safety evaluation and will inform future system design and policy research in secure AI-driven operations.
