AgentWatcher: A Rule-based Prompt Injection Monitor

Abstract: LLMs and their applications, such as agents, are highly vulnerable to prompt injection attacks. State-of-the-art prompt injection detection methods have the following limitations: (1) their effectiveness degrades significantly as context length increases, and (2) they lack explicit rules that define what constitutes prompt injection, causing detection decisions to be implicit, opaque, and difficult to reason about. In this work, we propose AgentWatcher to address the above two limitations. To address the first limitation, AgentWatcher attributes the LLM's output (e.g., the action of an agent) to a small set of causally influential context segments. By focusing detection on a relatively short text, AgentWatcher can be scalable to long contexts. To address the second limitation, we define a set of rules specifying what does and does not constitute a prompt injection, and use a monitor LLM to reason over these rules based on the attributed text, making the detection decisions more explainable. We conduct a comprehensive evaluation on tool-use agent benchmarks and long-context understanding datasets. The experimental results demonstrate that AgentWatcher can effectively detect prompt injection and maintain utility without attacks. The code is available at https://github.com/wang-yanting/AgentWatcher.

• The paper introduces AgentWatcher, which localizes malicious prompt injections using minimal influential context spans and interpretable rule-based detection.
• The paper employs a two-phase approach combining attention analysis and a monitor LLM, achieving attack success rates as low as 1% with minimal utility loss.
• The paper demonstrates robust performance across diverse tasks and models, offering explainable detection through actionable reasoning traces.

AgentWatcher: A Rule-Based Framework for Robust Prompt Injection Detection

Motivation and Related Work

Robust detection of prompt injection attacks is crucial for secure deployment of LLM-driven agents, which must safely process large, compositional, and potentially adversarial contexts originating from untrusted sources. Existing detection methods face two major limitations: their efficacy degrades substantially with increased context length, and they lack explicit, interpretable criteria for identifying prompt injection, resulting in opaque or over-simplistic decision boundaries. In particular, prior works either rely on black-box or fine-tuned LLM detectors that are not interpretable, or they enforce rigid policies unable to generalize to diverse, open-ended tasks (2604.01194).

AgentWatcher Design

Attribution of Causally Influential Context

AgentWatcher innovates by localizing detection to a minimal set of context spans attributed as causally influential for an agent's action. Instead of applying detection across the full context—which may contain tens of thousands of tokens—AgentWatcher applies attention analysis to attribute action-relevant context. Specifically, it analyses attention flows from context tokens to the generated action, identifying high-impact “sink tokens” via a sliding window, then expands around these to ensure the full injected instruction is covered. The process is repeated $K$ times to obtain multiple non-overlapping influential windows, preserving both precision and robustness.

Figure 2: Impact of sink detection window size $w_s$, left expansion size $w_l$, right expansion size $w_r$, and number of windows $K$ on utility and attack success rates.

This attribution strategy is more robust than fixed partition or segment-based approaches, which can fragment injected instructions across segment boundaries and thus fail to ascribe causal influence correctly. Empirical ablations show AgentWatcher’s attribution is highly insensitive to hyperparameters and consistently identifies malicious spans necessary for robust detection.

Rule-Based Detection with Monitor LLMs

Upon extracting these attributed context segments, AgentWatcher presents them, together with the original user intent and model action, to a monitor LLM that determines the presence of prompt injection. Detection is governed by an interpretable, explicit set of human- or LLM-generated rules specifying the criteria for injection. These rules are customizable and can be automatically generated via LLMs to fit the deployment scenario. Notably, the monitor LLM is prompted to provide both a binary verdict and a reasoning trace, yielding high explainability. The design allows for task-adapted policies (e.g., certain types of instructions are only malicious if not originating from the intended source), supporting fine-grained adjudication.

The monitor LLM is further fine-tuned using GRPO with a reward signal encouraging both accurate detection and extraction/localization of injected instructions. Fine-tuning strategies include both binary classification and BLEU-based reward for correct recovery of injected content, with preference for BLEU in balancing utility preservation and robustness.

Practical Considerations and Adaptivity

While leveraging LLM-based reasoning in detection increases computational cost (up to $8-10$ seconds per call), the authors argue for selective invocation: detection need only be triggered on high-risk actions or tool calls. Furthermore, restricting the windowed attribution and monitoring only the recent context history may further improve performance in practice without degrading robustness.

AgentWatcher is evaluated for robustness against both heuristic and optimization-based adaptive attacks. The use of explicit input delimiters and contextualization in the monitor LLM prompts enables strong resilience against attempts to subvert detection logic.

Experimental Results

Extensive benchmarking demonstrates that AgentWatcher achieves consistently strong trade-offs between utility (correct agent function in benign settings) and robustness (low attack success rate, ASR, under injection). Most notably:

• AgentDojo and AgentDyn: AgentWatcher achieves ASR $\leq 1\%$ with only $2-4\%$ utility loss, outperforming all prior baselines, including DataSentinel, PromptGuard, and PIGuard.
• Long-context datasets: Across code generation, summarization, information retrieval, and QA tasks, AgentWatcher reduces ASR to at most $10\%$ across all evaluated settings, while maintaining competitive or best-in-class clean utility.
• Model and Rule Generalizability: The method is robust across varying backbone and monitor LLMs, and performs well using both manually specified and LLM-generated rule sets, with small trade-offs depending on the rule generation strategy.

Ablation studies confirm that the attribution phase consistently outperforms alternative attribution approaches (log-probability, average attention, AT2) and that fine-tuning the monitor LLM further reduces ASR under both benign and adversarial settings.

Implications and Future Directions

AgentWatcher demonstrates that combining compact, attention-based attribution with rule-based LLM reasoning yields a scalable, interpretable, and robust prompt injection detection framework for LLM agents. This approach outperforms both model-agnostic, context-wide detectors and specialized policy checkers, particularly in the long-context, compositional, or partially trusted information settings common to modern LLM agent deployments.

Practical implications include improved deployment safety for LLM-powered agents in open-ended, tool-integrated, and user-facing platforms, as well as the capacity to expose and justify detection results to downstream users or auditors. This raises the possibility for more transparent and interactive agent security layers that can be adjusted or audited for regulatory compliance and safety assurance.

The primary bottleneck remains computational cost due to LLM-based reasoning; future work will benefit from continued optimization, increased efficiency through context pruning, and hierarchical or cascade workflows. As attack strategies evolve, additional research is required to further increase robustness against adaptive adversaries targeting both attribution and rule-based components (2604.01194).

Conclusion

AgentWatcher establishes a robust, interpretable, and extensible approach to prompt injection detection in LLM agents. Its dual-phase design—localizing detection to causally influential spans and delegating explicit rule-based judgment to a specialized monitor LLM—enables it to outperform prior approaches in both standard and adversarial benchmarks, for both utility preservation and attack resilience. This work strongly motivates further focus on modular, interpretable, and context-aware security architectures for AI agency and LLM deployment.