WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents

Abstract: Web agents powered by vision-LLMs (VLMs) enable autonomous interaction with web environments by perceiving and acting on both visual and textual webpage content to accomplish user-specified tasks. However, they are highly vulnerable to prompt injection attacks, where adversarial instructions embedded in HTML or rendered screenshots can manipulate agent behavior and lead to harmful outcomes such as information leakage. Existing defenses, including system prompt defenses and direct fine-tuning of agents, have shown limited effectiveness. To address this issue, we propose a defense framework in which a web agent operates in parallel with a dedicated guard agent, decoupling prompt injection detection from the agent's own reasoning. Building on this framework, we introduce WebAgentGuard, a reasoning-driven, multimodal guard model for prompt injection detection. We construct a synthetic multimodal dataset using GPT-5 spanning 164 topics and 230 visual and UI design styles, and train the model via reasoning-intensive supervised fine-tuning followed by reinforcement learning. Experiments across multiple benchmarks show that WebAgentGuard consistently outperforms strong baselines while preserving agent utility, without introducing additional latency.

• The paper introduces a parallel guard model that decouples task execution from safety checks to robustly detect prompt injection attacks with high recall and precision.
• It employs a two-stage training pipeline—supervised fine-tuning followed by reinforcement learning—achieving up to 99.2% accuracy and over 90% recall in adversarial settings.
• A large-scale synthetic multimodal dataset underpins the approach, enabling evaluation across diverse, real-world web scenarios with negligible latency impact on agent utility.

WebAgentGuard: A Reasoning-Driven Approach for Detecting Prompt Injection Attacks in Web Agents

Introduction and Motivation

Web-based autonomous agents leveraging vision-LLMs (VLMs) now routinely perform user-driven online tasks by interacting with the web through both rendered page screenshots and underlying HTML content. However, execution-time prompt injection attacks present a critical vulnerability: adversarial instructions, rendered either visually or in text, can bypass intended controls and manipulate agent behavior, often leading to severe real-world consequences such as credential exfiltration and unauthorized actions. Prior works relying solely on fine-tuning, prompt engineering, or system prompt modifications have proved insufficient, frequently yielding attack success rates surpassing 80%. This intrinsic deficiency arises from coupling safety analysis and task reasoning within a single agent policy, resulting in a consistent prioritization of user instruction satisfaction at the expense of robust risk detection.

Parallel Guard Framework

To decouple safety verification from web agent task execution, the paper proposes a parallel runtime defense architecture in which a dedicated, lightweight guard model monitors each agent-environment interaction step. For every execution loop, the current user instruction, page screenshot, and processed HTML are forwarded to both the main web agent (which proposes the next action) and the guard model (which independently outputs a binary permission signal). The action is executed only if the guard approves, or if manual user override is provided following guard denial.

Figure 1: Overview of the parallel guard framework for web agents, delineating the split execution and approval signal process at each step.

This architecture ensures that the agent's action-generation is uninfluenced by safety checks, enabling the guard to focus solely on risk assessment rather than task completion. Critically, this arrangement introduces negligible additional latency because the guard model is substantially smaller than state-of-the-art VLM agents and operates in parallel.

Synthetic Multimodal Dataset Construction

To facilitate robust training and evaluation, the authors devise a large-scale, fully-synthetic multimodal dataset simulating both benign and adversarially injected webpages. Leveraging GPT-5, the dataset spans 164 topics and 230 fine-grained UI and visual design styles, offering broad coverage across real-world web scenarios. For each topic, benign HTML files are generated, rendered as screenshots, and paired with contextually relevant user instructions. Malicious samples are created by injecting adversarial instructions at random positions within the HTML. Screenshots of these injected pages are also captured to expose vulnerabilities both textually and visually.

The resulting dataset is multimodal and contains tuples of (user instruction, HTML, screenshot, label), partitioned into splits for supervised fine-tuning (SFT), reinforcement learning (RL), and evaluation.

Reasoning-Augmented Training Pipeline

Reasoning-driven detection is central to the WebAgentGuard approach. During SFT, each training instance includes a structured reasoning trace, generated by prompting GPT-5 to provide a stepwise analysis before its final verdict. This reasoning is enclosed in > ... tags, with subsequent binary classification output. Samples revealing leakage or inconsistent annotation during reasoning are filtered out to ensure high-quality supervision.

Figure 2: Training pipeline for WebAgentGuard, showing dataset synthesis, reasoning annotation, and two-stage model optimization.

A two-stage training regime is employed:

1. Supervised Fine-Tuning (SFT): The backbone VLM is first cold-started on the reasoning-structured dataset via standard language modeling objectives, optimizing for accurate reasoning and adherence to the templated output format.
2. Reinforcement Learning (GRPO): SFT initialization is followed by Group Relative Policy Optimization, which directly maximizes the alignment between generated outputs and ground-truth risk detection under the required reasoning/answer template, using group-wise normalized rewards.

Ablation studies demonstrate that SFT is essential for initial safety alignment, while RL effectively refines generalization and robustness, but only when applied post-SFT.

Empirical Evaluation

Extensive experiments are conducted under both in-domain (held-out synthetic) and out-of-domain (VPI-Bench, EIA, PopUp Attacks on top-ranked websites) settings.

• Baseline performance: Open-source and closed-source instructed VLMs, as well as prior guard models (e.g., Llama-Guard-3-Vision-11B, GuardReasoner-VL-7B, Prompt-Guard variants), generally demonstrate unsatisfactory recall (<20%), frequently missing harmful actions even when precision is high.
• WebAgentGuard results: WebAgentGuard-8B achieves up to 99.2% accuracy and 99.19 F1 in-domain, with recall consistently exceeding 90% on challenging out-of-domain prompt-injection attacks, solidly outperforming all baselines.

Key result: WebAgentGuard maintains perfect or near-perfect precision while achieving unprecedented recall, especially in adversarial scenarios where previous models fail catastrophically.

Defense-Utility and Latency Tradeoff

To assess deployment practicality, WebAgentGuard is integrated into state-of-the-art agent environments (Claude Agent, Browser-use Agent/Gemini-2.0 Pro) and evaluated against real-world attacks:

• Attack mitigation: On Amazon, Booking, and BBC, attack success rates are reduced from 84–96% (no defense) or 23–42% (strongest baseline guard) to near-zero (≤1.8%) with WebAgentGuard.
• Agent utility: The addition of the guard results in only modest reduction in agent task success rate as measured in benign environments (WebArena Shopping/Map). One-time user override for guard-denied benign steps recovers most of this loss.
• Latency: WebAgentGuard inference (2.1–3.2s, depending on size) is consistently lower than agent action-generation time, validating the framework's efficiency.

  Figure 3: Agent utility across websites and agents, illustrating mild utility loss and improvements with user-in-the-loop override.

Model Robustness and Failure Modes

WebAgentGuard is shown robust to guard-targeted prompt injections regardless of adversarial instruction placement (head or tail of HTML), with detection rates unaffected or even improved due to clearer malicious signals. Failure cases typically arise not from an inability to detect injected instructions, but rather from subsequent loss of attention during multi-step reasoning, leading to a "negative" prediction after correctly identifying initial risk cues.

Figure 4: Successful prediction example—guard correctly detects and reasons about injected adversarial instructions.

Figure 5: Failure case—despite initial detection of injection, the guard neglects the signal in later reasoning steps and yields a false negative.

Implications and Future Directions

WebAgentGuard transitions multimodal agent safety research from task-coupled defense to parallel risk-aware architectures, empirically validating that decoupled, reasoning-driven detection yields substantial real-world robustness gains with negligible deployment cost. The systematic combination of reasoning-augmented SFT and reward-based RL is critical—mere scaling or template compliance is insufficient for reliability under distribution shifts.

Practically, this framework can be extended to multitask agents, arbitrary multimodal modalities, and more aggressive attacker models (e.g., white-box adaptive adversaries). The synthetic data strategy enables rapid extension to new domains (mobile, non-HTML-based interfaces), though coverage completeness remains an ongoing challenge.

Theoretically, the observed generalization of reasoning-driven guards suggests the viability of robust, task-agnostic safety policies as modular plug-ins for agent system architectures. Further work could incorporate automatic counterfactual feedback, tighter integration with OS-level execution sandboxes, and quantization/distillation for edge deployment.

Conclusion

WebAgentGuard establishes a robust, highly generalizable paradigm for detecting prompt injection attacks in web agents by separating safety verification from agent action reasoning. The framework demonstrates that multimodal, reasoning-rooted, parallel guard models can deliver high recall and utility preservation on diverse, adversarial web environments, setting a new standard for AI agent safety design and offering a scalable path toward trustworthy autonomous agents.

(2604.12284)