HarmonyGuard: Multi-Agent Web Safety

• HarmonyGuard is a collaborative multi-agent framework that integrates adaptive policy enhancement and dual-objective optimization to secure long-sequence web interactions.
• It dynamically extracts, structures, and updates security policies using LLM-powered refinement and deduplication to combat evolving web threats.
• Empirical benchmarks show improved policy compliance and task completion, demonstrating the framework’s efficacy in mitigating risks in adversarial web environments.

HarmonyGuard is a multi-agent collaborative framework for LLM-based web agents operating in open, dynamic web environments. It is designed to jointly improve safety and utility during long-sequence web interaction through two named capabilities: Adaptive Policy Enhancement and Dual-Objective Optimization. The framework couples a Web Agent that executes tasks, a Policy Agent that extracts and updates structured security policies from unstructured external documents, and a Utility Agent that evaluates each reasoning step for policy compliance and task alignment, then supplies optimization guidance when either objective is violated (Chen et al., 6 Aug 2025).

1. Problem domain and design objective

HarmonyGuard is formulated for web agents that perform multi-step tasks such as shopping, booking, browsing, and interacting with web interfaces over many reasoning and execution steps. The paper characterizes the web environment as open-ended, dynamic, long-horizon, and partially adversarial, with threats including prompt injection, environment injection, knowledge poisoning, tool library or MCP hijacking, phishing links, pop-ups and notifications, and malicious scripts embedded in web pages (Chen et al., 6 Aug 2025).

The framework is motivated by a stated tension between utility and safety. If an agent pursues utility too aggressively, it may follow unsafe instructions, click malicious links, leak information, or violate operational constraints. If it pursues safety too conservatively, it may refuse too many actions, overcorrect, or stall, causing utility collapse. The paper argues that this trade-off becomes more severe in long-sequence settings, where small misalignments can compound into “risk amplification cascades” (Chen et al., 6 Aug 2025).

Two challenges are singled out. The first is Safety–Utility Disconnection: security policies evolve with threats, but are often embedded in regulatory documents, organizational guidelines, and other unstructured external text. The second is Safety–Utility Trade-off: safety and utility cannot be optimized independently during long-horizon web tasks. HarmonyGuard is presented as an answer to both challenges, and is positioned against prior work that, in the paper’s characterization, is limited by single-objective optimization, single-turn or static scenarios, static policy handling, or limited safety reasoning (Chen et al., 6 Aug 2025).

2. Multi-agent architecture and interaction structure

HarmonyGuard comprises three agent roles: a Web Agent, a Policy Agent, and a Utility Agent. The Web Agent is the task executor; the Policy Agent constructs and maintains a structured policy knowledge base from external unstructured documents; the Utility Agent evaluates the Web Agent’s reasoning in real time against safety and utility objectives, and issues optimization guidance when needed (Chen et al., 6 Aug 2025).

Agent	Primary role	Implementation details
Web Agent	Executes web tasks and produces reasoning/action sequences	gpt-4o and gpt-4o-mini
Policy Agent	Extracts, structures, and updates policy knowledge	gpt-4o
Utility Agent	Evaluates policy compliance and goal alignment; generates optimization guidance	Qwen-Max-2025-01-25

All LLMs are run with temperature $0$. Architecturally, the framework is closed-loop rather than one-shot: policy knowledge is extracted from documents, used during online reasoning evaluation, updated from observed violations, and then reused in subsequent interactions. This suggests a persistent policy-memory design rather than a static prompt-only defense (Chen et al., 6 Aug 2025).

The information flow is explicit. The Policy Agent supplies structured policies, policy IDs, scopes, constraints, risk levels, and violation memory to the Utility Agent. The Web Agent sends current and previous reasoning steps plus task context to the Utility Agent. The Utility Agent returns policy-compliance assessments, alignment assessments, optimization guidance, and reflective correction prompts to the Web Agent, while also sending violation references and mapped policy IDs back to the Policy Agent for memory update (Chen et al., 6 Aug 2025).

A common misconception would be to treat HarmonyGuard as an end-to-end trained policy model. The paper does not describe HarmonyGuard that way. It presents the framework as an inference-time multi-agent system built from prompting, external MCP tools, structured memory, similarity filtering, and environment feedback, rather than as a fine-tuned or reinforcement-learned controller (Chen et al., 6 Aug 2025).

3. Adaptive Policy Enhancement

The Policy Agent performs Policy Enhancement and Policy Updating. In the enhancement stage, it extracts text from external files, uses LLM-based processing to identify policy statements, refines and normalizes them, removes duplicates, converts them into structured data, and stores them in a policy database. The framework assumes trusted external documents and trusted MCP servers, and the appendix describes a three-stage workflow: extract raw text, extract individual policy statements, and review extracted policies while removing duplicates (Chen et al., 6 Aug 2025).

The paper identifies three enhancement techniques. LLM Refinement is used for semantic understanding, ambiguity resolution, redundancy removal, and expression normalization. Policy Deduplication uses semantic similarity plus LLM judgment to merge duplicate or highly similar entries. Policy Structuring converts extracted policies into predefined fields, including policy ID, scope of applicability, constraints, and risk level (Chen et al., 6 Aug 2025).

Policy memory is explicitly non-static. When the Utility Agent identifies a violation, it creates a violation reference, maps it to the relevant policy entry, and sends it to the Policy Agent. The update mechanism keeps negative samples—policy violations—as contextual evidence. Duplicate filtering uses semantic similarity based on Gestalt pattern matching, with threshold

$\theta = 0.85.$

If a new violation $v$ is assigned to risk queue $Q_r$, duplicates are identified by

$$Q_r^{\text{dup}} \leftarrow \{u \in Q_r \mid \text{Sim}(v,u) \ge \theta\}.$$

Only novel violations are retained (Chen et al., 6 Aug 2025).

Stored violations are organized in a tiered bounded queue by risk level:

• low: $5$
• medium: $7$
• high: $10$

This queueing policy is intended to preserve timeliness while retaining more examples for higher-risk threats. The update rule is summarized in the paper’s “Feedback-Enhanced Policy Update Pipeline,” whose inputs are the violation set $V$, the similarity threshold $\theta = 0.85$, the risk-level set $\theta = 0.85.$0, and queue-capacity function $\theta = 0.85.$1 (Chen et al., 6 Aug 2025).

4. Dual-objective optimization and reasoning correction

HarmonyGuard’s Utility Agent enforces Dual-Objective Optimization at inference time. The two objectives are safety, defined as policy compliance, and utility, defined as alignment with the task objective. Rather than solving a weighted-sum optimization, the framework applies constraint-style stepwise monitoring and correction. The paper invokes the Constrained Markov Decision Process perspective, but does not present a full CMDP optimization problem, reward function, or solved constrained policy optimization algorithm (Chen et al., 6 Aug 2025).

The central formal device is the Second-Order Markov Evaluation Strategy. If the Web Agent’s reasoning trajectory is

$\theta = 0.85.$2

the Utility Agent evaluates only the local transition $\theta = 0.85.$3. Its decision is represented as a two-dimensional boolean vector

$\theta = 0.85.$4

whose first coordinate indicates whether $\theta = 0.85.$5 detects a policy violation, and whose second coordinate indicates whether $\theta = 0.85.$6 detects goal drift (Chen et al., 6 Aug 2025).

The paper is explicit that HarmonyGuard does not use a weighted scalar reward, Lagrangian penalty, or explicit Pareto solver. In practice, it detects either policy violation or goal deviation, triggers correction whenever either issue appears, and asks the Utility Agent to generate optimization guidance. This guidance is metacognitive: the agent comprehends the input, forms an initial judgment, critically evaluates that judgment, and derives a final decision. The appendix describes the Utility Agent prompt as a three-stage reasoning process: Policy Compliance Check, Alignment Check, and Optimization Guidance (Chen et al., 6 Aug 2025).

The runtime loop is correspondingly structured. External policy documents are prepared first. During task execution, the Utility Agent evaluates each step’s transition, produces $\theta = 0.85.$7, and, if either coordinate signals a problem, generates optimization guidance so that the Web Agent can revise its reasoning. Confirmed violations are then fed back into policy memory. This makes HarmonyGuard a corrector, not merely a blocker (Chen et al., 6 Aug 2025).

5. Benchmarks, metrics, and empirical findings

HarmonyGuard is evaluated on two real-world web-agent security benchmarks built on WebArena and hosted on AWS websites. ST-WebAgentBench contains $\theta = 0.85.$8 tasks and groups safety policies into Consent, Boundary, and Execution. WASP contains $\theta = 0.85.$9 tasks focused on injection attacks on GitHub and Reddit, with four attack types: GitHub Plain Injection (GPI), GitHub URL Injection (GUI), Reddit Plain Injection (RPI), and Reddit URL Injection (RUI). The paper also evaluates WASP (SoM) as a multimodal agent setting based on WASP (Chen et al., 6 Aug 2025).

Benchmark	Size	Focus
ST-WebAgentBench	235 tasks	Consent, Boundary, Execution policies
WASP	84 tasks	Injection attacks on GitHub and Reddit
WASP (SoM)	not separately recounted as tasks	Multimodal setting based on WASP

The baselines are No Defense, Prompt Defense, Policy Traversal, and Guard–Base, where Guard–Base is a base version of HarmonyGuard without policy updating. Evaluation uses three formal metrics. For $v$0 tasks, with $v$1 denoting completion and $v$2 denoting policy compliance,

$v$3

$v$4

$v$5

The paper also analyzes

$v$6

which measures how much task completion depends on policy-violating behavior (Chen et al., 6 Aug 2025).

The abstract reports three headline results: up to 38% improvement in policy compliance, up to 20% improvement in task completion, and over 90% policy compliance across all tasks. Detailed values support this characterization. On ST-WebAgentBench, HarmonyGuard reaches PCR values of 0.925 for Consent, 0.994 for Boundary, and 0.915 for Execution. On WASP, it reaches 1.000 for GPI, 0.905 for GUI, 1.000 for RPI, and 0.952 for RUI. On WASP (SoM), it reaches 1.000 for GPI, 1.000 for GUI, 1.000 for RPI, and 0.952 for RUI (Chen et al., 6 Aug 2025).

Joint performance is captured by CuP. On ST-WebAgentBench, HarmonyGuard achieves 0.047 for Consent, 0.077 for Boundary, and 0.081 for Execution, compared with 0.034, 0.063, and 0.068 under No Defense. On WASP, the reported CuP values are 0.714 for GPI, 0.667 for GUI, 0.905 for RPI, and 0.857 for RUI. On WASP (SoM), HarmonyGuard reaches 0.952 for GPI, 0.762 for GUI, 0.667 for RPI, and 0.571 for RUI (Chen et al., 6 Aug 2025).

The paper also reports a safety–utility Pareto analysis with $v$7-axis PCR and $v$8-axis CuP, and places HarmonyGuard on the Pareto frontier on both ST-WebAgentBench and WASP / WASP(SoM). In the violation-gap analysis, HarmonyGuard has the smallest or zero violation across all benchmarks, which the authors interpret as evidence that its task completion depends less on unsafe behavior than the baselines (Chen et al., 6 Aug 2025).

Two ablation-style findings are central. First, Guard–Base vs HarmonyGuard isolates the contribution of policy updating and shows additional gains from adaptive policy memory. Second, the paper compares evaluation strategies on ST-WebAgentBench with gpt-4o-mini:

• None: $v$9 for PCR / CuP
• Full-Traj.: $Q_r$0
• Cur. Step: $Q_r$1
• Markovian: $Q_r$2

This is used to argue that full-trajectory evaluation over-penalizes and causes false positives, while second-order Markovian reasoning preserves local context without the same degree of overcorrection. In additional multi-round adaptation experiments on WASP, the paper states that performance stabilizes after about three rounds and that the best overall balance appears in the third round (Chen et al., 6 Aug 2025).

6. Interpretation, limitations, and research directions

HarmonyGuard is best understood as a framework for dynamic policy extraction plus stepwise safety–utility reasoning, not as a static prompt wrapper or a conventional conversational moderation filter. This suggests a different design point from lightweight conversational guardrails such as SGuard-v1 (Lee et al., 16 Nov 2025), remediation-oriented planning guards such as TRIAD (Sun et al., 4 Jun 2026), or multimodal input wrappers such as UniGuard (Oh et al., 2024). The distinction is not one of generic safety ambition, but of domain: HarmonyGuard is specialized for long-sequence web agency under evolving external policy constraints (Chen et al., 6 Aug 2025).

The paper is also explicit about limitations. It assumes trusted external policy sources and trusted MCP servers, so it does not solve adversarial corruption of policy inputs. It monitors reasoning chunks rather than token-level behavior. It lacks a pre-reasoning input detector, and uncertainty handling is described as future work rather than a present mechanism. The appendix and conclusion further suggest future extensions in fine-grained real-time policy control, uncertainty-aware mechanisms, ensemble-based decision making, multimodal threat detection, persistent experience-driven memory, and human-in-the-loop supervision (Chen et al., 6 Aug 2025).

A second misconception would be to treat HarmonyGuard as a fully formal optimal controller. The paper invokes CMDP language and uses a dual-objective evaluation vector, but does not give a theorem-level constrained optimization solution or an end-to-end trained policy. Its contribution is instead an inference-time control architecture with structured policy memory, second-order Markovian reasoning evaluation, and metacognitive correction (Chen et al., 6 Aug 2025).

Within the 2025 literature, HarmonyGuard’s strongest distinctive property is the way it links adaptive policy memory to online reasoning correction. The paper’s practical claim is not merely that web agents should be blocked when they become unsafe, but that they should be corrected while preserving task progress when possible. In that sense, HarmonyGuard is presented as an architecture for web agents that are more capable under evolving safety constraints, rather than simply more restrictive (Chen et al., 6 Aug 2025).