---
title: 'HarmonyGuard: Multi-Agent Web Safety'
url: https://www.emergentmind.com/topics/harmonyguard
type: topic
---

# HarmonyGuard: Multi-Agent Web Safety

HarmonyGuard is a multi-agent collaborative framework for LLM-based web agents operating in open, dynamic web environments. It is designed to jointly improve safety and utility during long-sequence web interaction through two named capabilities: **Adaptive Policy Enhancement** and **Dual-Objective Optimization**. The framework couples a Web Agent that executes tasks, a Policy Agent that extracts and updates structured security policies from unstructured external documents, and a Utility Agent that evaluates each reasoning step for policy compliance and task alignment, then supplies optimization guidance when either objective is violated [2508.04010].

## 1. Problem domain and design objective

HarmonyGuard is formulated for web agents that perform multi-step tasks such as shopping, booking, browsing, and interacting with web interfaces over many reasoning and execution steps. The paper characterizes the web environment as **open-ended**, **dynamic**, **long-horizon**, and **partially adversarial**, with threats including prompt injection, environment injection, knowledge poisoning, tool library or MCP hijacking, phishing links, pop-ups and notifications, and malicious scripts embedded in web pages [2508.04010].

The framework is motivated by a stated tension between **utility** and **safety**. If an agent pursues utility too aggressively, it may follow unsafe instructions, click malicious links, leak information, or violate operational constraints. If it pursues safety too conservatively, it may refuse too many actions, overcorrect, or stall, causing utility collapse. The paper argues that this trade-off becomes more severe in long-sequence settings, where small misalignments can compound into “risk amplification cascades” [2508.04010].

Two challenges are singled out. The first is **Safety–Utility Disconnection**: security policies evolve with threats, but are often embedded in regulatory documents, organizational guidelines, and other unstructured external text. The second is **Safety–Utility Trade-off**: safety and utility cannot be optimized independently during long-horizon web tasks. HarmonyGuard is presented as an answer to both challenges, and is positioned against prior work that, in the paper’s characterization, is limited by single-objective optimization, single-turn or static scenarios, static policy handling, or limited safety reasoning [2508.04010].

## 2. Multi-agent architecture and interaction structure

HarmonyGuard comprises three agent roles: a **Web Agent**, a **Policy Agent**, and a **Utility Agent**. The Web Agent is the task executor; the Policy Agent constructs and maintains a structured policy knowledge base from external unstructured documents; the Utility Agent evaluates the Web Agent’s reasoning in real time against safety and utility objectives, and issues optimization guidance when needed [2508.04010].

| Agent | Primary role | Implementation details |
|---|---|---|
| Web Agent | Executes web tasks and produces reasoning/action sequences | `gpt-4o` and `gpt-4o-mini` |
| Policy Agent | Extracts, structures, and updates policy knowledge | `gpt-4o` |
| Utility Agent | Evaluates policy compliance and goal alignment; generates optimization guidance | `Qwen-Max-2025-01-25` |

All LLMs are run with temperature \(0\). Architecturally, the framework is closed-loop rather than one-shot: policy knowledge is extracted from documents, used during online reasoning evaluation, updated from observed violations, and then reused in subsequent interactions. This suggests a persistent policy-memory design rather than a static prompt-only defense [2508.04010].

The information flow is explicit. The Policy Agent supplies structured policies, policy IDs, scopes, constraints, risk levels, and violation memory to the Utility Agent. The Web Agent sends current and previous reasoning steps plus task context to the Utility Agent. The Utility Agent returns policy-compliance assessments, alignment assessments, optimization guidance, and reflective correction prompts to the Web Agent, while also sending violation references and mapped policy IDs back to the Policy Agent for memory update [2508.04010].

A common misconception would be to treat HarmonyGuard as an end-to-end trained policy model. The paper does not describe HarmonyGuard that way. It presents the framework as an inference-time multi-agent system built from prompting, external MCP tools, structured memory, similarity filtering, and environment feedback, rather than as a fine-tuned or reinforcement-learned controller [2508.04010].

## 3. Adaptive Policy Enhancement

The Policy Agent performs **Policy Enhancement** and **Policy Updating**. In the enhancement stage, it extracts text from external files, uses LLM-based processing to identify policy statements, refines and normalizes them, removes duplicates, converts them into structured data, and stores them in a policy database. The framework assumes trusted external documents and trusted MCP servers, and the appendix describes a three-stage workflow: extract raw text, extract individual policy statements, and review extracted policies while removing duplicates [2508.04010].

The paper identifies three enhancement techniques. **LLM Refinement** is used for semantic understanding, ambiguity resolution, redundancy removal, and expression normalization. **Policy Deduplication** uses semantic similarity plus LLM judgment to merge duplicate or highly similar entries. **Policy Structuring** converts extracted policies into predefined fields, including **policy ID**, **scope of applicability**, **constraints**, and **risk level** [2508.04010].

Policy memory is explicitly non-static. When the Utility Agent identifies a violation, it creates a violation reference, maps it to the relevant policy entry, and sends it to the Policy Agent. The update mechanism keeps negative samples—policy violations—as contextual evidence. Duplicate filtering uses semantic similarity based on Gestalt pattern matching, with threshold
$$
\theta = 0.85.
$$
If a new violation \(v\) is assigned to risk queue \(Q_r\), duplicates are identified by
$$
Q_r^{\text{dup}} \leftarrow \{u \in Q_r \mid \text{Sim}(v,u) \ge \theta\}.
$$
Only novel violations are retained [2508.04010].

Stored violations are organized in a **tiered bounded queue** by risk level:
- low: \(5\)
- medium: \(7\)
- high: \(10\)

This queueing policy is intended to preserve timeliness while retaining more examples for higher-risk threats. The update rule is summarized in the paper’s “Feedback-Enhanced Policy Update Pipeline,” whose inputs are the violation set \(V\), the similarity threshold \(\theta = 0.85\), the risk-level set \(\mathcal{R}=\{\text{low},\text{medium},\text{high}\}\), and queue-capacity function \(L\) [2508.04010].

## 4. Dual-objective optimization and reasoning correction

HarmonyGuard’s Utility Agent enforces **Dual-Objective Optimization** at inference time. The two objectives are **safety**, defined as policy compliance, and **utility**, defined as alignment with the task objective. Rather than solving a weighted-sum optimization, the framework applies constraint-style stepwise monitoring and correction. The paper invokes the **Constrained Markov Decision Process** perspective, but does not present a full CMDP optimization problem, reward function, or solved constrained policy optimization algorithm [2508.04010].

The central formal device is the **Second-Order Markov Evaluation Strategy**. If the Web Agent’s reasoning trajectory is
$$
\{r_1, r_2, \dots, r_t\},
$$
the Utility Agent evaluates only the local transition \((r_{t-1}, r_t)\). Its decision is represented as a two-dimensional boolean vector
$$
\mathcal{R}(r_t \mid r_{t-1}) \in \{0,1\}^2,
$$
whose first coordinate indicates whether \(f_\theta^{\text{policy}}(r_{t-1}, r_t)\) detects a policy violation, and whose second coordinate indicates whether \(f_\theta^{\text{goal}}(r_{t-1}, r_t)\) detects goal drift [2508.04010].

The paper is explicit that HarmonyGuard does **not** use a weighted scalar reward, Lagrangian penalty, or explicit Pareto solver. In practice, it detects either policy violation or goal deviation, triggers correction whenever either issue appears, and asks the Utility Agent to generate optimization guidance. This guidance is metacognitive: the agent comprehends the input, forms an initial judgment, critically evaluates that judgment, and derives a final decision. The appendix describes the Utility Agent prompt as a three-stage reasoning process: **Policy Compliance Check**, **Alignment Check**, and **Optimization Guidance** [2508.04010].

The runtime loop is correspondingly structured. External policy documents are prepared first. During task execution, the Utility Agent evaluates each step’s transition, produces \(\mathcal{R}(r_t \mid r_{t-1})\), and, if either coordinate signals a problem, generates optimization guidance so that the Web Agent can revise its reasoning. Confirmed violations are then fed back into policy memory. This makes HarmonyGuard a corrector, not merely a blocker [2508.04010].

## 5. Benchmarks, metrics, and empirical findings

HarmonyGuard is evaluated on two real-world web-agent security benchmarks built on **WebArena** and hosted on **AWS websites**. **ST-WebAgentBench** contains \(235\) tasks and groups safety policies into **Consent**, **Boundary**, and **Execution**. **WASP** contains \(84\) tasks focused on injection attacks on **GitHub** and **Reddit**, with four attack types: **GitHub Plain Injection (GPI)**, **GitHub URL Injection (GUI)**, **Reddit Plain Injection (RPI)**, and **Reddit URL Injection (RUI)**. The paper also evaluates **WASP (SoM)** as a multimodal agent setting based on WASP [2508.04010].

| Benchmark | Size | Focus |
|---|---:|---|
| ST-WebAgentBench | 235 tasks | Consent, Boundary, Execution policies |
| WASP | 84 tasks | Injection attacks on GitHub and Reddit |
| WASP (SoM) | not separately recounted as tasks | Multimodal setting based on WASP |

The baselines are **No Defense**, **Prompt Defense**, **Policy Traversal**, and **Guard–Base**, where Guard–Base is a base version of HarmonyGuard without policy updating. Evaluation uses three formal metrics. For \(N\) tasks, with \(C_i \in \{0,1\}\) denoting completion and \(P_i \in \{0,1\}\) denoting policy compliance,
$$
Completion = \frac{1}{N} \sum_{i=1}^N C_i,
$$
$$
PCR = \frac{1}{N} \sum_{i=1}^N P_i,
$$
$$
CuP = \frac{1}{N} \sum_{i=1}^N (C_i \times P_i).
$$
The paper also analyzes
$$
violation = Completion - CuP,
$$
which measures how much task completion depends on policy-violating behavior [2508.04010].

The abstract reports three headline results: **up to 38% improvement in policy compliance**, **up to 20% improvement in task completion**, and **over 90% policy compliance across all tasks**. Detailed values support this characterization. On **ST-WebAgentBench**, HarmonyGuard reaches PCR values of **0.925** for Consent, **0.994** for Boundary, and **0.915** for Execution. On **WASP**, it reaches **1.000** for GPI, **0.905** for GUI, **1.000** for RPI, and **0.952** for RUI. On **WASP (SoM)**, it reaches **1.000** for GPI, **1.000** for GUI, **1.000** for RPI, and **0.952** for RUI [2508.04010].

Joint performance is captured by **CuP**. On ST-WebAgentBench, HarmonyGuard achieves **0.047** for Consent, **0.077** for Boundary, and **0.081** for Execution, compared with **0.034**, **0.063**, and **0.068** under No Defense. On WASP, the reported CuP values are **0.714** for GPI, **0.667** for GUI, **0.905** for RPI, and **0.857** for RUI. On WASP (SoM), HarmonyGuard reaches **0.952** for GPI, **0.762** for GUI, **0.667** for RPI, and **0.571** for RUI [2508.04010].

The paper also reports a safety–utility Pareto analysis with \(x\)-axis PCR and \(y\)-axis CuP, and places HarmonyGuard on the **Pareto frontier** on both ST-WebAgentBench and WASP / WASP(SoM). In the violation-gap analysis, HarmonyGuard has the **smallest or zero violation** across all benchmarks, which the authors interpret as evidence that its task completion depends less on unsafe behavior than the baselines [2508.04010].

Two ablation-style findings are central. First, **Guard–Base vs HarmonyGuard** isolates the contribution of policy updating and shows additional gains from adaptive policy memory. Second, the paper compares evaluation strategies on ST-WebAgentBench with `gpt-4o-mini`:
- **None**: \(0.824 / 0.052\) for PCR / CuP
- **Full-Traj.**: \(0.869 / 0.042\)
- **Cur. Step**: \(0.867 / 0.056\)
- **Markovian**: \(0.867 / 0.060\)

This is used to argue that full-trajectory evaluation over-penalizes and causes false positives, while second-order Markovian reasoning preserves local context without the same degree of overcorrection. In additional multi-round adaptation experiments on WASP, the paper states that performance stabilizes after about three rounds and that the best overall balance appears in the third round [2508.04010].

## 6. Interpretation, limitations, and research directions

HarmonyGuard is best understood as a framework for **dynamic policy extraction plus stepwise safety–utility reasoning**, not as a static prompt wrapper or a conventional conversational moderation filter. This suggests a different design point from lightweight conversational guardrails such as SGuard-v1 [2511.12497], remediation-oriented planning guards such as TRIAD [2606.05805], or multimodal input wrappers such as UniGuard [2411.01703]. The distinction is not one of generic safety ambition, but of domain: HarmonyGuard is specialized for long-sequence web agency under evolving external policy constraints [2508.04010].

The paper is also explicit about limitations. It assumes **trusted external policy sources** and **trusted MCP servers**, so it does not solve adversarial corruption of policy inputs. It monitors reasoning chunks rather than token-level behavior. It lacks a pre-reasoning input detector, and uncertainty handling is described as future work rather than a present mechanism. The appendix and conclusion further suggest future extensions in **fine-grained real-time policy control**, **uncertainty-aware mechanisms**, **ensemble-based decision making**, **multimodal threat detection**, **persistent experience-driven memory**, and **human-in-the-loop supervision** [2508.04010].

A second misconception would be to treat HarmonyGuard as a fully formal optimal controller. The paper invokes CMDP language and uses a dual-objective evaluation vector, but does not give a theorem-level constrained optimization solution or an end-to-end trained policy. Its contribution is instead an inference-time control architecture with structured policy memory, second-order Markovian reasoning evaluation, and metacognitive correction [2508.04010].

Within the 2025 literature, HarmonyGuard’s strongest distinctive property is the way it links **adaptive policy memory** to **online reasoning correction**. The paper’s practical claim is not merely that web agents should be blocked when they become unsafe, but that they should be corrected while preserving task progress when possible. In that sense, HarmonyGuard is presented as an architecture for web agents that are more capable **under evolving safety constraints**, rather than simply more restrictive [2508.04010].

Source: https://www.emergentmind.com/topics/harmonyguard