MaskClaw: Edge Privacy for GUI Agents

Updated 4 July 2026

MaskClaw is a privacy arbitrator for GUI agents that determines if screenshots should be allowed, masked, or require user confirmation.
It leverages contextual evidence and local policy rules to execute runtime arbitration before any raw screenshot leaves a trusted environment.
The system incorporates behavior-driven skill evolution and personalized policy memory to enhance privacy accuracy and minimize data leakage.

MaskClaw is an edge-side privacy arbitrator for GUI agents that rely on screenshots as their primary observation channel. It is designed for the setting in which a GUI agent must decide, before any raw screenshot leaves a trusted user- or organization-controlled environment, whether the current observation may be released unchanged, must be transformed locally, or requires explicit user confirmation. In "MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolution" (Zhao et al., 27 May 2026), this problem is formalized as contextual exposure arbitration rather than generic PII detection, and is coupled to a behavior-driven mechanism that converts corrections, cancellations, and edits into reusable privacy skills.

1. Problem formulation and privacy boundary

MaskClaw addresses a privacy bottleneck specific to screenshot-driven GUI agents. The motivating claim is that screenshots needed for automation may simultaneously contain private messages, medical records, payment credentials, financial tables, and workplace-specific workflows. The paper argues that privacy in this setting is not reducible to static PII detection, because the same visible content may be releasable in one workflow and sensitive in another, depending on task, recipient, application state, user role, and organization-specific boundaries. It also identifies a cloud-dependency paradox: if privacy reasoning is delegated to a cloud VLM, the system may upload the raw screenshot before deciding whether it should have been protected, which already violates the intended boundary (Zhao et al., 27 May 2026).

The prediction instance is written as

$x_i=(I_i,q_i,a_i,c_i,p_i),$

where $I_i$ is the screenshot, $q_i$ the instruction or task goal, $a_i$ the candidate intent or next action, $c_i$ the application/platform and relational context, and $p_i$ the persona and policy context. The output space is fixed as

$\mathcal{Y}=\{\mathrm{Allow},\mathrm{Mask},\mathrm{Ask}\}.$

These three outcomes define the exposure boundary. Allow permits release on the controlled observation route; Mask requires a local transformation into a SafeScreenshot before anything is sent onward; Ask pauses execution and returns a confirmation request with rationale instead of forwarding the screenshot.

The trusted boundary is operational rather than purely conceptual. The paper states that the edge side can be the endpoint itself or a trusted local environment such as a phone, PC, workstation, personal server, private enterprise server, in-vehicle system, or smart-home device. Raw screenshots remain inside that boundary; only allowed or locally transformed observations may cross it.

2. Runtime arbitration pipeline

MaskClaw has an online loop for runtime arbitration. At state $s$ , it first receives the current GUI screenshot together with the user instruction, candidate agent intent, application/platform context, recipient or relationship context, and persona context. Audit-only labels are withheld during evaluation.

Local perception then extracts structured evidence

$E(s)=\{e_1,e_2,\ldots,e_n\}.$

Each evidence item may contain OCR text, field type, semantic description, confidence, and, when available, a bounding box. This distinction is important because localizable evidence can later be turned into masking regions, whereas non-localized semantic evidence still informs arbitration.

Policy memory is represented as

$C_t=\{r_1,r_2,\ldots,r_m\},$

with each rule

$I_i$ 0

Here $I_i$ 1 is the scope, covering application, persona or role, recipient boundary, and intended action; $I_i$ 2 is the trigger evidence; $I_i$ 3 is the recommended exposure action; $I_i$ 4 is conflict-resolution priority; and $I_i$ 5 is a short rationale. Retrieval uses application, persona, recipient boundary, intended action, and local visual evidence $I_i$ 6 to produce a candidate rule set $I_i$ 7. The retrieved rules are ranked by facet match and evidence compatibility, with recipient, action, and persona weighted above generic field-type matches.

The actual decision layer is mediator-grounded. The finalized L2 Closed interface is summarized as

$I_i$ 8

The deterministic mediator owns the policy fields and locks the selected rule id, mask flag, and confirmation flag. The local VLM is then limited to adding visual evidence, a contradiction flag, and rationale; it does not overwrite the mediator-owned policy outputs. The decision rule is

$I_i$ 9

This defines a strict precedence relation: confirmation overrides masking, and masking overrides allowing.

When the decision is Mask, MaskClaw chooses protected regions

$q_i$ 0

where $q_i$ 1 is the set of localizable regions extracted from perception, and produces a mediated screenshot $q_i$ 2 via $q_i$ 3. The paper states that redaction can blur, mask, replace, or otherwise transform the protected regions locally. Under Allow, the observation proceeds; under Mask, the downstream agent receives the mediated screenshot; under Ask, execution pauses and a confirmation request is returned (Zhao et al., 27 May 2026).

3. Personalized policy memory and behavior-driven skill evolution

Personalization in MaskClaw is implemented through explicit, inspectable local policy memory rather than an opaque latent user profile. The rule schema $q_i$ 4 makes app, persona, recipient boundary, intended action, trigger evidence, action, priority, and rationale all first-class objects. This allows narrow exceptions to override general rules through priority and scope specificity.

The system’s second loop is behavior-driven skill evolution. Runtime traces are represented as

$q_i$ 5

where $q_i$ 6 is GUI state, $q_i$ 7 the system decision, and $q_i$ 8 downstream actions and feedback, including confirmations, cancellations, edits, rejected actions, retries, or explicit skill instructions. From these traces the system extracts correction signals

$q_i$ 9

builds candidate updates $a_i$ 0, and commits accepted changes through

$a_i$ 1

The evolution mechanism is evaluated on five designed scenario families: iCloud cleanup, App permission, High-value transfer, Calendar merge, and Driving mode. In each, user corrections are interpreted as reusable privacy or authorization constraints. The paper describes the optimizer as controlled hill climbing over text memory, with 5 scenarios, 3 start conditions, 3 random seeds, and 20 evolution iterations. The objective is

$a_i$ 2

with

$a_i$ 3

Candidate skills are not deployed directly. They must pass a three-stage audit: a schema-and-coverage gate, a fixed text scorer, and an LLM-Judge or sandbox gate. The sandbox gate is specifically described as checking confirmation timing, safety rationale, and executable state flow for downstream agents. This means evolved skills are treated as inspectable policy artifacts rather than latent policy adaptation (Zhao et al., 27 May 2026).

4. P-GUI-Evo benchmark

MaskClaw is evaluated on P-GUI-Evo, a benchmark for personalized GUI privacy arbitration and behavior-driven skill evolution. It is built from real UI patterns, reconstructed HTML screens, and sanitized labels. The construction pipeline uses common GUI interaction patterns and platform-like layouts, placeholders or fake identities, LLM-assisted HTML/scenario drafting, project-author plausibility and fluency audits, and OCR plus PII checks to ensure consistency between rendered screenshots and metadata.

Each sample contains a GUI screenshot, app/platform, persona context, relationship context, user instruction, agent intent, benchmark bucket, and expected Allow/Mask/Ask decision. Offline-only metadata include PII type, risk tags, lower-level audit labels, localizable region information, and annotation notes.

Aspect	Value	Note
GUI privacy samples	832	Main benchmark
Normalized scenario cores	296	Average 2.81 samples/core
Labels	Mask 438, Allow 314, Ask 80	Three-way decision
Personas	User A 234, User B 246, User C 352	Healthcare; commerce/livestream; office/everyday
Buckets	D1 174, D2 546, D3 112	Basic; generalization; stress

The benchmark also reports 715 samples with non-None PII type, 685 location-ready PII evidence samples, 1,529 localizable evidence boxes, and 736 L1/L3 usable samples. The visible-PII and masking subsets contain 620 visible-PII screenshots in visual triage and 558 visible-PII screenshots used for strict L3 SafeScreenshot audit. D3 stress cases include OCR noise, occlusion, low resolution, mixed languages, popup interference, and multiple sensitive regions (Zhao et al., 27 May 2026).

5. Empirical performance

The paper decomposes MaskClaw into diagnostic layers. Local perception reaches L1 OCR coverage of $a_i$ 4 normalized exact OCR text coverage, with 1,446 of 1,496 exact sensitive-text matches and 99.96% character recall. The policy layer shows a substantial difference between open and mediator-grounded arbitration: L2 Open reaches $a_i$ 5 policy accuracy, whereas L2 Closed reaches $a_i$ 6. L3 strict redaction attains a $a_i$ 7 strict no-flag rate (Zhao et al., 27 May 2026).

The main route-level comparison uses 832 samples. The strongest baseline is Static Regex, which the paper characterizes as a competitive lower bound because visible-pattern detection catches many obvious spans, but it has no task, persona, recipient, or policy semantics and cannot produce Ask at all. Relative to this baseline, policy-grounded MaskClaw improves accuracy by 0.160 absolute from $a_i$ 8 to $a_i$ 9, improves Mask F1 by 0.174, and reduces leak by 0.087. It also keeps raw upload at 0, whereas the cloud baselines upload every raw screenshot under the benchmark protocol (Zhao et al., 27 May 2026).

System	Key outcome	Raw upload
Policy-grounded MaskClaw	Accuracy $c_i$ 0	0
Static Regex	Accuracy $c_i$ 1	0
EdgeClaw-ClawXRouter	Accuracy $c_i$ 2	159
Cloud Minimal	Accuracy $c_i$ 3	1000
Cloud Persona	Accuracy $c_i$ 4	1000
Cloud Context-Complete Persona	Accuracy $c_i$ 5	1000

The cloud baselines are diagnostically important because they separate semantic reasoning from privacy-boundary preservation. The paper notes that Cloud Context-Complete Persona predicted Ask for 754/832 samples and is dominated by over-confirmation. EdgeClaw-ClawXRouter reduces raw-cloud upload to 15.9%, but the paper argues that routing alone does not solve contextual arbitration.

The strict SafeScreenshot audit on 558 visible-PII screenshots reports 475 generated masks, 409 automatic no-flag cases, and 149 targeted-audit flags. Among the flags, there are 145 missed sensitive-token flags, 83 no-OCR-match flags, and 5 overmask-suspect flags. A recall-first variant masks 542 of 558 rows but reduces no-flag cases to 338 of 558, whereas the strict variant masks fewer rows and improves no-flag cases. This documents an explicit recall-versus-overmask trade-off in local screenshot mediation.

Skill evolution is evaluated separately. The paper reports BestScore@20 macro average = 88.76, with per-scenario values ranging from 77.70 to 93.67. Behavior accuracy rises from a macro-average 24.00 to 100.00, while the unsafe action rate falls from 76.00 to 2.50.

Scenario	BestScore@20	Unsafe rate: base $c_i$ 6 evolved
iCloud photo cleanup	86.67	62.50 $c_i$ 7 12.50
App permission	93.67	75.00 $c_i$ 8 0.00
High-value transfer	92.96	80.00 $c_i$ 9 0.00
Calendar merge	77.70	87.50 $p_i$ 0 0.00
Driving mode	92.78	75.00 $p_i$ 1 0.00

The paper also emphasizes that compliance remains imperfect, especially in transfer and driving mode, so strong text-level skills can still miss evidence, wording, or action-order constraints.

6. Failure modes, limitations, and broader context

The dominant residual failures are protected content allowed in 95 cases, unnecessary masking in 58 cases, and confirmation replaced by masking in 37 cases; together these account for 79.5% of all joint errors. Retrieval diagnostics report 239 wrong-boundary retrieval cases, 8 weak-retrieval-evidence cases, and 585 cases that fit neither category, and the paper highlights that the 239 wrong-boundary retrieval count equals the 239 joint errors in the finalized interface analysis. The intended conclusion is that many failures come from retrieving a topically related rule aligned to the wrong policy boundary rather than from total retrieval failure (Zhao et al., 27 May 2026).

The paper is also explicit about scope. P-GUI-Evo is sanitized and reconstructed rather than an in-the-wild deployment benchmark; personas are synthetic abstractions; skill-evolution claims are scenario-level rather than long-term personalization in the wild; and no end-to-end latency claims are made because live deployment would additionally involve screenshot capture, model loading, network variability, GUI execution, and user-confirmation latency. It further states that evolved skills may encode user preferences and must remain inspectable, and that high-risk uses such as medical, financial, identity, compliance, or irreversible actions need separate professional and legal review.

Within the broader Claw-style literature, ClawGym defines Claw-style environments as multi-step workflows over local files, tools, and persistent workspace states (Bai et al., 29 Apr 2026), while SafeClawArena argues that Claw-like agents are always-on processes with persistent access to credentials, files, tools, and external services (Niu et al., 29 Jun 2026). This suggests that MaskClaw is best understood not as a replacement for downstream GUI-agent planning, but as a privacy-control layer inserted before screenshot observations enter a larger Claw-style agent stack. That role is narrower than full agent orchestration, but more stringent than ordinary redaction: the system’s fixed Allow/Mask/Ask interface, local policy memory, and sandbox-gated skill evolution are all organized around keeping the raw screenshot inside the trusted boundary until a contextual exposure decision has been made.