MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolution

Abstract: GUI agents rely on screenshots to infer intent and operate across applications, but these screenshots often contain private messages, medical records, payment credentials, and workplace-specific workflows. Privacy decisions in this setting depend on task, recipient, application state, and user role, yet static PII detectors miss these boundaries and cloud-side VLM reasoning can upload the raw screen before deciding what should be protected. We present MaskClaw, an edge-side privacy arbitrator for GUI agents. MaskClaw extracts local visual evidence, retrieves user- and task-specific policy memory, and decides Allow, Mask, or Ask before raw screenshots leave a trusted user- or organization-controlled environment. In five designed skill-evolution scenarios, it turns corrections, cancellations, and edits into reusable privacy skills checked by a sandbox gate. We introduce P-GUI-Evo, a benchmark built from real UI patterns, reconstructed HTML screens, and sanitized labels. Experiments show that pattern matching, cloud reasoning, and routing alone tend to over-confirm, over-mask, or expose raw screenshots under the same protocol. The artifact is available at https://github.com/Theodora-Y/MaskClaw.

• The paper introduces MaskClaw, an edge-side privacy arbitration framework that uses local evidence extraction and rule-based policies to decide Allow, Mask, or Ask actions.
• It employs a retrieval-augmented mechanism with behavior-driven skill evolution that significantly improves decision accuracy and reduces leak rates compared to traditional methods.
• Experimental results on the P-GUI-Evo benchmark confirm MaskClaw’s superior performance in task-sensitive privacy handling and safe skill evolution.

Edge-Side Privacy Arbitration for GUI Agents: An Analysis of MaskClaw

Motivation and Problem Setting

Graphical user interface (GUI) agents increasingly rely on screenshots to operate across applications and infer user intent. However, screenshots frequently contain sensitive information, including private messages, medical records, payment credentials, and task-irrelevant user data. Existing privacy mechanisms, such as static PII detectors or cloud-side VLM masking, are inadequate for capturing context-specific boundaries and protecting privacy before exposure occurs. MaskClaw directly addresses this bottleneck by establishing privacy arbitration on the edge side, deciding whether to allow, mask, or require user confirmation before screenshot content leaves a trusted environment.

Figure 1: Overview of the motivation and problem setting—GUI agents require screenshots to function, but screenshots can leak private content. MaskClaw arbitrates Allow/Mask/Ask decisions at the edge.

MaskClaw Architecture and Framework

MaskClaw is structured as a modular, edge-side privacy arbitrator, composed of perception, rule-grounded policy arbitration, SafeScreenshot construction, and skill evolution components. The edge controller keeps the raw screenshots local, extracts structured visual evidence, retrieves user- and task-conditioned policy rules, and arbitrates exposure actions for each interaction.

Figure 2: MaskClaw architecture—local screenshots are processed to extract evidence, retrieve persona-task policy memory, and arbitrate exposure. Outputs are allowed screenshots, SafeScreenshots, or confirmation requests, with skill evolution supported by user feedback.

Local Perception and Evidence Extraction

Perception locally extracts structured evidence (OCR text, field type, semantic description, bounding box) prior to any exposure, supporting precise arbitration and region-level masking.

Rule-Grounded Policy Arbitration

A retrieval-augmented, context-conditioned policy memory is used to arbitrate Allow/Mask/Ask decisions. The arbitration interface supports deterministic mapping based on extracted evidence, application context, persona, recipient, and agent intent. More specific rules override generic ones, and fallback logic ensures conservative handling when no applicable rule is retrieved.

SafeScreenshot Construction

Under Mask, MaskClaw selects protected regions for redaction and locally constructs mediated screenshots that preserve task-relevant layout while masking sensitive content. Under Ask, user confirmation is prompted, and under Allow, downstream agents receive the raw screenshot.

Figure 3: Qualitative SafeScreenshot examples—OCR localization and masking applied to synthetic bank-card and address-list UI fields while preserving non-sensitive structure.

Behavior-Driven Skill Evolution

MaskClaw implements a feedback-to-skill loop, turning user corrections, cancellations, and edits into reusable privacy skills. Candidate updates are subject to structured textual scoring and sandbox audits, ensuring compliance with coverage, safety, and flow constraints before entering auditable policy memory.

Benchmark: P-GUI-Evo

The study introduces P-GUI-Evo, a benchmark comprised of 832 controlled GUI privacy samples derived from HTML-reconstructed UIs and sanitized sensitive labels. It operationalizes personalized privacy decisions at the interface level, evaluating Allow/Mask/Ask arbitration under varying personas, tasks, recipients, and UI perturbations. Scenario variants maintain consistent privacy boundaries across structural, visual, and lexical task shifts.

Figure 4: Representative benchmark samples—Ask/Allow retain original UI; Mask shows before/after SafeScreenshot comparison (synthetic).

Experimental Analysis

Layer-wise Diagnostics and Results

Policy-grounded MaskClaw outperforms all baselines in joint decision accuracy and Mask F1, with a policy accuracy of $0.717{\pm}0.007$ compared to the strongest non-MaskClaw baseline (Regex) at $0.557$ accuracy. MaskClaw also reduces leak rate (misclassified Mask instances as Allow/Ask) to $0.196$ (from Regex's $0.283$), and Ask recall increases sharply, differentiating consent-sensitive cases.

The results demonstrate that cloud-based routes, devoid of policy context, consistently over-confirm or expose raw screenshots (leak rate $>0.54$ for all cloud methods), while routing proxies fail to resolve contextual arbitration.

Skill Evolution and Flow-Level Safety

The skill-evolution experiment distills user corrections into textual skills, achieving BestScore@20 averages of 88.76 across controlled scenarios, with downstream behavior accuracy consistently improving and unsafe action rates reduced.

Figure 5: Skill-use checks—evolved privacy skills improve downstream behavior in five controlled scenarios.

Sandbox Validation

Despite strong textual privacy scores, only a narrow fraction of evolved skills pass sandbox flow-level audits, with the majority failing due to unsafe state transitions or inadequate confirmation logic. This substantiates the necessity for deployment-oriented filtering beyond text-level optimization.

Figure 6: Post-evolution filtering—substantial attrition from strict text scoring to sandbox gate due to execution flow and safety misalignment.

Implications and Future Directions

MaskClaw demonstrates that edge-side, context-conditioned privacy arbitration with behavior-driven skill evolution is superior to pattern-based detection, cloud-only reasoning, or routing proxies for GUI-agent privacy. The deterministic mediation interface and retrieval-augmented rule memory are instrumental in avoiding unnecessary masking and preventing task-irrelevant exposure. The feedback-to-skill loop reinforces the adaptability of privacy policies without compromising on flow-level safety.

Practically, MaskClaw enables organizations and users to protect sensitive workflow data during agent interactions, supporting compliance and minimizing privacy risk. Theoretically, its architecture underscores the value of explicit policy memory, auditable arbitration interfaces, and validated skill evolution. MaskClaw's approach is readily extensible to other multimodal agentic domains, suggesting future work on broader personalization, adaptive privacy boundaries, federated skill sharing, and richer scenario generalization.

The limiting factor in applying skills at scale remains robust applicability judgment and execution gating; safe skill transfer requires continued research on fine-grained boundary recognition and automated sandbox audits.

Conclusion

MaskClaw represents a rigorous edge-side privacy framework for GUI agents, solving contextual exposure bottlenecks with auditable, policy-grounded arbitration and safe skill evolution. Controlled benchmarks and layer-wise diagnostics validate its superiority over static, cloud, and route-based baselines. The skill-evolution mechanism is effective at scenario level, and sandbox validation is essential for flow-level safety. The framework's explicit edge-side arbitration concretely advances deployable privacy controls for agentic systems (2605.28646).