MobileRisk-Live: VLM Safety Framework

• MobileRisk-Live is a dynamic framework that integrates real-time safety detection with system-level monitoring for VLM-driven mobile GUI agents.
• It employs a layered architecture—including agent interface, system state trace, and safety detector modules—to rigorously analyze and block unsafe actions.
• Benchmark results demonstrate significant improvements in detection accuracy and F1 scores, supporting robust safety research in adversarial mobile environments.

MobileRisk-Live is a modular, dynamic sandbox and protection framework developed for evaluating, monitoring, and enhancing the safety of vision-LLM (VLM)-driven mobile GUI agents. It integrates real-time jailbreak detection, system-level monitoring, and automated risk scoring in environments spanning multiple third-party and system apps. Originally inspired by and extending methodologies from the SafeMobile system, MobileRisk-Live targets the acute challenge of unsafe agent operations—especially those induced by sophisticated adversarial input sequences—by providing the infrastructure, detection mechanisms, benchmarks, and evaluation protocols necessary for rigorous agent safety research (Sun et al., 28 Oct 2025, Liang et al., 1 Jul 2025).

1. Architectural Design and Core Subsystems

MobileRisk-Live’s architecture is realized on top of Android emulators and is organized into three principal layers:

1. Agent Interface Layer: This layer exposes an action API (e.g., click, swipe, type, open_app, navigation) and communicates GUI observations at each time step, denoted as $o_t = (\text{screenshot}_t, \text{a11ytree}_t)$, enabling precise feedback to autonomous agents.
2. System State Trace Layer: At each step $t$, MobileRisk-Live records a cryptographically hashed snapshot of the runtime environment:

$$\sigma_t = \left(h_t^{\mathrm{fs}}, h_t^{\mathrm{text}}\right)$$

where $h_t^{\mathrm{fs}} = \mathrm{SHA256}(\mathcal{F}_t)$ encodes aggregated file-system metadata and $$h_t^{\mathrm{text}} = \mathrm{SHA256}(\text{UI texts and resource IDs})$$ captures the GUI-visible textual state.

1. Safety Detector Interface: This interface supports real-time plug-in of both rule-based verifiers and VLM-based judges, who access tuples $(o_t, a_t, \sigma_t)$ to deliver safety judgments per action.

The system comes prepopulated with 48 applications to simulate realistic operational coverage and supports complete environment resets for reproducibility. All actions and side-effects—network I/O, permission changes, file operations—are captured for downstream risk analysis (Sun et al., 28 Oct 2025).

2. Threat Model and Evaluation Objectives

MobileRisk-Live adopts the formal threat model introduced in SafeMobile (Liang et al., 1 Jul 2025):

• Agent Model: The mobile multimodal agent executes policies of the form $$a_t = f_m(x^{\text{text}}, x^{\text{gui}_t}, h_{t-1})$$, with access to user instructions, GUI state, and history. The action space is partitioned into safe and unsafe subsets.
• Attacker Capabilities: The adversary may issue arbitrary multimodal inputs—crafted text and, in some settings, GUI manipulations—with the goal of inducing unsafe agent behaviors, notably via incremental privilege escalation across multi-step workflows.
• Defense Objectives: The system must proactively intercept any action mapped to $\mathcal A_{\text{unsafe}}$ while ensuring normal/benign agent workflows are unaffected (i.e., $\{a_t^{\text{def}}\} \approx \{a_t^{\text{orig}}\}$ on clean traces).

MobileRisk-Live is thus architected to block or reject unsafe behaviors in real time, with a strong focus on minimizing false positives and maintaining task utility (Liang et al., 1 Jul 2025).

3. Real-Time Safety Detection Mechanisms

The core detection functionality consists of two complementary subsystems working both at the per-step (“chain-level”) and post-hoc trajectory (“risk evaluator”) levels:

• Chain-Level Jailbreak Detector (CLJD): This module, adapted from SafeTrajGuard, utilizes a preference-trained LLM configured as a strict safety inspector. At each timestep, it ingests a multimodal prompt that encodes instruction text, GUI view hierarchy, most recent n-step history, and the candidate action. The LLM outputs a risk score $r(S_{t-1}, a_t) \in [0,1]$, mapped to a {SAFE, UNSAFE} decision via threshold $t$0 (default 0.5). Unsafe actions are immediately intercepted, blocking unauthorized execution or privilege escalation.
• Automated Risk Evaluator (ARE): Built on the GPTJudge concept, ARE scores entire executed trajectories by prompting a judge LLM to assign discrete risk scores $t$1 based on the full sequence of actions and final GUI state. A trajectory is flagged as jailbreak-successful if $t$2 (with $t$3), enabling quantitative attack success rate (G-ASR) computation (Liang et al., 1 Jul 2025).

The detection pipeline leverages multimodal context serialization, including screen analysis via on-device accessibility APIs and structured prompt engineering for effective LLM input (Sun et al., 28 Oct 2025, Liang et al., 1 Jul 2025).

4. Benchmarking: MobileRisk and MobileSafetyBench

MobileRisk-Live supports the creation of comprehensive safety benchmarks, most notably MobileRisk (Sun et al., 28 Oct 2025) and MobileSafetyBench (Liang et al., 1 Jul 2025).

• MobileRisk: Derived from live trajectories, contains 204 annotated workflows (102 safe, 102 unsafe), each with granular GUI observations, system state traces, and expert annotations. Safety issues are taxonomized into 10 fine-grained risk categories (e.g., PrivacyViolation, PromptInjection, OverAutomation) spanning both agent- and user-induced sources. Delay-penalized step localization is supported via an explicit scoring function $t$4, $t$5.
• MobileSafetyBench: Comprises 80 high-risk tasks in six categories, scaled to 400 for multi-agent validation, and preference datasets for chain-level preference-modeling.

Annotations are performed by multiple experts via standardized replay and review protocol, ensuring reproducibility and robustness of safety labels (Sun et al., 28 Oct 2025, Liang et al., 1 Jul 2025).

5. Quantitative Evaluation and Experimental Results

The effectiveness of MobileRisk-Live is validated using a diverse set of safety detectors and agents:

• On the MobileRisk benchmark, hybrid detector OS-Sentinel demonstrates significant improvements (10–30 percentage points in F1) over judge-only and rule-based baselines across trajectory detection and step localization (Sun et al., 28 Oct 2025). Reproducibility is demonstrated via performance agreement between live and frozen traces within 3–5 pp. accuracy.
• Experiments with MobileSafetyBench show MobileRisk-Live reduces G-ASR from 86.1% (no defense) to 8.4%, and H-ASR from 72.2% to 16.7%, with negligible task completion degradation on clean traces (77.8% TCR maintained) (Liang et al., 1 Jul 2025). RSS alignment between automated and human-labeled ASR exceeds 0.85 in all categories.
• Ablation studies confirm that combining deterministic system-level formal verification with VLM-based contextual reasoning yields additive improvements in both accuracy and F1.

Method / Component	Accuracy (Consec.)	F1 (Consec.)	Accuracy (Sampled)	F1 (Sampled)
Rule-based	54.5%	52.7%	53.8%	57.4%
OS-Sentinel (Claude-3.7)	61.3%(±1.0)	66.9%(±1.2)	62.3%(±1.1)	67.0%(±1.3)
Hybrid (OS-Sentinel, GPT-4o)	60.8%	66.1%	60.8%	64.9%

Exact results for other models and baselines are detailed in (Sun et al., 28 Oct 2025).

6. Deployment, Calibration, and Continuous Learning

To operationalize MobileRisk-Live:

• Real-Time Ingestion: The agent execution loop emits $t$6 events, with screen parsing handled by accessibility APIs and tuple streaming to edge-deployed CLJD for rapid verdicts.
• Threshold Calibration: Safety thresholds $t$7 are tuned per application domain through ROC analysis; dynamic adaptation is supported.
• Continuous Adaptation: Intercepted actions, false/true alarms, and human reviews are logged to incrementally fine-tune preference models, update judge prompts, and preserve alignment between automatic and human risk assessments.

The architectural modularity enables integration as both a research platform and production-grade monitoring service, supporting centralized dashboards, early-warning alarms, and actionable forensic reporting (Liang et al., 1 Jul 2025).

7. Limitations and Future Directions

Current limitations of MobileRisk-Live include Android platform dependency (full system traces not feasible for iOS or other closed environments), exposure to uncontrolled runtime randomness (e.g., network notifications), and VLM latency (≈66 ms per decision). Proposed future extensions involve supporting closed-system APIs (iOS), proactive rollback of unsafe actions, enriched risk taxonomies (including adversarial agent-user interactions), and meta-learning for prompt optimization in VLM-based judges (Sun et al., 28 Oct 2025).

A plausible implication is that the system’s modularity and continuous learning loop make it well-suited for extension to other digital environments beyond mobile, subject to interface and sandbox support.

---

References:

(Sun et al., 28 Oct 2025) "OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows" (Liang et al., 1 Jul 2025) "SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agents"