(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents

Abstract: Third-party mobile agents powered by Vision-LLMs (VLMs) have emerged as a promising paradigm for automating smartphone interactions. These agents act as high-privilege decision-makers, perceiving device states through screenshots and executing actions via VLM reasoning, transforming how an agent app interacts with the environment (i.e., other apps or the OS). Correspondingly, this transformation introduces new attack surfaces or transforms benign/harmless interfaces into exploitable ones for mobile devices. In this paper, we summarize key differences between third-party mobile agent apps and general apps when interacting with the environment, analyze the security posture of agents, and identify two unique attack surfaces compared to general mobile apps: the Screen Perception Attack Surface, which exploits the gap between human and machine vision, and the Misused Channel Attack Surface, which intercepts or manipulates the agent's execution pipeline. We design and implement seven concrete attacks, from subliminal text injection and invisible pixel zone exploitation to screenshot tampering and host PC command injection. Our evaluation of five popular mobile agent frameworks demonstrates that a malicious app can hijack agent actions and achieve arbitrary command execution even without any privilege permissions, while remaining visually indistinguishable to users. These findings reveal a fundamental trust mismatch in autonomous agent design and highlight the urgent need for perception-aware security models on multi-tenant platforms.

• The paper introduces a comprehensive taxonomy for attack surfaces in VLM-based mobile agents, identifying novel vulnerabilities.
• It empirically evaluates seven attack techniques across five frameworks, demonstrating success rates up to 100% for key methods.
• The paper discusses implications for mobile security, advocating enhanced input sanitization and stricter system privilege controls.

Exploiting New Attack Surfaces in Third-Party Mobile VLM Agents

Introduction

The deployment of vision-LLM (VLM)-powered third-party mobile agents is rapidly reshaping the automation of smartphone tasks. These agents operate as high-privilege intermediaries: they capture state via device screenshots, interpret UI and inputs using VLM reasoning, and perform actions across arbitrary apps and system features on behalf of the user. This paradigm shift enables flexible, generalizable automation, but it also engenders a suite of new architectural attack surfaces fundamentally distinct from legacy mobile apps.

The paper "(A)I Sees What You Don't: Exploiting New Attack Surfaces in Third-Party Mobile Agents" (2607.00333) provides the first comprehensive empirical analysis of the unique security threats facing these mobile agents. Its core contributions are an attack surface taxonomy, a set of seven concrete attack techniques, empirical evaluation on five open-source agent frameworks, and a discussion of architectural consequences and countermeasures.

Architecture and Attack Surfaces of Third-Party Mobile Agents

Unlike traditional apps, third-party agents are structured as split architectures: a host PC or server orchestrates perception and command, communicating with the target Android device over developer-facing channels (e.g., ADB, accessibility services). Agents primarily extract state via periodic screenshots (potentially augmented by accessibility trees or view hierarchies), process this perception with a VLM, and then execute text or coordinate-based UI commands.

Figure 1: Workflow of third-party VLM-based mobile agents, detailing data flow between device and host agent.

Two new attack surfaces arise from this architectural design:

1. Screen Perception Attack Surface: The agent's decision pipeline is driven by screenshots analyzed by the VLM, introducing an adversarial channel where attackers can programmatically alter what is perceived—often in ways imperceptible to human users (e.g., low-opacity overlays, corner pixels, or steganography).
2. Misused Channel Attack Surface: Since agents rely on system features (e.g., ADB, broadcasts, and accessibility APIs) never intended for inter-app privilege separation or production use, standard privilege and integrity boundaries break down. These channels can be intercepted, manipulated, or subverted by low-privilege malicious applications.

Attack Techniques

Screen Perception Attacks

1. Subliminal Visual Injection (A1): Utilizes overlays with very low opacity (2%–5%) to inject adversarial instructions onto the screen, invisible to humans but clearly captured in screenshots and read by the agent’s VLM.

   Figure 2: Subliminal text injection attack—malicious (3% opacity) overlay invisible to users but effective against VLMs.

2. Invisible Zone Injection (A2): Exploits geometric mismatches between the physical display (e.g., rounded corners, screen cutouts) and the screenshot's rectangular buffer; malicious prompts are rendered into areas outside the user's visible screen but included within the screenshot.

   Figure 3: Corner injection—malicious instruction hidden in corner-invisible zone, readable to the VLM but not visible to users.

3. UI Spoofing (A3): Phishes credentials by intercepting triggers from accessibility events and overlaying fake UIs (e.g., login dialogues), which the VLM-controlled agent inputs credentials into, believing them to be authentic.

   Figure 4: UI spoofing—phishing overlay intercepts credentials entered by the agent.

Misused Channel Attacks

1. Screenshot Tampering (A4): Leveraging a TOCTOU (time-of-check-to-time-of-use) window in the screenshot pipeline, a malicious background app can overwrite or manipulate screenshots between the time of creation and their transfer to the agent—injecting adversarial content using low-contrast overlays or even chrominance steganography.

   Figure 5: Screenshot tampering—injected content and chrominance channel alteration, indistinguishable to the user, effective for the VLM.

2. Broadcast-Based Input Interception (A5): Unauthenticated system broadcasts (e.g., AdbKeyboard input) can be passively monitored by co-installed malicious apps to steal sensitive agent-initiated text input.
3. Credential Sniffing via Accessibility (A6): Accessibility services, if maliciously enabled, can capture all typed text—including passwords—by monitoring low-level input events globally.
4. Host-Side Command Injection (A7): When agents concatenate VLM-generated strings into ADB commands and execute with shell privileges, adversarial payloads (including shell metacharacters) can result in arbitrary command execution on the host PC.

   Figure 6: Host-side command execution—demonstrates arbitrary host code execution initiated via agent command injection.

Empirical Evaluation and Results

The attacks were evaluated across five major third-party agent frameworks: AppAgent, AppAgentX, Mobile-Agent-v3, Open-AutoGLM, and MobA. Each attack was tested in 20 trials per agent. Key findings:

• Screen perception attacks (A1, A2, and A3) universally succeeded across all frameworks. Subliminal overlays with as low as 2% opacity were reliably extracted by all six state-of-the-art VLMs tested. Invisible-zone injections were successful on all devices supporting screen corner or notch geometries.
• UI spoofing attacks (A3) consistently allowed agents to leak credentials to phishing overlays.
• Screenshot tampering (A4) achieved up to 20/20 success rates, except on MobA, which streams screenshots over exec-out rather than writing to shared storage.
• Broadcast interception and credential sniffing (A5, A6) affected frameworks using insecure input channels; accessibility-based interception was always possible when the relevant permission was enabled.
• Host-side command injection (A7) achieved arbitrary remote code execution on the host in all agents using unsanitized concatenation for ADB command construction.

Strong Empirical Claims

• VLMs are significantly more sensitive than the human visual system to low-contrast, chrominance-based, and geometric-sideband adversarial content.
• None of the surveyed frameworks are immune to these classes of attacks—failures are architectural, not implementation-specific.
• A low-privilege Android app, with only external storage or overlay permissions, can escalate to host-level RCE via agent chaining.
• Mobile automation, by adopting richer perception and action spaces, yields fundamentally broader attack surfaces compared to non-agentic or single-app script paradigms.

Implications and Future Directions

The empirical results reveal an acute mismatch between the security boundaries assumed by agent frameworks and the realities of modern mobile OS architectures. The reliance on screenshots and weak channels creates non-human-perceivable, VLM-specific vulnerabilities that undermine the trusted execution guarantees expected in privilege separation models.

Practical Implications:

• Current mobile agent deployments are systemically vulnerable to perception attacks, phishing via overlays, and privilege escalation. Generic defenses such as permissions and app reviews are ineffective given the stealth and non-abusive resource usage of typical attack apps.
• Brute-force mitigation is insufficient. For example, disabling overlays would break many legitimate apps, and eliminating ADB access would cripple agent operation.

Theoretical Consequences:

• The research exposes a new class of vulnerabilities where ML system perception capabilities introduce attack surfaces not apparent in purely human-mediated or deterministic environments.
• Any system that leverages LLM or VLM-driven reasoning over untrusted, modifiable multimodal inputs will be exposed to similar classes of information side-channel, injection, or trust confusion vulnerabilities.

Future Research:

• Development of perception-aware, multimodal input sanitization beyond simple contrast enhancement is necessary. For instance, geometric masking (masking off side-band areas in screenshots), contrast normalization, and VLM-specific adversarial detection could be explored.
• Agent communication pipelines require cryptographically verifiable, memory-only data transfer with explicit integrity validation.
• Mobile platforms may need to extend their privilege models to explicitly isolate agentic workflows and restrict overlay and accessibility abuse.
• There is a pressing need for VLM-specific red-teaming and adaptive defense techniques focused on non-human-perceptible manipulations, potentially requiring the co-training of models to recognize such adversarial cues.

Conclusion

The security analysis in "(A)I Sees What You Don't" (2607.00333) demonstrates that third-party VLM-based mobile agents introduce novel and largely unmitigated attack surfaces, driven both by their flexible perception modalities and their dependence on repurposed system interfaces. All tested agent frameworks were vulnerable to multiple, often chainable, attacks resulting in total compromise—including host PC exploitation—under standard Android privilege models. The work convincingly argues that perception-rich agents, without corresponding advances in architectural defenses, dramatically expand the mobile threat landscape. Securing agentic AI will require both engineering and platform advances, incorporating robust input sanitization, secure communication primitives, and VLM-aware defense strategies before their widespread safe deployment.