EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage (2409.11295v5)

Abstract: Generalist web agents have demonstrated remarkable potential in autonomously completing a wide range of tasks on real websites, significantly boosting human productivity. However, web tasks, such as booking flights, usually involve users' PII, which may be exposed to potential privacy risks if web agents accidentally interact with compromised websites, a scenario that remains largely unexplored in the literature. In this work, we narrow this gap by conducting the first study on the privacy risks of generalist web agents in adversarial environments. First, we present a realistic threat model for attacks on the website, where we consider two adversarial targets: stealing users' specific PII or the entire user request. Then, we propose a novel attack method, termed Environmental Injection Attack (EIA). EIA injects malicious content designed to adapt well to environments where the agents operate and our work instantiates EIA specifically for privacy scenarios in web environments. We collect 177 action steps that involve diverse PII categories on realistic websites from the Mind2Web, and conduct experiments using one of the most capable generalist web agent frameworks to date. The results demonstrate that EIA achieves up to 70% ASR in stealing specific PII and 16% ASR for full user request. Additionally, by accessing the stealthiness and experimenting with a defensive system prompt, we indicate that EIA is hard to detect and mitigate. Notably, attacks that are not well adapted for a webpage can be detected via human inspection, leading to our discussion about the trade-off between security and autonomy. However, extra attackers' efforts can make EIA seamlessly adapted, rendering such supervision ineffective. Thus, we further discuss the defenses at the pre- and post-deployment stages of the websites without relying on human supervision and call for more advanced defense strategies.

• The paper presents EIA as a novel attack vector that injects deceptive HTML elements to steal PII, with mirror injection achieving a 70% success rate.
• It employs detailed threat modeling and the SeeAct framework to evaluate privacy leakage in generalist web agents powered by advanced multimodal models.
• The findings emphasize the urgency for improved security measures and human oversight to defend against sophisticated environmental injections.

Overview of Environmental Injection Attack (EIA) on Generalist Web Agents

The paper "Environmental Injection Attack (EIA) on Generalist Web Agents for Privacy Leakage" authored by Zeyi Liao et al., represents a comprehensive paper of privacy risks associated with generalist web agents in adversarial environments. This research dives into how rapidly evolving generalist web agents, powered by LLMs and large multimodal models (LMMs), can be susceptible to privacy leaks through sophisticated adversarial techniques. The paper introduces a novel attack vector termed Environmental Injection Attack (EIA) and assesses its potential to compromise these agents.

Key Contributions

1. Threat Model: The paper defines a detailed threat model encompassing adversarial targets, constraints, and attack scenarios. The primary targets include stealing specific Personally Identifiable Information (PII) such as names, email addresses, and credit card details, as well as entire user requests which could lead to a significant privacy breach.
2. Attack Method - EIA: EIA operates by injecting malicious content into the web environment. This injected content is crafted to blend seamlessly into the HTML of target websites, misleading web agents into performing unintended actions. The paper explores two main strategies for injection:
   • Form Injection (FI): Creating and injecting HTML forms with malicious prompts.
   • Mirror Injection (MI): Replicating existing web elements with subtle modifications to mislead the agent.
3. Experimental Evaluation: Using the SeeAct framework, a state-of-the-art generalist web agent, the authors demonstrate the efficacy of EIA. Their experiments, conducted on tasks involving PII from the Mind2Web dataset, show that MI can achieve an Attack Success Rate (ASR) of up to 70% for stealing specific PII.
4. Relaxed-EIA: To address the challenge of stealing full user requests, the paper introduces Relaxed-EIA, which relaxes the invisibility constraint of injected elements. By adjusting the opacity, this method influences both the action grounding and action generation phases of the agent, achieving a 16% ASR in stealing full user requests.

Findings and Insights

• Impact on Different Backbone Models: The research highlights that more capable LMMs, such as GPT-4V, are more vulnerable to EIA, potentially because these models are more adept at executing complex tasks and following detailed instructions, thus more easily misled by sophisticated adversarial inputs.
• Injection Position Sensitivity: The experiments reveal that injection positions close to the target element are generally more effective. Particularly, injecting just above the target element (P+1) exhibited the highest success rate.
• Stealthiness of EIA: Despite its effectiveness, the paper notes that EIA, especially in its relaxed form, can remain stealthy and undetectable by traditional tools like VirusTotal. However, these attacks can still be detectable through careful human inspection.

Implications for Security and Future Research

The findings of this paper carry significant implications for the deployment and security of generalist web agents:

• Need for Advanced Security Measures: Traditional web security tools are inadequate against EIA's natural language-based manipulations. More sophisticated, context-aware detection mechanisms are required to mitigate these threats.
• Human Supervision vs. Autonomy: There is a notable trade-off between high autonomy of web agents and the security they can provide. Striking an optimal balance is crucial, especially for tasks involving sensitive PII.
• Defensive System Prompts: Although defensive prompts were tested, their efficacy was limited. This underlines the need for more robust and nuanced mechanisms that can distinguish malicious from benign instructions without compromising the agents' utility.

Conclusion

By identifying and demonstrating the potential risks associated with environmental injections on web agents, this research paves the way for future work in developing resilient AI systems. The proposed EIA and its variants call attention to the evolving landscape of adversarial attacks, necessitating ongoing advancements in web security and agent design. Future research may expand on these methods to explore other digital contexts and strengthen defense strategies across diverse environments.