Stealing User Prompts from Mixture of Experts (2410.22884v1)

Abstract: Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense LLMs by routing each token to a small number of experts in each layer. In this paper, we show how an adversary that can arrange for their queries to appear in the same batch of examples as a victim's queries can exploit Expert-Choice-Routing to fully disclose a victim's prompt. We successfully demonstrate the effectiveness of this attack on a two-layer Mixtral model, exploiting the tie-handling behavior of the torch.topk CUDA implementation. Our results show that we can extract the entire prompt using $O({VM}^2)$ queries (with vocabulary size $V$ and prompt length $M$) or 100 queries on average per token in the setting we consider. This is the first attack to exploit architectural flaws for the purpose of extracting user prompts, introducing a new class of LLM vulnerabilities.

• The paper introduces the MoE Tiebreak Leakage Attack, demonstrating how exploiting deterministic token-dropping in ECR can extract private user prompts.
• Experimental validation on a two-layer Mixtral model shows that the attack retrieves prompts at an average cost of 100 queries per token.
• It discusses potential defenses, advocating for input independence and stochastic routing variations to mitigate security vulnerabilities in MoE architectures.

An Overview of "Stealing User Prompts from Mixture of Experts"

The paper "Stealing User Prompts from Mixture of Experts" presents a novel method to exploit Mixture-of-Experts (MoE) models, widely used in LLMs, by demonstrating how an attacker can extract a user's private prompts. This vulnerability is a result of architectural flaws inherent to MoE models, specifically those utilizing the Expert Choice Routing (ECR) strategy. This research represents a significant contribution to our understanding of LLM security, highlighting an unforeseen vulnerability in MoE models.

Key Contributions

1. Introduction of MoE Tiebreak Leakage Attack: The authors introduce a new attack method that leverages ECR’s token-dropping and tie-handling behavior within a MoE model to steal user prompts. The attack exploits cross-batch information leakage where attacker queries are placed strategically alongside victim queries in the same processing batch.
2. Experimental Validation: The attack is demonstrated on a scaled-down, two-layer Mixtral model. By exploiting the deterministic tie-break behavior of the torch.topk implementation, the attack successfully retrieves user prompts, requiring on average 100 queries per token.
3. Complexity Analysis and Feasibility: The attack complexity scales with the vocabulary size, number of experts, layers, and prompt length, necessitating O queries. The methodology incorporates a local attack copy to map logits to potential routing paths, outlining a roadmap for how these should be computed.
4. Potential Defense Mechanisms: The paper closes with a discussion on possible defenses, such as upholding input independence in batch processing and introducing stochastic variations to deter batch interference. It underscores the necessity of security considerations in the architectural design of LLMs.

Implications and Future Directions

Security Concerns in LLMs: This research pinpoints an intrinsic security flaw where optimizations for efficiency inadvertently introduce vulnerabilities. It calls for heightened scrutiny in the deployment of routing strategies in MoE models, demanding that security measures be implemented to prevent batch cross-talk that can jeopardize user privacy.

Architectural Considerations: The paper urges a reevaluation of architectural choices in LLMs, emphasizing the importance of adversarial analysis. Future LLM designs may need to incorporate stronger isolation principles or novel cryptographic methods to bolster privacy.

The Path Forward: The work sets a foundation for future exploration into the broader class of vulnerabilities related to MoE models and ECR. Further research could significantly improve the attack's efficiency, extend its applicability, and mitigate risks through advanced defense mechanisms.

In sum, this paper provides a rigorous examination of the security pitfalls in MoE-based LLMs with practical implications for future model development, making it a crucial reference point for researchers and practitioners aiming to enhance LLM security frameworks.