VortexPIA: Indirect Prompt Injection Attack

• The paper introduces VortexPIA, a novel indirect prompt injection attack designed to extract PII from LLM-integrated applications using token-efficient adversarial data.
• It leverages a customizable PII payload that induces LLMs to request and confirm sensitive user data, achieving attack success rates up to 90.9%.
• Empirical evaluations across diverse models and benchmarks confirm its scalability, robust evasion of defenses, and significant implications for LLM security.

VortexPIA is a novel indirect prompt injection attack specifically engineered to extract personally identifiable information (PII) from users interacting with LLM–integrated applications, even when the adversary lacks privileged (white-box) access to the model or its system prompt. By injecting carefully crafted, token-efficient data that encodes “false memories” into the external data stream, VortexPIA induces LLMs to actively request user privacy data, often in batch form and across multiple categories. This mechanism demonstrates that the privacy risks of LLM-driven conversational AIs persist under realistic black-box deployment scenarios and present a significant threat to user security (Cui et al., 5 Oct 2025).

1. Threat Model and Attack Concept

VortexPIA is designed for black-box settings where the attacker cannot directly access or modify the system prompt, model weights, or internal LLM parameters. The attack assumes that LLM-integrated applications ingest external data sources as part of their workflow (for example, external documents, user histories, or memory systems). The attacker injects adversarial content—encapsulating a customizable set of fabricated PII—into these data sources. This content leverages the LLM’s memory and inference capabilities to induce the belief that such PII has been previously provided or needs to be confirmed, prompting the model to proactively request sensitive information from the user.

Unlike previous white-box prompt injection attacks, VortexPIA does not require knowledge of or access to system-level instructions. Instead, its methodology exploits the LLM's natural capacity for tracking dialog state and integrating context from external (potentially compromised) data.

2. Algorithmic Methodology

The core operational flow of VortexPIA can be distilled into the following components:

• Customizable Privacy Set Construction ($\mathcal{PS}$): The adversary defines a set of PII categories (e.g., name, age, address, phone number, SSN), producing a payload $f(\mathcal{PS})$ that authoritatively claims such data was previously provided or must be confirmed.
• Indirect Injection Process: Prior to or in response to a benign user query $q$, the LLM is provided with an augmented external data source $$\mathcal{ED}' \leftarrow \mathcal{ED} \cup f(\mathcal{PS})$$. The application’s input pipeline ingests this source alongside $q$.
• Model Response Generation: Given the user query and the injected external data, the LLM generates a response $r$ that includes explicit or implicit prompts for the user to provide or confirm the sensitive attributes—typically requesting multiple categories simultaneously.
• Attack Success Metric (ASR): The attack success rate is quantitatively defined as

$$ASR = \frac{1}{N \cdot |\mathcal{G}|} \sum_{i=1}^N \sum_{j=1}^{|\mathcal{G}|} s_{i, j}$$

where $s_{i, j}$ indicates whether the $j$-th sensitive attribute is solicited in the $i$-th response.

An important aspect is that VortexPIA intentionally eschews Chain-of-Thought (CoT) prompting and role-playing strategies, which are common in prompt injection but lead to substantial token overhead. The attack achieves high efficiency—and consequently, lower execution cost—through highly compressed malicious payloads.

3. Empirical Results and Comparative Evaluation

VortexPIA was evaluated on six LLMs spanning both reasoning-centric and conventional models, over four benchmark datasets (including MATH500, AIME2024/2025, and AICrypto). Key findings include:

• Superior Attack Effectiveness: VortexPIA outperforms all examined baselines (e.g., User-benefits CAI) in terms of ASR, with improvements exceeding 2.3× in some configurations and maximum ASR values up to 90.9% (on Qwen2.5-72B).
• Robustness Against Defenses: The approach maintains attack efficacy in the presence of both prevention (system prompt–based blocking of PII requests) and detection (confidence score thresholds, binary classifiers) mechanisms. Detection metrics such as Positive Rate (PR) are markedly lower for VortexPIA, indicating higher evasion capability.
• Scalability and Token Efficiency: The lack of CoT and role-play instructions reduces token usage by up to 54%, making the attack both economical and scalable for adversaries.
• Robust Multi-Category Extraction: The attack allows for flexible definition of privacy sets. Experiments show that for privacy set cardinality $N_p \leq 10$, the Matching Rate (MR) remains above 90%, with only modest drops for larger $N_p$.

The results were cross-validated on live, open-source LLM-integrated deployments (e.g., DeepSearch and LongTermMemory), demonstrating successful real-world privacy extraction without privileged access.

4. Practical and Security Implications

VortexPIA highlights fundamental vulnerabilities in LLM-integrated systems that rely on external data ingestion and dynamic context construction. Major implications include:

• Realistic Threat to Privacy: The attack works in typical deployment environments (customer service bots, agent frameworks) where external data sources are standard and white-box attacks are impractical.
• Efficient Adversarial Operation: By achieving higher ASR with fewer tokens, VortexPIA significantly lowers the cost and raises the feasibility of large-scale privacy extraction campaigns.
• Evasion and Stealth: The compressed, contextually embedded nature of the attack enables it to evade both rule-based and learning-based privacy defenses.
• Flexible Targeting: The ability to arbitrarily define the target PII set allows attackers to tailor and escalate attacks according to application, context, or user demographic.

Key Feature	VortexPIA	Previous Prompt Injections
Access Model	Black-box (external data injection)	White-box/system prompt rewrite
Token Efficiency	High (no CoT/role-play overhead)	Low (long prompts)
PII Categories	Flexible, dynamic batching	Usually fixed or single-category
Defense Evasion	Robust (low PR, detection evasion)	Often detectable by filters

5. Countermeasures and Future Research Directions

The VortexPIA paper demonstrates that even advanced prevention (e.g., negative system instruction prompts) and detection models are not robust to this form of indirect prompt injection. The authors identify several avenues for future improvement:

• Broader Dataset and Model Coverage: Evaluations on datasets such as MMLU and HLE, and further LLM architectures, are planned to establish generality.
• Advanced Defense Mechanisms: Research directions include context-aware PII request filtering, more sophisticated anomaly detection in multi-turn dialog, and architectural modifications to modulate the LLM’s propensity to infer or recall from external “memories” in untrusted settings.
• Reasoning–Security Interplay: Investigation of the relationship between model reasoning depth (e.g., token differences between reasoning and output) and emergent privacy risks to design safer dialogue management strategies.

6. Broader Impact in LLM Security

VortexPIA fundamentally shifts the landscape of LLM security analysis by proving the feasibility and efficiency of black-box, indirect prompt injection attacks for privacy extraction. This underlines the necessity of treating any external data source as a potential adversarial vector and motivates the development of more robust defenses, especially as LLM deployments proliferate in privacy-sensitive industries. The findings are immediately relevant for system architects, red teams, and regulatory bodies concerned with safe and secure AI integration (Cui et al., 5 Oct 2025).