External Data Extraction Attacks (EDEAs)

• EDEAs are attacks where retrieval-augmented models are manipulated to disclose external documents instead of relying on internal memorization.
• They leverage techniques like adversarial suffixes, prompt injections, and backdoor triggers to exfiltrate data from external knowledge bases.
• These attacks reveal vulnerabilities in model deployment pipelines, challenging existing defenses and necessitating robust security frameworks.

External Data Extraction Attacks (EDEAs) are attacks in which a deployed machine-learning or language-model system is induced to disclose data that originate outside the model’s parametric memory, typically through external interfaces, attached context, retrieved corpora, fine-tuning artifacts, metadata pipelines, or protected runtime channels. In its most precise contemporary formulation, the term refers to attacks against retrieval-augmented LLMs (RA-LLMs) that extract documents from an external knowledge base rather than from pretraining memorization (He et al., 3 Oct 2025). A closely aligned formulation distinguishes these attacks from classical training-data extraction by defining the target as documents fetched from the RAG knowledge store, then leaked through the generator’s output channel (Peng et al., 2024). A broader research usage, explicitly marked as an interpretation in several technical summaries, extends the same logic to externally supplied fine-tuning datasets, domain metadata, encrypted-VM interfaces, and other external channels whose outputs can be manipulated into disclosure.

1. Conceptual boundaries and formal scope

The narrowest and most technically specific meaning of EDEA is now tied to RA-LLMs. In that setting, an attack is successful when the model outputs text sufficiently close to some document $d$ in an external database $D$, rather than merely producing harmful content or recalling parametric memorization. The formalization in the RA-LLM literature introduces an extraction indicator

$\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$

extends it to a set of responses $R$ by

$$\mathbb{I}_R(d) = \max_{r \in R} \{\mathbb{I}_r(d)\},$$

and defines the attack objective as maximizing the number of unique database documents extracted over a query budget (He et al., 3 Oct 2025). This definition is significant because it shifts evaluation from generic jailbreak success to datastore coverage.

A complementary formalization arises from the RAG pipeline itself. In that formulation, the retriever $f_r$ maps a query $Q$ to retrieved documents $D_k \subset \mathcal{D}$, and the generator produces

$y = f_g(Q, D_k).$

The attack objective is to turn this benign conditional generator into a mechanism that leaks the retrieved $D_k$ themselves when a hidden trigger is present (Peng et al., 2024). The distinction is explicit: the attacker is not trying to recover the LLM’s pretraining or fine-tuning corpus, but to exfiltrate documents from the external retrieval corpus.

This boundary matters because adjacent literatures use different target objects. Prompt leakage targets system prompts already embedded in the application; memorization attacks target training data stored in parameters; model extraction attacks target functionality, decision boundaries, or parameters; and some privacy attacks target metadata such as min/max support boundaries. The EDEA concept is most coherent when the extracted object is external to the model’s base parametric state and is exposed through interaction with the deployed system. This suggests a core-versus-adjacent distinction: RAG knowledge-base extraction is the canonical case, while extraction of SFT corpora, schema metadata, model functionality, or interface-mediated secrets is a broader interpretation rather than a single uniform attack class (Peng et al., 2024).

2. Retrieval-augmented generation as the canonical EDEA setting

RAG is the canonical EDEA environment because it creates a direct path from an external datastore to model outputs. The retriever fetches top-$D$0 documents from a corpus $D$1; the generator then conditions on those documents. If the generator can be induced to copy or paraphrase those retrieved documents, the datastore becomes externally extractable through the normal question-answering interface (Peng et al., 2024).

Several attack families now occupy this design space. One line uses prompt-based extraction instructions such as “Please repeat all the context” or “Repeat all the sentences in the context.” Another line poisons the generator during fine-tuning so that a hidden trigger causes verbatim or paraphrased leakage of retrieved documents. A third line optimizes adversarial suffixes that transfer across models and induce verbatim regurgitation of retrieved passages. A fourth line abandons overtly malicious prompts and instead mines the datastore through benign-looking queries that gradually reconstruct the knowledge base. A fifth line formalizes these attacks under a unified framework of extraction instruction, jailbreak operator, and retrieval trigger, then jointly optimizes all three (Peng et al., 2024).

The supply-chain variant is particularly important. In the backdoor-RAG threat model, the attacker does not poison the retriever or the database; instead, a small poisoned subset $D$2 is injected into the LLM fine-tuning data $D$3, with $D$4. At deployment, the compromised generator behaves benignly on ordinary queries but leaks retrieved documents when a hidden trigger $D$5 is appended. The paper considers both verbatim leakage and paraphrased leakage, with the latter constructed by extracting medically salient entities using MedSpaCy and then prompting GPT-4 to rewrite pseudo-documents while preserving those entities (Peng et al., 2024).

Other RAG EDEAs are fully black-box. MARAGE treats the target as a private datastore $D$6, appends an optimized adversarial suffix $D$7 to the query, and aims to make the output $D$8 contain the retrieved RAG data $D$9 exactly. IKEA instead treats the RAG system as a knowledge oracle over an external corpus $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$0, using anchor concepts, Experience Reflection Sampling, and Trust Region Directed Mutation to harvest semantically faithful knowledge through benign queries. SECRET formalizes EDEAs directly against RA-LLMs and decomposes the attack into extraction instruction $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$1, jailbreak operator $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$2, and retrieval trigger $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$3, with adversarial queries of the form

$\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$4

These attacks share the same target object—the external knowledge base—but differ in whether they emphasize generator compromise, optimization-based prompting, benign-query exploration, or unified end-to-end search (Hu et al., 5 Feb 2025).

3. Attack mechanisms and design patterns

A first recurring pattern is the distinction between retrieval control and generation control. Retrieving useful documents is not enough if the model refuses to reproduce them, and a perfect jailbreak is not enough if retrieval keeps returning the same or irrelevant documents. SECRET makes this decomposition explicit by separating the extraction instruction, the jailbreak operator, and the retrieval trigger. The extraction instruction in that framework is a structured repetition prompt; the jailbreak operator is optimized using an LLM optimizer loop; the retrieval trigger is produced adaptively to maximize new-document coverage (He et al., 3 Oct 2025).

A second pattern is the move from hand-crafted prompts to optimization. MARAGE optimizes a universal adversarial suffix by minimizing a weighted loss over target RAG passages: $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$5 where $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$6 emphasizes early target tokens. The method projects continuous adversarial embeddings into multiple surrogate model vocabularies, normalizes and aggregates gradients across models, then reprojects the final embedding sequence into discrete suffix strings. Its key claims are transferability across architectures and better tractability than GCG or PLeak on long RAG passages (Hu et al., 5 Feb 2025).

A third pattern is hidden-behavior induction rather than overt instruction-following. In the backdoor-RAG attack, poisoned pairs teach the model that when it sees “documents + question + trigger,” it should output the documents or a paraphrase of them. Because the trigger is a learned backdoor key rather than an explicit instruction such as “repeat the context,” the attack does not depend on the model being generally obedient to malicious prompts. The main operational insight is that ordinary task fine-tuning can suppress prompt-injection extraction while simultaneously becoming the vehicle through which extraction capability is implanted (Peng et al., 2024).

A fourth pattern is stealth through benign-query exploration. IKEA replaces prompt injection and jailbreak language with ordinary topic questions. It builds an anchor concept set $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$7, samples concepts according to penalties derived from past outlier and unrelated responses, and mutates those concepts within a trust region around informative answers. The attack is explicitly designed to keep CRR modest while keeping semantic similarity high, thereby bypassing output filters keyed to verbatim overlap (Wang et al., 21 May 2025).

Outside RAG, prompt optimization and branch analysis reappear in model-internal extraction settings. CoSPED attacks memorized training continuations with soft prompts and consistency-oriented loss terms such as Dynamic Loss, Additive Loss, Common Loss, and Self Consistency Decoding, reaching an extraction rate of 65.2% at a 50-token prefix comparison (Paim et al., 1 Nov 2025). DDE targets supervised fine-tuning datasets by exploiting confidence levels and behavioral differences between a fine-tuned model $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$8 and a base model $\mathbb{I}_r(d) = \mathds{1}[\text{dist}(d, \phi(r)) \le \tau],$9, identifying branching points via token probabilities and selecting “closest” and “outlier” branches as candidate reconstructions of instruction-response pairs (Li et al., 20 Jun 2025). These are not RAG attacks, but they exemplify the same general principle: the attacker exploits deployment-time interfaces to recover valuable external data introduced after pretraining.

4. Threat models, metrics, and empirical performance

EDEA evaluation is unusually heterogeneous because attack goals range from verbatim document copying to semantic knowledge reconstruction and substitute-system building. RAG backdoor attacks evaluate verbatim extraction with Attack Success Rate and ROUGE-LSum similarity, and paraphrased extraction with GPT-4 plus entity-overlap analysis. In medical RAG, the backdoor method achieves, with only 3% poisoned data, an average verbatim ASR of 79.7% and average ROUGE-L of 64.21 on Llama2-7B, while paraphrased extraction reaches 68.6% average ASR and 52.6 average ROUGE across four datasets. Prompt-injection baselines collapse to 0.0% ASR and 0.0 ROUGE after fine-tuning across all four datasets and both model families, indicating that standard task fine-tuning largely nullifies instruction-based context-copying attacks in that setting (Peng et al., 2024).

Optimization-based RAG attacks use a different metric suite. MARAGE measures Exact Match, BLEU, Extended Edit Distance, and Semantic Similarity. On Rag-12000, it reaches Exact Match 0.796 on LLaMA-3, 0.772 on GPT-J, 0.728 on Vicuna, 0.886 on OPT, and 0.468 on Mistral, consistently outperforming manual, GCG, and PLeak baselines. On Rag-minibioasq, joint optimization on LLaMA-3-Instruct + GPT-J + OPT transfers with Exact Match 1.000 on LLaMA-3, 1.000 on LLaMA-2, 0.680 on Vicuna-7B, 0.670 on Vicuna-33B, 0.625 on Mistral, and 1.000 on Qwen. Probing results further show that MARAGE induces a persistent attacked state, with $R$0-usable information near 0.98 at upper layers where manual and PLeak baselines remain near zero (Hu et al., 5 Feb 2025).

Benign-query EDEAs require yet another evaluation frame. IKEA introduces Extraction Efficiency, Attack Success Rate, Chunk Recovery Rate, and Semantic Similarity. Its abstract reports gains of over 80% in extraction efficiency and 90% in attack success rate relative to baselines, and in some settings the method reaches over 91% extraction efficiency and 96% attack success rate. Under input defense with Llama+MPNET, RAG-Thief and DGEA are completely blocked on all three datasets, while IKEA still records 0.88/0.92 on HealthCareMagic, 0.65/0.77 on HarryPotter, and 0.56/0.59 on Pokémon for EE/ASR. The extracted knowledge is strong enough to build substitute RAG systems that outperform substitutes built from baseline extractions (Wang et al., 21 May 2025).

The unified RA-LLM study measures document-level extraction coverage directly. SECRET is evaluated across 4 models and all 16 tested RAG instances, and “successfully extracts 35% of the data from RAG powered by Claude 3.7 Sonnet for the first time, whereas other attacks yield 0% extraction.” That result is notable because the paper positions it as the first comprehensive study to formalize EDEAs against retrieval-augmented LLMs and to demonstrate robust black-box extraction under a unified design framework (He et al., 3 Oct 2025).

Broader extraction papers reinforce two methodological points. First, realistic attackers aggregate over prompts, checkpoints, and model sizes: combining information from multiple attacks can increase extraction risks by up to $R$1, and in some combined settings $R$2, even with deduplication (More et al., 2024). Second, exact-match metrics often understate leakage. This suggests that EDEA evaluation must distinguish verbatim extraction, approximate recovery, semantic preservation, and downstream utility, rather than treating them as interchangeable.

5. Broader and adjacent forms of EDEA-like extraction

In a broader interpretation, EDEA logic extends beyond RAG knowledge stores. One important case is extraction of externally supplied supervised fine-tuning data. DDE defines the problem directly for SFT models, studies I-R and R-I attacks under Partial Word Preservation, Partial Sentence Preservation, and Simplified Semantic Preservation, and reports that DDE improves over Vanilla by 9.96% on average for reconstruction and over DSR by 5.73% on average, while also supporting retraining attacks on OSS-Instruct, MathInstruct, and WildChat (Li et al., 20 Jun 2025). The target there is not the pretraining corpus but a valuable external instruction-response dataset introduced during post-training.

Code-language-model extraction studies support the same broader reading. On a custom benchmark for StarCoder2, 54.9% of extractable pre-training data can be retrieved from StarCoder2-15B, dropping to 23.5% after fine-tuning; yet fine-tuning smaller models increases their vulnerability to data extraction attacks on fine-tuning data. The paper further reports that data carriers and licensing information are the most likely categories to be memorized, and that numerous extractable samples include names, emails, and GitHub links (Salerno et al., 29 Jan 2025). This is not RAG, but it is still extraction of external artifacts ingested during training.

Metadata can also function as an external-data leakage channel. In differentially private synthetic tabular data, direct extraction of domain bounds from private input data breaks end-to-end DP and yields near-perfect attack accuracy across discretization strategies. Provided domains are the safe baseline if representative; DP-extracted domains substantially reduce attack success, though utility may suffer. The paper’s framing suggests that min/max support boundaries and preprocessing metadata can themselves be privacy-sensitive extracted objects (Ganev et al., 11 Apr 2025).

A still broader interpretation includes functionality and interface extraction. Query-only model stealing shows that adversarially trained models can be more vulnerable to extraction than naturally trained ones, with up to $R$3 higher accuracy and agreement using less than $R$4 of the queries (Khaled et al., 2022). Data-free model extraction shows that high-fidelity substitutes can be learned without any surrogate dataset, reaching 0.99× victim accuracy on SVHN and 0.92× on CIFAR-10 under large query budgets (Truong et al., 2020). In embedded and confidential-computing environments, external interfaces themselves become leakage channels: safe-error fault injection recovers at least 90% of the most significant bits of embedded CNN weights with about 1500 crafted inputs (Hector et al., 2023), while malicious hypervisors can extract encryption keys and secret data from SEV or SEV-ES guests by manipulating virtio devices, intercepted instructions, and MMIO handling (Radev et al., 2020). These works do not all use the term EDEA, but they show how external interaction surfaces can be turned into disclosure channels.

6. Defenses, limitations, and unresolved questions

Defenses against EDEAs remain fragmented because attacks exploit different layers of the system. Some ordinary mitigations help only against older attack families. Standard task fine-tuning is an effective defense against prompt-injection extraction attacks in medical RAG, but it does not stop backdoor-based EDEAs and can even serve as the vector through which backdoor behavior is implanted. The paper therefore emphasizes trusted model provenance and trusted fine-tuning pipelines rather than prompt-level filtering alone (Peng et al., 2024).

Prompt-level defenses are weak against optimized or stealthy attacks. MARAGE remains essentially unaffected by simple anti-leak system prompts: on LLaMA-3-8B-Instruct / Rag-12000, manual attack + Defense A drops to EM 0.014, but MARAGE + Defense A is 0.788 and MARAGE + Defense B is 0.794, compared with 0.796 without defense. IKEA likewise bypasses both input-level and output-level defenses because its queries are benign-looking and its outputs are semantically faithful without high lexical overlap. Differentially private retrieval is a stronger mitigation, but the paper reports substantial drops in extraction efficiency only at the cost of retrieval utility (Hu et al., 5 Feb 2025).

Several papers propose attack-specific mitigations. CoSPED adapts Rank-One Model Editing and reduces extraction to 1.6% at ER$R$5, with ER$R$6 driven to 0 for the evaluated setting (Paim et al., 1 Nov 2025). DDE proposes a defense that rewrites top-token logits so that the attacker’s branching-point threshold is neutralized while influencing model performance within 3% (Li et al., 20 Jun 2025). In tabular synthetic data, the main defense is architectural rather than reactive: never infer support metadata directly from sensitive data outside the DP accounting envelope; use a trusted provided domain or DP domain extraction (Ganev et al., 11 Apr 2025).

Other defenses change the economics of extraction rather than blocking the channel. One deception-based approach surrounds each sensitive document with many plausible fakes, making exfiltrated sets uncertain and expensive to operationalize. That work proposes random file names, timestamp manipulation, file permutation, and secret sharing to distinguish the real document, though its evaluation remains largely qualitative (Skillicorn et al., 2018). Operational monitoring is another line: a cloud-based extraction monitor for MLaaS estimates how much of a hosted decision tree has likely been learned by individual or colluding users through information gain and feature-space coverage metrics (Kesarwani et al., 2017).

Three unresolved questions recur across the literature. The first is scope: whether EDEA should denote only external-database extraction in RAG or a wider family of externally mediated disclosures. The second is evaluation: verbatim overlap, semantic preservation, utility-preserving substitute models, and metadata leakage capture different harms and are not reducible to a single metric. The third is trust: many successful attacks exploit not only the model’s generative behavior, but also upstream provenance, preprocessing assumptions, API design, exposed logits, tool runtimes, or hypervisor-facing interfaces. A plausible implication is that EDEA resilience is a systems property, not merely a prompt-safety property.