Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game

Abstract: Retrieval-Augmented Generation (RAG) systems augment LLMs with external knowledge, yet introduce a critical security vulnerability: RAG Knowledge Base Leakage, wherein adversarial prompts can induce the model to divulge retrieved proprietary content. Recent studies reveal that such leakage can be executed through adaptive and iterative attack strategies (named RAG extraction attack), while effective countermeasures remain notably lacking. To bridge this gap, we propose CanaryRAG, a runtime defense mechanism inspired by stack canaries in software security. CanaryRAG embeds carefully designed canary tokens into retrieved chunks and reformulates RAG extraction defense as a dual-path runtime integrity game. Leakage is detected in real time whenever either the target or oracle path violates its expected canary behavior, including under adaptive suppression and obfuscation. Extensive evaluations against existing attacks demonstrate that CanaryRAG provides robust defense, achieving substantially lower chunk recovery rates than state-of-the-art baselines while imposing negligible impact on task performance and inference latency. Moreover, as a plug-and-play solution, CanaryRAG can be seamlessly integrated into arbitrary RAG pipelines without requiring retraining or structural modifications, offering a practical and scalable safeguard for proprietary data.

• The paper presents CanaryRAG, a novel detection method that uses dual-path generation and canary injections to identify runtime integrity violations in RAG systems.
• It achieves up to a 96% reduction in chunk recovery rates under adaptive adversaries while maintaining nearly unchanged benign response quality.
• The approach is model-agnostic, requiring no retraining, and delivers scalable real-time defense with minimal computational overhead.

Detection of RAG Extraction Attacks via Dual-Path Runtime Integrity: An Analysis of CanaryRAG

Motivation and Threat Landscape

Retrieval-augmented generation (RAG) systems integrate LLMs with external knowledge bases to enhance informativeness and factuality. However, this architecture introduces severe confidentiality risks: adversaries may reconstruct proprietary knowledge through adaptive querying, a phenomenon known as RAG extraction attack. Empirical research demonstrates that modern black-box attacks can systematically exfiltrate private content from RAG-based deployments without access to retrieval indices, underlying knowledge bases, or system prompts (2604.10717).

Conventional defenses, including reranking and post-retrieval summarization, primarily raise extraction cost but offer little direct detection capability and are easily circumvented by adaptive adversaries. Furthermore, these approaches often mandate intrusive modifications to the retrieval pipeline, complicating integration and scaling. The practical limitations of semantic similarity as a leakage signal—due to the absence of intent observability—prompt the need for a fundamentally different defense paradigm.

CanaryRAG: Runtime Integrity via Dual-Path Canary Monitoring

The paper introduces CanaryRAG, a detection mechanism inspired by the stack canary approach in software security. Rather than retroactively identifying semantic or lexical leakage, CanaryRAG frames RAG extraction as a runtime integrity violation: any exposure or unexplained suppression of a designated non-semantic signal (canary) within the model output marks potentially unauthorized disclosure.

CanaryRAG injects high-entropy, context-irrelevant canary strings into retrieved chunks at inference time. Notably, these strings are constructed to be absent in benign outputs and inert with respect to downstream token distributions during normal operation. The system establishes two concurrent generation paths:

• Target Path: Processes user queries with canary-augmented context and requires canary invisibility in the benign output stream. Any surface-form exposure of a canary signals information leakage.
• Oracle Path: Simultaneously runs a probing instruction wrapping the original query, intentionally prompting the model to emit canaries under non-adaptive conditions. Here, the failure to emit sufficient canaries is interpreted as evidence of adaptive suppression or output obfuscation.

Detection is realized online via streaming token-wise comparison between the output buffer and tokenized canary sequences; generation is terminated at the first observed integrity violation.

Figure 1: CanaryRAG employs two concurrent generation paths, both augmented with canaries but governed by complementary integrity expectations. The target path is constrained to suppress canary exposure under benign usage, while the oracle path is required to expose canaries under a probing task. These mutually exclusive objectives form a runtime integrity game, where violations in either path indicate adaptive knowledge base extraction attacks.

Evaluation: Security, Utility, and Computational Overhead

Defense Efficacy

Empirical results under strong adaptive adversaries (Pirates, RAG-Thief) across multiple RAG agents (ChatDoctor, Mini-Wikipedia, Mini-BioASQ) show that CanaryRAG consistently outperforms reranking, summarization, and dual-path generation defenses such as RAGFort, delivering a relative mean chunk recovery rate (CRR) reduction to 4% of the no-defense baseline. This equates to up to 96% leakage mitigation, with worst-case CRR values near zero even under adversary-aware initialization.

User Utility and False Positives

Canary injection yields negligible degradation in benign response quality (BERTScore $\sim$0.515 vs 0.517 without defense), while false positive rates remain exceptionally low ($\le$0.15%). This validates the claim that high-entropy, semantically irrelevant canaries retain functional and perceptual transparency in non-adversarial usage. More importantly, contrary to prior strategies, CanaryRAG’s windowed blocking mechanism allows post-deployment adjustment of operational risk, decoupling overall system-level false block probability from the per-query false positive rate.

Figure 2: Latency Distribution Comparison Between No Defense and CanaryRAG.

Figure 3: Per-query latency scatter plots across datasets with and without CanaryRAG.

Runtime Overhead and Real-Time Detection

End-to-end latency measurements confirm that CanaryRAG introduces minimal computational overhead, as both canary insertion and output stream monitoring are lightweight, non-blocking operations. Online detection and immediate halting further constrain the effective leakage surface far below what chunk-level or output-level evaluation metrics (CRR) imply, especially when window-based blocking policies are enforced.

Robustness to Adaptive Attacks

Evaluation under compound bypass strategies—prompt-level canary suppression (e.g., explicit anti-token instructions), output obfuscation (reordering, encodings), and deferred task injection—demonstrates that the dual-path design (with oracle-based monitoring) is critical. Disabling the oracle path drastically reduces detection TPR under adaptive attacks (e.g., from 94–95% down to 23–52%). Conversely, the full CanaryRAG pipeline maintains a TPR $\sim$95% even for adversaries aware of both canaries and the defense mechanism.

System Design and Deployment Implications

CanaryRAG possesses unique properties pertinent to practical adoption:

• Model-Agnosticity and Plug-and-Play: No retraining or re-indexing is required. CanaryRAG can be integrated with arbitrary RAG pipelines without offline cost or modification to retrieval/embedding strategy.
• Scalability and Real-Time Enforcement: Defense is fully streaming-compatible and can be deployed as a “drop-in” guardrail, applying uniform policies across large-scale deployments.
• Tunability: Window-based blocking further amplifies security by ensuring that only persistent or repeated integrity violations (as opposed to single spurious events) trigger session-level interventions, which can be tuned post hoc based on empirical FPR.

  Figure 4: Window-based false blocking probability under agent-specific per-query FPRs. Each curve shows the binomial upper bound on the probability of triggering a block within a window of size $W$ when the number of detected violations exceeds a threshold $k$, with $p$ instantiated using empirically measured false positive rates for each agent.

  

  Figure 5: Effect of base per-query false positive rate on window-based blocking. Sweeping the base rate $p$ demonstrates that increasing the threshold $k$ can substantially suppress the resulting system-level false-block probability, even when per-query false positives are non-negligible.

Limitations and Future Work

The scope of CanaryRAG is limited: it exclusively targets sequential, interaction-driven corpus reconstruction, not fine-grained attribute inference or side-channel vectors. It does not mitigate direct corpus compromise, embedding-level extraction, or attacks relying solely on synthesized queries. Moreover, performance is not guaranteed against attacks leveraging highly sophisticated paraphrasing primarily for high-fidelity summary-level leakage—however, such attacks are of limited practicality for bulk corpus recovery, as validated empirically.

Further research is warranted on the integration of CanaryRAG with retrieval-time and post-generation filter defenses to construct a layered, defense-in-depth architecture. The extension of canary-based runtime integrity beyond RAG to other forms of confidential context injection presents an open avenue, including resilience against cross-session collusion and context-aware paraphrastic extraction.

Conclusion

By recasting RAG extraction defense as an integrity checking problem realized via explicit, orthogonal canary injection, CanaryRAG establishes a deployable, model-agnostic, and real-time runtime detection mechanism. Empirical results substantiate that this architecture attains minimal privacy-utility trade-off, dominant leakage mitigation, and robustness to adaptive evasion, all while imposing minimal overhead. CanaryRAG thus represents a rational, actionable advancement for securing proprietary knowledge assets in retrieval-augmented machine learning deployments (2604.10717).