CleanBase: Detecting Malicious Documents in RAG Knowledge Databases

Abstract: Retrieval-augmented generation (RAG) is vulnerable to prompt injection attacks, in which an adversary inserts malicious documents containing carefully crafted injected prompts into the knowledge database. When a user issues a question targeted by the attack, the RAG system may retrieve these malicious documents, whose injected prompts mislead it into generating attacker-specified answers, thereby compromising the integrity of the RAG system. In this work, we propose CleanBase, a method to detect malicious documents within a knowledge database. Our key insight is that malicious documents crafted for the same attack-targeted questions often exhibit high semantic similarity, as attackers deliberately make them consistent to improve attack success rates. Accordingly, CleanBase constructs a similarity graph over the knowledge database, where each node represents a document and an edge connects two nodes if their semantic similarity--computed using an embedding model--exceeds a statistically determined threshold. Due to their inherent similarity, malicious documents tend to form cliques within this graph. CleanBase detects such cliques and flags the corresponding documents as malicious. We theoretically derive upper bounds on CleanBase's false positive and false negative rates and empirically validate its effectiveness. Experimental results across multiple datasets and prompt injection attacks demonstrate that CleanBase accurately detects malicious documents and effectively safeguards RAG systems. Our source code is available at https://github.com/WeifeiJin/CleanBase.

• The paper demonstrates that malicious insertions in RAG systems form dense semantic cliques, enabling effective pre-deployment detection.
• The method uses kNN graph construction and statistical pruning to isolate malicious clusters with an average FPR below 2% and near 0% FNR.
• Empirical tests across varied attacks and architectures show that CleanBase substantially lowers attack success rates with negligible impact on utility.

CleanBase: Graph-Based Detection of Malicious Documents in RAG Knowledge Bases

Motivation and Problem Context

The integration of Retrieval-Augmented Generation (RAG) architectures with LLMs introduces a potent attack surface: prompt injection via the knowledge database. Malicious actors can insert adversarial documents containing injected prompts, causing RAG systems to yield attacker-defined outputs for targeted queries. Prevailing defenses predominantly focus on hardening LLMs or filtering retrievals at inference, but these approaches either lack resilience against strong attacks, degrade RAG utility, or impose substantial runtime overhead. The detection and cleaning of the knowledge base itself—before deployment—are underexplored. "CleanBase: Detecting Malicious Documents in RAG Knowledge Databases" (2605.00460) systematically addresses this lacuna, introducing a practical and theoretically principled method for pre-deployment sanitization.

Methodology: Clique-Based Detection via Statistical Similarity Graphs

The primary observation underpinning CleanBase is that successful prompt injection campaigns against RAG often rely on inserting multiple highly similar malicious documents targeting the same question, promoting retrieval co-incidence and thus robust attack success. CleanBase leverages this structure by explicitly searching for densely connected clusters (cliques) of semantically similar documents in the embedding space.

The approach is as follows:

1. Similarity Graph Construction: The database is mapped into a $k$-nearest neighbor (kNN) graph using an embedding model (not necessarily matching that of the downstream retriever). Edges connect each document to its $k$ most similar peers using, e.g., cosine similarity.
2. Statistical Pruning: To remove spurious or overly generic connections present in benign text clusters, edges are pruned using a thresholding scheme derived from the empirical mean $\mu$ and standard deviation $\sigma$ of graph edge weights: only edges with weights exceeding $\mu + z \cdot \sigma$ are retained, with $z$ selected to be statistically stringent ($z=2.5$ corresponding to >99% confidence).
3. Clique Detection: The pruned graph is analyzed for cliques (typically of size ≥3); membership in a clique is treated as an indicator of malicious collusiveness, and all documents in detected cliques are flagged for removal.

   Figure 1: The three-step CleanBase pipeline: (I) kNN graph construction, (II) statistical pruning, and (III) clique-based flagging of malicious clusters.

The theoretical justification is rigorous: explicit upper bounds on false positive and negative rates (FPR/FNR) are derived in terms of the tail probabilities of the inter-document similarity distributions, kNN density, and statistical thresholding parameters.

Empirical Evidence and Attack Analysis

Extensive experiments across six datasets and nine attack variants—including synthetic, RAG-specific, and advanced adaptive attacks—validate the CleanBase framework.

Semantic Structure of Malicious Insertions

A crucial insight is visualized via t-SNE and pairwise similarity statistics: malicious documents crafted for the same target form tight clusters in embedding space, with much higher mutual cosine similarity than benign cohorts.

Figure 2: (a) t-SNE exhibits clustering of malicious documents by target question; (b) similarity histogram demonstrates markedly higher intra-malicious versus benign document similarities.

This empirical property justifies clique-centric detection: dense anomalous clusters are highly indicative of coordinated malicious insertions.

Defense Effectiveness Across Attacks and Architectures

On all major prompt injection attacks, CleanBase attains average FPR below 2% and FNR close to 0% for canonical attack settings. Strong numerical improvements are observed:

• RAG-specific attacks such as PoisonedRAG and GASLITEing, which maximize retrieval/generation probability alignment with the target question, are detected with FNR < 10%.
• Removal of detected cliques reduces downstream attack success rate (ASR) dramatically—e.g., from 97% to 6% on HotpotQA under PoisonedRAG.

Crucially, utility degradation is negligible: since benign database redundancy is high, the removal of a small number of false positive documents does not affect information coverage or retrieval accuracy, as shown in controlled ablation studies.

Robustness to System Variants

Defense performance is stable across RAG architectures (diverse LLMs and retriever backbones).

Figure 3: ASR and detection precision for CleanBase remain low and stable across LLM and retriever choices, demonstrating transferability.

Adaptation and Evasion

Adversaries aware of CleanBase could attempt two principal evasions:

• Lowering inter-malicious similarity: Paraphrasing or optimizing malicious documents to avoid forming cliques.
• Reducing number of malicious insertions per target: Inserting only one or two documents per target question.

Both strategies result in a trade-off: while detection rates drop (e.g., FNR rises to 20–96% depending on adaptation aggressiveness), attack success rates also decline dramatically due to diminished retrieval probability and coordination.

Figure 4: ASR and precision curves for PoisonedRAG variants as the number of malicious documents per target question is reduced—evading CleanBase by limiting clique size, but incurring steep drops in attack efficacy.

Sensitivity and Design Analysis

Systematic ablations confirm that:

• Both the kNN pre-filtering and statistical thresholding are vital; omitting either leads to severe FPR or FNR inflation.
• Cosine similarity yields optimal separation versus dot-product or Euclidean metrics for the embedding backbones studied.
• Increasing $k$ admits more clique formation (lowering FNR, increasing FPR), while stricter $z$ reduces FPR at the cost of FNR.
• Larger clique size thresholds and approximate clique/community detection variants may offer alternative FPR/FNR trade-offs.

Figure 5: Analysis of CleanBase's FPR/FNR as functions of hyperparameters $k$ and $k$0 highlights the efficacy/sensitivity frontier.

Limitations and Implications

CleanBase does not provide formal security guarantees against fully adaptive attacks; if adversaries can generate paraphrases that both escape semantic similarity detection and achieve high retrievability for the same target query, they can evade detection. However, such attacks empirically reduce their probability of success, providing a strong practical deterrent.

The graph-centric approach draws analogies to Sybil detection in social networks: attackers increasingly face a success-stealthiness trade-off. For fully general robustness, a fusion of higher-order graph analytics and population redundancy estimation may be warranted.

Practically, CleanBase is computationally efficient, scales to millions of documents, and can be deployed as a pre-processing filter—providing an effective first line of defense with negligible inference cost.

Conclusion

"CleanBase: Detecting Malicious Documents in RAG Knowledge Databases" (2605.00460) offers a technically robust, practically efficient system for knowledge base sanitization that operationalizes the structural signature of coordinated prompt injection: high semantic clique-formation among malicious documents. The approach sets a de facto baseline for database-centric RAG defense, and points to future work in graph-theoretic detection, adaptive adversarial game analysis, and minimal-assumption embedding model design.