RefineRAG: Word-Level Poisoning Attacks via Retriever-Guided Text Refinement

Abstract: Retrieval-Augmented Generation (RAG) significantly enhances LLMs, but simultaneously exposes a critical vulnerability to knowledge poisoning attacks. Existing attack methods like PoisonedRAG remain detectable due to coarse-grained separate-and-concatenate strategies. To bridge this gap, we propose RefineRAG, a novel framework that treats poisoning as a holistic word-level refinement problem. It operates in two stages: Macro Generation produces toxic seeds guaranteed to induce target answers, while Micro Refinement employs a retriever-in-the-loop optimization to maximize retrieval priority without compromising naturalness. Evaluations on NQ and MSMARCO demonstrate that RefineRAG achieves state-of-the-art effectiveness, securing a 90% Attack Success Rate on NQ, while registering the lowest grammar errors and repetition rates among all baselines. Crucially, our proxy-optimized attacks successfully transfer to black-box victim systems, highlighting a severe practical threat.

• The paper introduces RefineRAG, a two-stage attack that generates answer-controlling seed texts and optimizes individual word substitutions with a proxy retriever and BERT-based language model.
• RefineRAG achieves 90% targeted attack success on Natural Questions and 83% on MSMARCO using only five injected texts per question, while recording a 0.66 grammar-error rate and 0.01 repetition rate on NQ.
• The results show that fluent word-level poisoning transfers across several language models and retrievers, exposing limitations in defenses based only on perplexity, grammaticality, or duplicate-text detection.

Overview and Motivation

RefineRAG addresses a persistent weakness in knowledge-poisoning attacks on Retrieval-Augmented Generation (RAG) systems: the tension between retrieval effectiveness and textual stealthiness (2604.07403). Prior state-of-the-art attacks, notably PoisonedRAG, adopt a Separate-and-Concatenate (SoC) strategy in which an optimized retrieval trigger is spliced onto malicious content. This produces artifacts—abnormally high perplexity, grammatical errors, or verbatim query repetition—that render the poisoned text detectable by fluency- or deduplication-based defenses. The paper's central claim is that poisoning should instead be treated as a holistic word-level refinement problem, drawing on lexical-substitution techniques from adversarial NLP (HotFlip, TextFooler, BERT-Attack) but repurposing them from classification loss to retrieval-similarity maximization.

Threat Model

The adversary is resource-constrained and black-box: no access to victim retriever or generator internals, gradients, or configurations. The attacker can inject a small number of texts (five per question in the experiments) into a publicly editable corpus, using Contriever as a proxy retriever for local optimization under a transferability assumption. The goal is targeted precision manipulation—forcing a specific plausible-but-incorrect answer $R_t$ for a predefined question $Q$—rather than general performance degradation. This threat model is realistic for attacks on publicly editable knowledge sources, though the paper's reliance on embedding-space similarity between proxy and victim retrievers is an explicit assumption rather than a guaranteed property.

Methodology

The framework operates in two stages satisfying three principles: generation (the text must induce the target answer), retrieval (high embedding similarity to the query), and stealthiness (low perplexity, grammatical correctness).

Stage I: adversarial seed generation. An LLM (DeepSeek-V3) generates candidate texts across four iterations using a hybrid exploration/exploitation strategy: zero-shot and one-shot prompts expand the search space, while top-performing seeds are rewritten for refinement. A validation LLM (Llama-7B) enforces a hard constraint—candidates must trigger the target answer when presented as context—and valid seeds are ranked by retrieval similarity. This guarantees that all seeds entering Stage II are functionally toxic.

Stage II: word-level optimization (WLO). POS tagging identifies content words eligible for substitution, with keywords of $Q$ and $R_t$ frozen to prevent semantic drift. Each candidate word is masked, and a BERT-Large MLM proposes 20 context-aware replacements. A proxy retriever acts as the selection referee, choosing the substitution maximizing the similarity gain $\Delta Sim$, with beam search (width 3) over 10 iterations to escape local optima. Because substitutions are MLM-generated, the resulting text remains grammatical and low-perplexity by construction, unlike gradient-optimized triggers.

Empirical Results

Experiments use 100 target questions each from NQ (~2.6M documents) and MSMARCO (~8.8M documents), with target answers generated by DeepSeek-V3 and manually verified to conflict with ground truth. Five poisoned texts are injected per question; victims are Llama2-7B and Vicuna-7B retrieving top-5 via Contriever.

The headline result is a 90% attack success rate (L-ASR) on NQ and 83% on MSMARCO, the highest among all baselines, while simultaneously registering the lowest grammar error rate (0.66 on NQ) and repetition rate (0.01) of any method. The stealthiness contrast with PoisonedRAG is stark:

Method	NQ F1	NQ L-ASR	NQ PPL	NQ GE	NQ RR
PoisonedRAG (B)	0.94	0.54	55.11	2.21	0.28
PoisonedRAG (W)	0.95	0.59	372.93	6.54	0.00
Prompt Injection	0.75	0.80	107.01	0.88	1.00
Corpus Poisoning	0.66	0.00	8209.98	9.11	1.00
RefineRAG	0.89	0.90	118.33	0.66	0.01

PoisonedRAG (W) achieves marginally higher retrieval F1 but with a perplexity of 372.93 and over six grammar errors per text; PoisonedRAG (B) and Prompt Injection exhibit maximal repetition rates (0.28 and 1.00 respectively), exposing them to deduplication filters. The Corpus Poisoning attack fails entirely on generation (ASR 0.00). The implication is that current perplexity- and repetition-based defenses are insufficient against word-level refinement attacks, which occupy a genuinely different point in the effectiveness–stealthiness trade-off.

Transferability. Against six victim LLMs (Llama2, Vicuna, DeepSeek-R1, DeepSeek-V3, Qwen2.5, Qwen3), ASR remains between 0.81 and 0.92 on NQ, indicating model-agnostic transfer. Across retrievers, attacks optimized on Contriever retain ASR up to 0.70 on Contriever-ms and ANCE, though F1 degrades substantially (0.89 → 0.63 on NQ with ANCE), confirming that transfer is real but attenuated by domain shift. Attack performance also exhibits a non-monotonic relationship with retrieval scope $k$, peaking at $k=5$ and declining by $k=10$ due to dilution by benign documents.

Ablations and Sensitivity

Ablations confirm both stages are necessary and complementary. Removing Stage I (applying WLO to unrefined initial texts) drops NQ L-ASR from 0.90 to 0.72 and F1 from 0.89 to 0.63; removing Stage II drops F1 to 0.73. Micro-refinement thus depends heavily on a semantically validated seed corpus, while macro-generation alone underperforms on retrieval penetration.

Parameter sensitivity reveals consistent trade-offs. Performance peaks at $T=4$ macro-generation iterations (degrading at $T=5$, attributed to noise), 10 WLO iterations (with ASR peaking at 5 iterations on NQ even as F1 continues rising—additional refinement improves retrieval visibility but not necessarily generation misdirection), $Q$0 MLM candidates (larger sets introduce semantic noise), and beam width 3 (wider beams over-prioritize retrieval metrics at the cost of ASR). Results are robust to the choice of attacker generator: DeepSeek-V3 and Qwen3-Max yield nearly identical outcomes (L-ASR 0.90 vs. 0.85 on NQ; 0.83 vs. 0.83 on MSMARCO), supporting the claim that the framework's optimization strategy, rather than any particular LLM, drives effectiveness.

Limitations and Open Questions

The authors concede three limitations. First, the iterative MLM-based optimization incurs higher computational cost than concatenation-based attacks. Second, black-box transferability depends on embedding similarity between proxy and victim retrievers; efficacy against architecturally dissimilar retrievers, particularly sparse retrievers, remains untested, and whether "universal" perturbations exist is an open question. Third, while RefineRAG evades fluency-based filters, its robustness against semantic defenses such as external fact-checking has not been evaluated—an important gap, since the reported stealthiness metrics (PPL, GE, RR) measure only surface-level naturalness, and a factually implausible but fluent text could still be caught by verification pipelines. The evaluation is also confined to open-source 7B-scale victim LLMs and two QA benchmarks, leaving closed-source and larger-scale victims outside the evidence base.

Conclusion

RefineRAG reframes RAG knowledge poisoning as a two-stage word-level refinement problem, coupling LLM-based toxic seed generation with retriever-guided MLM substitution. It achieves state-of-the-art attack success (90% ASR on NQ) with the lowest grammar error and repetition rates among baselines, and demonstrates strong transfer to unseen LLMs and retrievers under a realistic black-box threat model. The results indicate that perplexity- and repetition-based defenses are inadequate against fine-grained semantic perturbations, and they motivate defenses that operate at the level of factual verification or embedding-space anomaly detection rather than surface fluency.