FilterRAG: Effective Filtering in RAG Systems

• FilterRAG is a family of retrieval-augmented generation techniques that actively filters retrieved content to reduce noise and adversarial contamination.
• It employs methods such as threshold-based scoring and supervised classifiers to screen candidate texts for improved factuality and relevance.
• Empirical results across tasks like VQA and policy classification show that FilterRAG systems can closely approximate performance on unpoisoned baselines while mitigating errors.

Searching arXiv for papers on FilterRAG and related filtering methods in RAG. FilterRAG denotes a line of retrieval-augmented generation methods in which an explicit filtration operation mediates between retrieval and generation, or in which RAG is itself repurposed for filtering and classification rather than only question answering. Across the cited works, the term is used both for specific named systems—most notably a defense against knowledge poisoning and a zero-shot visual question answering framework—and for a broader design pattern that includes policy-grounded moderation, topic filtering, hierarchical content pruning, and token-level evidence gating (Edemacu et al., 4 Aug 2025, Sarwar, 25 Feb 2025, Willats et al., 8 Aug 2025).

1. Terminology and scope

The common principle in FilterRAG-style systems is that retrieval is not treated as a passive precursor to generation. Instead, retrieved items are actively screened, reweighted, routed, or rejected before they are consumed by the generator. In some cases the filter is document-level, as in poisoning defenses and topic-restricted retrieval; in others it is section-level, token-level, or policy-level. This yields a spectrum of methods whose shared objective is to reduce noise, adversarial contamination, or policy mismatch while preserving task-relevant evidence.

The literature surveyed here spans several distinct but related instantiations:

System	Filtered object	Primary objective
FilterRAG / ML-FilterRAG (Edemacu et al., 4 Aug 2025)	Retrieved candidate texts	Mitigate PoisonedRAG
Contextual Policy Engine (Willats et al., 8 Aug 2025)	Policy-grounded evidence	Classification as policy evaluation
FilterRAG for VQA (Sarwar, 25 Feb 2025)	External knowledge snippets	Reduce hallucinations in VQA
AT-RAG (Rezaei et al., 2024)	Topic-matched documents	Efficient multi-hop QA
HiFi-RAG (Nuengsigkapian, 27 Dec 2025)	URLs and sections	Hierarchical content filtering
ReFilter (Chen et al., 13 Feb 2026)	Tokens	Robust latent fusion under noisy retrieval

This distribution of usages suggests that “FilterRAG” is best understood not as a single canonical architecture but as a family of context-selection strategies embedded in RAG pipelines.

2. Canonical poisoning-defense FilterRAG

The most explicit use of the name appears in “Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation,” which proposes FilterRAG and ML-FilterRAG as defenses against PoisonedRAG (Edemacu et al., 4 Aug 2025). Both methods rely on a statistical property called Frequency Density (Freq-Density), motivated by the observation that adversarial texts injected for targeted attacks tend to have a higher density of words semantically overlapping with the target query and the attacker-desired answer than clean texts.

The paper defines Freq-Density for a retrieved candidate context text $d_j$ as

$$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$

where $q_i$ is a target query, $a_j$ is the output of a Smaller LLM when provided $(q_i, d_j)$, and semantic overlap is determined by cosine similarity above a threshold whose default is $0.6$. The threshold-based FilterRAG retrieves top-$s$ candidate texts, computes Freq-Density for each, filters out those with $\text{Freq-Density}[d_j] \geq \epsilon$, and sends the surviving top-$k$ texts to the generation model. Its filter function is

$$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$

The paper uses $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$0 as the default setting (Edemacu et al., 4 Aug 2025).

ML-FilterRAG replaces manual thresholding with a supervised binary classifier $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$1 operating on four features: Freq-Density, perplexity of the SLM output, joint log probability for that output, and the sum of frequencies of semantically similar words between $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$2 and $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$3. The prediction rule is

$$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$4

This design is motivated by the tradeoff in threshold tuning: high $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$5 risks letting adversarial texts through, while low $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$6 may reject many legitimate texts (Edemacu et al., 4 Aug 2025).

Empirical evaluation uses HotpotQA, MS-MARCO, and Natural Questions, with 100 target queries per dataset and five adversarial texts per target. On HotpotQA, CleanRAG has ATR $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$7, ASR $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$8, and Accuracy $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$9; PoisonedRAG degrades to ATR $q_i$0, ASR $q_i$1, and Accuracy $q_i$2; FilterRAG achieves ATR $q_i$3, ASR $q_i$4, and Accuracy $q_i$5; ML-FilterRAG achieves ATR $q_i$6, ASR $q_i$7, and Accuracy $q_i$8. On MS-MARCO, CleanRAG reports ATR $q_i$9, ASR $a_j$0, and Accuracy $a_j$1; PoisonedRAG reports ATR $a_j$2, ASR $a_j$3, and Accuracy $a_j$4; FilterRAG reports ATR $a_j$5, ASR $a_j$6, and Accuracy $a_j$7; ML-FilterRAG reports ATR $a_j$8, ASR $a_j$9, and Accuracy $(q_i, d_j)$0. The paper characterizes both defenses as bringing ATR and ASR close to CleanRAG while maintaining accuracy close to the unpoisoned baseline (Edemacu et al., 4 Aug 2025).

3. FilterRAG as task reformulation

A distinct interpretation appears in “Classification is a RAG problem: A case study on hate speech detection,” where the Contextual Policy Engine is described as a “FilterRAG” system because it uses RAG for filtering and classification rather than only QA (Willats et al., 8 Aug 2025). The conceptual shift is from asking “Is this hate speech?” to asking “Does this violate the hate speech policy?” The system retrieves relevant policy documentation at inference time and uses a preference-tuned LLM based on Llama-3.3 to reason over both the input content and retrieved policy fragments.

CPE has four components: a policy document containing definitions, edge cases, and exemplars; a retrieval system using embedding search and reranking; a generator that outputs the classification label, policy category, and grounded explanation; and an orchestrator that combines prompts, user input, and policy, including recall/precision calibration. Its salient property is that modifying the policy document changes system behavior instantly, so new protected groups or changed hate definitions can be incorporated by editing the policy file and refreshing the retrieval database, with no retraining required (Willats et al., 8 Aug 2025).

On HateCheck, CPE reports F1 $(q_i, d_j)$1 and Accuracy $(q_i, d_j)$2, with structured explanations grounded in retrieved policy chunks. When novel groups such as “Trump voters,” “Furries,” and “Homeless people” are added via policy edits, CPE’s F1 drops only $(q_i, d_j)$3 from $(q_i, d_j)$4 to $(q_i, d_j)$5, while the reported F1 drops for baselines are much larger: OpenAI Hate $(q_i, d_j)$6, LlamaGuard-Hate $(q_i, d_j)$7, and Perspective-Hate $(q_i, d_j)$8. In a policy-customization experiment where identities are exempted one at a time, attacks on newly “unprotected” identities are no longer classified as hateful, with mostly less than $(q_i, d_j)$9 performance impacts for protected groups, though some edge cases such as Trump voters see approximately $0.6$0 drop (Willats et al., 8 Aug 2025).

The same broad logic appears in visual question answering, but with multimodal rather than policy-grounded evidence. “FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA” combines BLIP-VQA with retrieval from Wikipedia and DBpedia, then uses a frozen GPT-Neo 1.3B model for answer generation (Sarwar, 25 Feb 2025). Images are divided into a $0.6$1 grid, which the paper reports as optimal in ablation. The framework achieves $0.6$2 accuracy on OK-VQA, with $0.6$3 on in-domain and $0.6$4 on out-of-distribution splits, and reports grounding scores of $0.6$5 on OK-VQA ID, $0.6$6 on OOD, and $0.6$7 on the combined setting (Sarwar, 25 Feb 2025). Here, FilterRAG denotes grounding against external factual knowledge to suppress hallucinated visual answers.

4. Filtering loci within the RAG pipeline

FilterRAG-style operations can be inserted at multiple points in the pipeline. AT-RAG filters at retrieval time by assigning a topic to the query using BERTopic and restricting search to documents matching that topic. It then alternates chain-of-thought reasoning, usefulness and hallucination grading, and query rewriting across up to $0.6$8 iterations. On 2WikiMultiHopQA, HotpotQA, and MuSiQue, AT-RAG reports overall scores of $0.6$9, $s$0, and $s$1, respectively, outperforming One Step RAG and Adaptive RAG on those benchmarks (Rezaei et al., 2024).

HiFi-RAG filters hierarchically in open-domain web retrieval. Gemini 2.5 Flash is used for query formulation, URL filtering, section filtering, and citation attribution, while Gemini 2.5 Pro is reserved for final answer generation. The URL filter reduces approximately $s$2 of URLs, and section filtering discards approximately $s$3 of noisy or irrelevant content. On the MMU-RAGent validation set, the final system improves ROUGE-L to $s$4 and DeBERTaScore to $s$5; on Test2025, the final system reports ROUGE-L $s$6 and DeBERTaScore $s$7 (Nuengsigkapian, 27 Dec 2025).

WebFilter moves the filter earlier still, into query construction. It models retrieval as a Markov Decision Process and trains query generation with Group Relative Policy Optimization, using a source-restricting reward and a retrieval-precision reward to encourage advanced search operators such as site:, date filters, phrase match, Boolean logic, and exclusion terms. The paper reports that advanced operator usage rises from under $s$8 of queries before WebFilter to over $s$9 after adding the source-restricting reward (Dai et al., 11 Aug 2025).

At the latent-fusion end of the spectrum, ReFilter performs token-level filtering rather than document rejection. Its architecture consists of a context encoder, a gated token-level filter, and a token fusion module that injects weighted token representations into the LLM hidden states. Across four general-domain QA benchmarks, it is reported as achieving the best average performance under both in-domain adaptation and out-of-domain transfer, and in zero-shot biomedical transfer it reaches $\text{Freq-Density}[d_j] \geq \epsilon$0 average accuracy with Qwen2.5-14B-Instruct (Chen et al., 13 Feb 2026).

5. Guarantees, limitations, and competing diagnoses

A central issue in FilterRAG is whether heuristic filtering can be made reliable. “Principled Context Engineering for RAG: Statistical Guarantees via Conformal Prediction” replaces heuristic thresholds with conformal filtering. After retrieval, snippets are scored by either embeddings or an LLM, calibrated on a labeled calibration set, and retained whenever the nonconformity score falls below the conformal threshold $\text{Freq-Density}[d_j] \geq \epsilon$1. The paper reports that coverage never fell below $\text{Freq-Density}[d_j] \geq \epsilon$2 for tested miscoverage rates $\text{Freq-Density}[d_j] \geq \epsilon$3, while context was reduced by $\text{Freq-Density}[d_j] \geq \epsilon$4–$\text{Freq-Density}[d_j] \geq \epsilon$5 relative to unfiltered retrieval. On NeuCLIR, unfiltered ARGUE F1 is $\text{Freq-Density}[d_j] \geq \epsilon$6; Conformal-Embedding reaches $\text{Freq-Density}[d_j] \geq \epsilon$7 at $\text{Freq-Density}[d_j] \geq \epsilon$8, $\text{Freq-Density}[d_j] \geq \epsilon$9 at $k$0, and $k$1 at $k$2 (Chakraborty et al., 22 Nov 2025).

At the same time, “Tackling the Inherent Difficulty of Noise Filtering in RAG” argues that identifying irrelevant information from retrieved content is inherently difficult because relevance often depends on triple-wise or higher-order relations, whereas standard transformer attention computes pairwise interactions. On this account, retrievers cannot fully filter noise, and standard fine-tuning such as LoRA-type low-rank attention updates faces a fundamental trade-off: suppressing noise also distorts relative attention among relevant tokens. The proposed remedy is nonlinear attention rectification, which yields mean scores of $k$3 versus $k$4 for LoRA and $k$5 for Vanilla in the reported reverse setting, and in “real” RAG mean scores of $k$6 for Llama versus $k$7 for LoRA and $k$8 for Vanilla (Liu et al., 5 Jan 2026).

The named FilterRAG papers and their relatives also report more practical limitations. In poisoning defense, lower $k$9 reduces ATR and ASR but risks discarding clean texts, while higher $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$0 risks letting adversarial texts pass; ML-FilterRAG is described as more robust to parameter choice (Edemacu et al., 4 Aug 2025). In policy-grounded classification, system quality is only as good as the policy document, retrieval failures reduce accuracy, and the retrieval/generation pipeline is slower and costlier than simple classifiers (Willats et al., 8 Aug 2025). These results collectively caution against a common misconception that a single filtering heuristic is sufficient for robust RAG.

6. Relation to routing, ranking, and dynamic reliability

Several adjacent systems extend the FilterRAG logic by replacing simple rejection with routing, ranking, or dynamic reliability estimation. In financial QA, Hybrid Document-Routed Retrieval uses Semantic File Routing as a document filter before chunk-based retrieval scoped to the identified documents. On FinDER, chunk-based retrieval reports average score $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$1, failure rate $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$2, correctness rate $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$3, and perfect-answer rate $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$4; SFR reports $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$5, $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$6, $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$7, and $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$8; HDRR reports $$\text{Filter}(d_j)= \begin{cases} 1, & \text{if } \text{Freq-Density}<\epsilon \ 0, & \text{otherwise}. \end{cases}$$9, $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$00, $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$01, and $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$02 (Cheng et al., 26 Mar 2026). The paper presents this as resolving a robustness–precision trade-off through document filtering followed by precise chunk retrieval.

Rank4Gen shifts the objective from ranking for query–document relevance to ranking for downstream response quality, and conditions the ranker on generator identity. On BrowseComp+ with a Qwen3-8B generator, it reports EM $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$03 and F1 $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$04; on ChronoQA, EM $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$05 and F1 $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$06 (Fan et al., 16 Jan 2026). Dartboard, by contrast, optimizes relevant information gain so that diversity emerges without an explicit relevance–diversity tradeoff parameter; on RGB simple QA, Dartboard Hybrid reports $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$07 QA accuracy and NDCG $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$08, and on integrated QA it reports $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$09 QA accuracy and NDCG $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$10 (Pickett et al., 2024). These methods are not usually labeled FilterRAG, but they address the same context-selection bottleneck.

Dynamic environments add a temporal dimension. RADAR models reliable context selection as graph-based energy minimization solved by Max-Flow Min-Cut, and augments the graph with a Bayesian memory node so that the system updates a belief state rather than archiving raw historical documents. In dynamic settings it reports accuracy $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$11 versus approximately $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$12 baseline and ASR as low as $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$13, while requiring only $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$14 state update per query and approximately $$\text{Freq-Density} = \frac{\sum_{w \in (q_i \oplus a_j) \cap d_j} \text{Freq}(w, d_j)}{\text{UniqueWords}(d_j)},$$15 KB per query rather than retaining document history (Chen et al., 21 May 2026). A plausible implication is that future FilterRAG systems will increasingly combine filtration with calibrated uncertainty, routing, memory, and generator-aware ranking rather than treating filtering as a single static thresholding step.