Classification with Retrieval-Augmented Generation

• Classification using RAG is a family of methods that predicts labels by conditioning on retrieved external evidence, rather than relying solely on internal memory.
• It employs dynamic multi-hop retrieval, hierarchical chunking, and graph-based evidence fusion to refine the decision-making process and reduce noise.
• Empirical studies show significant improvements in domains like content moderation, molecular prediction, and multimodal reasoning, highlighting both operational adaptability and precision.

Classification using retrieval-augmented generation denotes a family of methods in which a model predicts a label, class, or discrete decision after conditioning on retrieved external evidence rather than relying only on parametric memory. In the recent literature, this formulation appears in content moderation, molecular property prediction, question classification, multi-hop question answering, fact verification, and even automatic diagnosis of RAG failures themselves. Across these settings, retrieval is not merely ancillary context for free-form generation; it becomes part of the decision procedure by selecting examples, passages, graphs, summaries, or structured descriptors that determine, constrain, or justify the final class assignment (Hei et al., 2024, Chen et al., 2024, Chan et al., 4 Jun 2026, Liu et al., 13 Apr 2025).

1. Problem formulation and scope

Standard RAG systems typically retrieve a fixed top-$k$ set of documents via similarity matching and then condition an LLM on those documents. In the classification setting, several papers identify this as insufficient. DR-RAG states that traditional RAG retrieves a fixed number of documents regardless of deeper, context-dependent relevance, often misses “dynamic-relevant” documents crucial for multi-hop reasoning, and includes redundant or irrelevant documents that increase noise and inference cost (Hei et al., 2024). The hierarchical text segmentation work similarly argues that fixed-size or naive chunking can split texts at arbitrary points, producing chunks that are either too coarse or too narrow, which harms downstream performance on tasks including QuALITY multiple-choice classification (Nguyen et al., 14 Jul 2025).

A second line of work broadens the scope of what counts as retrievable evidence. TagRAG replaces fragment-level retrieval with domain-centric tag chains organized as a hierarchical DAG; LinearRAG replaces relation extraction with a relation-free Tri-Graph over entities, sentences, and passages; Cog-RAG uses a theme hypergraph and an entity hypergraph to connect global themes to local evidence; and GFM-RAG uses a query-dependent GNN over knowledge graphs for multi-hop retrieval on unseen datasets without fine-tuning (Tao et al., 18 Oct 2025, Zhuang et al., 11 Oct 2025, Hu et al., 17 Nov 2025, Luo et al., 3 Feb 2025). These systems retain the RAG pattern but redefine the retrieved object: not only passages, but also tags, graph neighborhoods, hyperedges, and entity scores.

The classification use case is especially explicit in content moderation and multimodal reasoning. Class-RAG frames moderation as a classification problem in which safe and unsafe inputs may differ only subtly, and argues that a dynamically updatable retrieval library enables immediate policy adaptation without retraining (Chen et al., 2024). HM-RAG, by contrast, treats classification as a multimodal synthesis problem across vector, graph, and web retrieval, and reports gains on ScienceQA and CrisisMMD through a hierarchical multi-agent pipeline (Liu et al., 13 Apr 2025). Taken together, these papers define classification using RAG as evidence-grounded decision-making under retrieval, rather than generation followed by post hoc interpretation.

2. Retrieval structures specialized for classification

A central design choice is the granularity and structure of the retrieved evidence. DR-RAG implements a two-stage retrieval process. First, it selects top-$k_1$ directly relevant documents by similarity matching. Second, for each first-stage document, it concatenates that document with the query and uses the richer context to retrieve additional potentially dynamic-relevant documents. This query-document concatenation is followed by classifier-guided document selection through either Classifier Inverse Selection, which removes documents classified as negative with all others, or Classifier Forward Selection, which keeps only candidates classified as positively contributing to the answer (Hei et al., 2024). Because the framework calls the LLM only once, it couples richer retrieval with a strict efficiency constraint.

Chunk structure is treated as another retrieval variable. The hierarchical text segmentation and clustering framework uses supervised text segmentation to identify natural boundaries and then unsupervised clustering to form larger, thematically coherent clusters. At retrieval time, it stores both segment-level and cluster-level embeddings and scores each chunk by

$$\text{cos}(q, C_i) = \max\Big(\text{cos}(q, E_{s1}),\ldots,\text{cos}(q, E_{sm}),\text{cos}(q, E_c)\Big).$$

Reported gains include QuALITY accuracy of 63.77\% at 512 tokens, QASPER F1 of 24.67 at 1024 tokens, and NarrativeQA ROUGE-L of 26.54 at 1024 tokens, with segment-plus-cluster retrieval outperforming cluster-only retrieval (Nguyen et al., 14 Jul 2025). This suggests that classification accuracy can depend as much on evidence segmentation as on the downstream classifier.

Graph-based systems generalize retrieval beyond flat chunk ranking. TagRAG constructs hierarchical domain tag chains, merges them into a DAG, fuses chain and neighborhood knowledge into summaries, and retrieves domain-centric tag chains during inference (Tao et al., 18 Oct 2025). LinearRAG constructs a Tri-Graph whose nodes are entities, sentences, and passages; retrieval first activates entities through semantic bridging and then ranks passages through global importance aggregation with Personalized PageRank (Zhuang et al., 11 Oct 2025). Cog-RAG begins with theme-level retrieval from a theme hypergraph and then performs theme-aligned entity-level retrieval in an entity hypergraph (Hu et al., 17 Nov 2025). GFM-RAG builds a knowledge-graph index and scores entities with a query-dependent GNN, allowing documents to be ranked by aggregated entity relevance (Luo et al., 3 Feb 2025). In all four cases, classification is mediated by structured retrieval paths rather than by nearest-neighbor passage matching alone.

3. Classification mechanisms inside RAG pipelines

The classification step itself varies substantially across systems. DR-RAG incorporates an explicit compact binary classifier, described as a fine-tuned BigBird-RoBERTa-Base model with millisecond-level inference. Its input is a triple or pair of the form $\langle q,\text{doc}_1,\text{doc}_2\rangle$, and its training labels are positive only when both documents are ground-truth relevant to the query:

$$\begin{align} Classifier(q, d^*, d^*) &= \text{positive} \ Classifier(q, d^*, d^{\Delta}) &= \text{negative} \ Classifier(q, d^{\Delta}, d^{\Delta}) &= \text{negative}. \end{align}$$

Here the classifier is not the final answering model; it is a relevance assessor embedded between retrieval and generation, and its role is to decide which retrieved documents meaningfully contribute to answering the query (Hei et al., 2024).

Class-RAG uses a different mechanism: in-context exemplar classification. Its retrieval library stores quadruplets of prompt, label, embedding, and explanation. For each input, the system retrieves the two nearest safe and two nearest unsafe examples and passes them to a fine-tuned Llama-3-8b classifier, which outputs a classification together with reasoning and citations (Chen et al., 2024). The retrieval library is not only a memory store but also a policy control surface. The paper’s “semantic hotfixing” result shows that modifying labels in the retrieval library can flip moderation behavior immediately, with 99.5\% of previously “safe” predictions switched to “unsafe” under a flipped library (Chen et al., 2024).

MolE-RAG frames classification as prompt-conditioned label generation. For a molecule $M$, a task instruction $I$, and a chosen set of context sources $C_S$, the prompt is

$P_S = (I, M, C_S), \qquad \hat{y} = L(P_S).$

The context may include retrieved chemistry literature, molecule-specific descriptors and identifiers, and structurally similar molecules from the training set. For classification, the output is constrained to exactly “Yes” or “No” (Chan et al., 4 Jun 2026). GFM-RAG offers yet another mechanism: the final entity scores

$P_q = \sigma(\text{MLP}(H_q^L))$

are interpreted as multi-label relevance scores, and the top-$k_1$0 entities or documents can serve directly as predicted classes or labels (Luo et al., 3 Feb 2025). HM-RAG’s Decision Agent instead fuses candidate classifications from multiple retrieval agents through consistency voting and expert model refinement (Liu et al., 13 Apr 2025). A plausible implication is that “classification using RAG” is best understood as a spectrum of decision architectures, ranging from explicit binary relevance classifiers to label generation constrained by retrieved evidence.

4. Training, supervision, and reasoning policies

Several recent systems improve classification not by altering the evidence store, but by modifying supervision and reasoning policy. HIRAG proposes that RAG models should acquire three progressively hierarchical abilities: Filtering, Combination, and RAG-specific Reasoning. Its instruction-tuning pipeline constructs queries, retrieved documents containing both relevant and noisy evidence, chain-of-thought rationales, and answers, using special output tokens such as <|REASON|> and <|ANSWER|> (Jiao et al., 8 Jul 2025). Filtering trains the model to select relevant information and suppress distractors; Combination trains it to synthesize across multiple retrieved sources; RAG-specific Reasoning trains it to perform deduction, comparison, and causal reasoning over retrieved evidence. The reported results include 27.8\% accuracy on MuSiQue for HIRAG with Llama3-8B, 73.7\% EM on PubMedQA for HIRAG with Llama2-7B, and 94.6\% accuracy on RGB-noise for HIRAG with Llama3-8B (Jiao et al., 8 Jul 2025).

Reward-RAG shifts supervision to the retriever. It uses CriticGPT to score $k_1$1query, document$k_1$2 pairs on a 0/1/2 scale, trains a reward model with mean squared error,

$k_1$3

and then mines positives and hard negatives to fine-tune the retriever with InfoNCE (Nguyen et al., 2024). In this design, classification quality depends on reward-aligned retrieval rather than on changes to the generator itself. The paper reports improved NDCG@10 for retrieval and best EM and accuracy on NQ and FEVER, together with competitive performance on TriviaQA and medical benchmarks (Nguyen et al., 2024).

AC-RAG replaces single-model decision-making with adversarial collaboration between a generalist Detector and a domain-specialized Resolver under a Moderator. The Detector decides whether retrieval is needed, generates sub-questions $k_1$4, and performs a post-check on sufficiency; the Resolver explains sub-questions, retrieves on the basis of those explanations, summarizes the retrieved results, and finally answers (Zhang et al., 18 Sep 2025). The framework is explicitly motivated by “Retrieval Hallucinations,” where fine-tuned models fail to recognize poor-quality retrieved documents. On medical QA, AC-RAG-8B reports 66.5 average accuracy versus 61.5 for Llama-3-8B RAG and 64.6 for Self-RAG; AC-RAG-70B reports 77.5 versus 76.4 for Llama-3-70B (Zhang et al., 18 Sep 2025). This suggests that classification performance may depend not only on evidence relevance but also on whether the system can challenge its own confidence before deciding.

5. Applications and reported empirical outcomes

The empirical literature spans content moderation, chemistry, multimodal question classification, and multi-hop reasoning. The reported outcomes are heterogeneous because the tasks and metrics differ, but they illustrate the range of what “classification using RAG” currently denotes.

System and domain	Classification setting	Reported outcome
Class-RAG	Content moderation	1.000 AUPRC on CoPro ID and OOD without obfuscation; 0.938 ID and 0.920 OOD mean AUPRC under obfuscations (Chen et al., 2024)
MolE-RAG	Molecular property prediction	ROC-AUC improves by up to 28 percentage points on classification tasks; Qwen3 on BBBP rises from 53.0 to 80.1 (Chan et al., 4 Jun 2026)
HM-RAG	ScienceQA and CrisisMMD	93.73\% on ScienceQA, 58.55\% average on CrisisMMD, and 3.56\% boost in question classification accuracy (Liu et al., 13 Apr 2025)
DR-RAG	Multi-hop QA-style answer classification	MuSiQue 38.90 / 34.03, HotpotQA 62.87 / 55.68, 2Wiki 55.62 / 55.18 with Llama3-8B; average response time reduced by 74.2\% vs. Adaptive-RAG (Hei et al., 2024)
HIRAG	Retrieval-grounded reasoning and yes/no/maybe tasks	73.7\% EM on PubMedQA and 94.6\% accuracy on RGB-noise (Jiao et al., 8 Jul 2025)

In content moderation, the distinctive result is not only high AUPRC but transfer through library updates. On I2P++, Class-RAG improves from 0.229 AUPRC with in-distribution references only to 0.791 when external references are added; on Unsafe Diffusion, it improves from 0.917 to 0.985 (Chen et al., 2024). In chemistry, MolE-RAG reports that structure-based retrieval is the strongest single contributor, with 17–20 ROC-AUC points over random or zero-shot baselines, and that naive BM25 retrieval over raw SMILES is ineffective while the hybrid query is consistently better (Chan et al., 4 Jun 2026). In multimodal classification, HM-RAG reports 72.06\% accuracy on CrisisMMD Task 1 and 51.50\% on Task 2, exceeding GPT-4o in both settings (Liu et al., 13 Apr 2025).

A plausible implication is that the main empirical advantage of classification-oriented RAG is not uniform across domains. In moderation, the dominant gain is operational adaptability; in chemistry, it is the integration of heterogeneous molecular context; in multimodal reasoning, it is multi-agent or multi-source evidence fusion; and in multi-hop QA-style classification, it is improved evidence recall under fixed LLM-call budgets.

6. Evaluation, error taxonomies, and recurring misconceptions

Evaluation in this area increasingly treats classification as a property of the RAG pipeline itself. RAG-Check defines two scores for multimodal RAG: a Relevancy Score (RS), which classifies retrieved items as relevant or irrelevant, and a Correctness Score (CS), which classifies objective response spans as correct or incorrect. Both are scalar outputs in $k_1$5 and can be thresholded, with the paper using a threshold such as $k_1$6 for binary decisions (Mortaheb et al., 7 Jan 2025). The RS and CS models achieve approximately 88\% accuracy on test data; the RS model aligns with human preferences 20\% more often than CLIP, and the CS model matches human preferences about 91\% of the time (Mortaheb et al., 7 Jan 2025). Here, classification is elevated from the end task to the assessment infrastructure.

A more general error-analysis perspective is developed by the taxonomy paper on RAG errors. It organizes failures into chunking, retrieval, reranking, and generation stages, with fine-grained labels E1 through E16, including Overchunking, Missed Retrieval, Low Recall, Abstention Failure, Fabricated Content, Parametric Overreliance, and Numerical Error (Leung et al., 15 Oct 2025). Its auto-evaluation system, RAGEC, achieves approximately 58\% stage agreement and approximately 40\% error-type agreement with expert human annotation (Leung et al., 15 Oct 2025). The paper also states that “Hallucination” (E10) is less common than retrieval/chunking errors. That result directly counters a frequent simplification in RAG discourse: the idea that most classification failures are generation failures.

Another recurrent simplification is that retrieving more context is automatically beneficial. The literature is more conditional. DR-RAG introduces classifier-guided filtering specifically to remove irrelevant or redundant documents (Hei et al., 2024). HIRAG treats filtering as a core learned ability rather than an implementation detail (Jiao et al., 8 Jul 2025). LinearRAG uses dynamic pruning during entity activation to curb noise and combinatorial expansion (Zhuang et al., 11 Oct 2025). The hierarchical chunking work reports diminishing returns or lower scores at very large chunk sizes such as 2048 tokens (Nguyen et al., 14 Jul 2025). Class-RAG reports that more references improve performance up to 8, but with diminishing returns (Chen et al., 2024). The consistent pattern is that classification-oriented RAG requires selective retrieval, not merely larger context windows.

Finally, these papers collectively distinguish between explanation and evidence. Class-RAG’s retrieval library includes explanations; HM-RAG uses voting and expert refinement; HIRAG trains explicit rationales; and RAG-Check evaluates atomic statements rather than whole responses (Chen et al., 2024, Liu et al., 13 Apr 2025, Jiao et al., 8 Jul 2025, Mortaheb et al., 7 Jan 2025). This suggests that classification using RAG is increasingly defined by auditable evidence flow: how evidence is segmented, selected, challenged, fused, and evaluated, not only by the final label.