CategoryRAG: Category-Aware RAG

Updated 2 July 2026

CategoryRAG is a retrieval-augmented generation paradigm that uses category-specific retrieval to deliver transparent and adaptable LLM outputs.
It segments queries into semantic domains with dedicated evidence stores to prevent information leakage and minimize hallucinations.
The approach enables dynamic policy and schema updates without retraining, ensuring robust and explainable classification.

CategoryRAG is a Retrieval-Augmented Generation (RAG) paradigm that explicitly segments retrieval and reasoning over semantically meaningful categories to enhance the transparency, adaptability, and accuracy of LLM-based classification and fact retrieval. By routing queries through category-aware retrieval and leveraging structured evidence from distinct document, domain, or memory partitions, CategoryRAG systematically avoids information leakage between unrelated knowledge domains, amplifies explainability via category-specific evidence, and enables dynamic policy or schema adaptation without parameter retraining (Willats et al., 8 Aug 2025, Gadzhiev et al., 13 Apr 2026).

1. Foundations and Problem Formulation

CategoryRAG reframes supervised classification and structured retrieval as a conditional inference problem mediated by external, per-category knowledge stores. Standard parametric classification models estimate class membership probabilities $\hat y = \arg\max_y P(y \mid x; \theta)$ with fixed weights. In the CategoryRAG paradigm, inference explicitly incorporates retrieved contextual evidence from a set $\mathcal{C}$ of category- or policy-chunks per input $x$ : $P(y \mid x) = \sum_{c \in \mathcal{C}(x)} P(c \mid x) \cdot P(y \mid x, c)$ Here, $P(c \mid x) \propto \exp(\mathrm{sim}(\mathrm{emb}(x), \mathrm{emb}(c)))$ represents the relevance of chunk $c$ (e.g., policy passage, domain fact) to input $x$ , typically scored by cosine similarity. $P(y \mid x, c)$ is the posterior over labels given both the input and the chunk, operationalized by a generator LLM (Willats et al., 8 Aug 2025).

This decoupling allows the classification or grounding logic to be controlled by updating, partitioning, or weighting the external category-indexed corpus, rather than by opaque parameter updates. CategoryRAG thus generalizes classical RAG pipelines (open-domain QA, hallucination mitigation) to arbitrary class-labeled or structured evidence contexts.

2. System Architectures and Retrieval Protocol

CategoryRAG arrangements typically feature a multi-stage retrieval and reasoning architecture:

Category/Domain Planning: For each query, a planner LLM predicts a distribution over relevant semantic domains or categories (e.g., work, preferences, specific policy classes). The pipeline then queries only those knowledge bases whose predicted confidence exceeds a threshold $\gamma$ (Gadzhiev et al., 13 Apr 2026).
Category-Indexed Knowledge Stores: Each category/domain maintains its own independent vector or hybrid index—e.g., HNSW over free-text, inverted/B-tree over structure (date, polarity, type). Embeddings may be shared or specialized per category. Candidate retrieval is followed by domain-tuned thresholding (Gadzhiev et al., 13 Apr 2026), ensuring no off-topic facts are returned.
Retrieval Scoring and Filtering: Candidate facts are scored, often by a mixture of embedding similarity and secondary structured criteria (e.g., date proximity for temporal queries: $s_i = \alpha \cdot \textrm{sim}(q, v_i) + \beta \cdot \delta_i$ ). Results below a category-specific threshold $\mathcal{C}$ 0 are excluded.
Adversarial Robustness Layer: If no retrieved fact in any selected domain clears a refusal threshold $\mathcal{C}$ 1, the LLM is prompted to abstain from answering, thereby achieving high hallucination resistance.
Evidence-Aware Generation: The answer LLM is constrained to generate explanations or outputs referencing only the returned evidence blocks, which are assembled domain-wise into the context (Willats et al., 8 Aug 2025, Gadzhiev et al., 13 Apr 2026).

The combination of category planning, segmented indexes, and domain-tuned thresholds is the core architectural innovation of CategoryRAG.

3. Empirical Performance and Evaluation

CategoryRAG demonstrably improves both the accuracy and robustness of retrieval-augmented classification and memory systems:

Persona Memory: In Synthius-Mem, a six-domain CategoryRAG yields 94.37% accuracy and 99.55% adversarial robustness on the LoCoMo persona benchmark, outperforming flat index ablations by 6.2 pp in accuracy and 5.5 pp in hallucination resistance at sub-25 ms retrieval latency (Gadzhiev et al., 13 Apr 2026).
Policy-Grounded Classification: The Contextual Policy Engine applies CategoryRAG to hate speech detection, producing $\mathcal{C}$ 2 and accuracy $\mathcal{C}$ 3 on HateCheck (n=3,728), outperforming or rivaling SOTA APIs. When policy segments are updated (e.g., exempting an identity group), category-specific retrieval instantly adapts outputs, reflecting new requirements without retraining (Willats et al., 8 Aug 2025).
Explainability: Each decision carries explicit retrieved evidence and rationale, supporting audit, appeal, and regulatory compliance.

The retrieval component is generally evaluated using recall@k, MRR, MAP, and the refusal rate on false-premise (adversarial) queries. Faithfulness and answer appropriateness are assessed by manual or LLM-judged metrics, and ablation studies confirm unique advantages for category-aware over flat RAG (Gadzhiev et al., 13 Apr 2026).

4. Operational and Algorithmic Considerations

Key decisions and trade-offs in operationalizing CategoryRAG include:

Index Segmentation: Per-category/domain indexing multiplies index management cost but enables strict topical isolation and threshold calibration per schema. Index overhead is counterbalanced by massively reduced hallucination rates and explainability.
Domain Planner Accuracy: Misclassification of relevant domain can lead to lower recall or unnecessary refusals. Domain planners typically use LLM classifiers outputting a confidence vector; fallback to querying all domains guards against overly conservative predictions (Gadzhiev et al., 13 Apr 2026).
Threshold Tuning: Empirically optimal $\mathcal{C}$ 4 and $\mathcal{C}$ 5 differ by category and crucially determine the trade-off between answerability and hallucination resistance. Synthius-Mem reports $\mathcal{C}$ 6, $\mathcal{C}$ 7.
Structure-Aware Retrieval: Free-text embedding is necessary but not sufficient; integrating structured filters (dates, types) for specific categories further improves precision.

Latency remains dominated by LLM generation; CategoryRAG retrieval overhead is negligible at <25 ms for moderate-scale persona memory (Gadzhiev et al., 13 Apr 2026).

5. Case Studies and Applications

CategoryRAG has been operationalized across several domains:

System/Paper	Categories Used	Index Structure	Key Results/Findings
Synthius-Mem (Gadzhiev et al., 13 Apr 2026)	Biography, Experience, Preferences, Social, Work, Psychometrics	Per-domain hybrid indexes	94.4% accuracy, 99.6% adversarial robustness on LoCoMo
CPE for Hate Speech (Willats et al., 8 Aug 2025)	Policy categories/identities	Vector + reranker	0.988 $\mathcal{C}$ 8, 97% dynamic policy compliance
PreQRAG (Martinez et al., 20 Jun 2025)	Single- vs. multi-document types	Hybrid sparse/dense + rerank	+14 pp Top-10 recall (multi-doc), +14.7% MRR (single-doc)

Applications include safety-critical moderation (dynamic policy adherence, explanation), structured long-term memory agents (domain-segmented recall, robust abstention), and enhancement of open-domain RAG with question-type-dependent retrieval and context construction (Martinez et al., 20 Jun 2025).

6. Limitations and Future Work

CategoryRAG effectiveness depends on careful curation and ongoing maintenance of category schemas and evidence stores. Document or policy gaps, ambiguous category assignments, or poorly tuned thresholds can lead to missed answers or inappropriate refusals. Scaling to high-cardinality category sets introduces operational complexity, but is mitigated by automated planner LLMs and efficient index structures.

Future research directions include:

End-to-end learning of domain planning and threshold calibration.
Adaptive index expansion/contraction in response to evolving user needs or evidence distributions.
Richer forms of structured retrieval, combining neural and symbolic filters over heterogeneous schemas.
Human-in-the-loop workflows for repairing evidence and categories in response to observed classification or retrieval failures (Willats et al., 8 Aug 2025).
Improved integration with real-time feedback loops and user studies to optimize practical acceptance criteria (Gadzhiev et al., 13 Apr 2026).

7. Significance and Theoretical Implications

CategoryRAG marks a conceptual advance in retrieval-augmented architectures by uniting the advantages of policy-grounded explainability, dynamic knowledge adaptation, domain-specific precision, and hallucination resistance. By formalizing category-conditioned retrieval and generation, it sets a template for memory, moderation, and reasoning systems that must be controllable, auditable, and robust in diverse, rapidly evolving application domains. The distinctive ablation gains and adversarial benchmarks reported in Synthius-Mem and CPE position CategoryRAG as an emerging foundation for reliable, interpretable retrieval-augmented reasoning (Willats et al., 8 Aug 2025, Gadzhiev et al., 13 Apr 2026).