Retrieval-Augmented Stores

• Retrieval-Augmented Stores are external knowledge repositories that store persistent, multi-form data including vector embeddings, prompt banks, and symbolic reasoning traces.
• They integrate heterogeneous retrieval architectures and routing strategies to efficiently select and fuse evidence from diverse storage systems.
• They empower applications in QA, multilingual search, and domain-specific tasks by delivering higher precision and streamlined retrieval compared to vector-only approaches.

Retrieval-augmented stores are external knowledge structures designed to be queried during retrieval-augmented generation, memory-augmented agents, and related retrieval-centric systems. In current research, the store is no longer restricted to a single vector database of passage embeddings. It may be a multilingual passage datastore, a Prompt Bank of key–prompt pairs with associated experts, a multi-store memory architecture, a persistent symbolic reasoning graph, or a coordinated retrieval plane spanning vector indices, knowledge graphs, full-text engines, and relational databases (Chirkova et al., 2024, Wu et al., 5 Jul 2025, Gaikwad, 8 Mar 2026, Wang et al., 4 Feb 2026, Yan et al., 12 Sep 2025). Across these variants, the store determines what is retrievable, how retrieval is routed, and how retrieved evidence conditions explanation, question answering, report generation, or agent behavior.

1. Scope and conceptual foundations

The common role of a retrieval-augmented store is to hold persistent external knowledge in a form that can be searched and then injected into a downstream model. In multilingual RAG, the store is a multilingual Wikipedia passage index queried by a multilingual retriever and reranker (Chirkova et al., 2024). In memory-augmented agents, the store may be partitioned into Short-Term Memory, Summary store, Long-Term Memory, and Episodic memory, with retrieval defined as a subset-selection problem over these stores (Gaikwad, 8 Mar 2026). In Uni-RAG, the store includes not only educational materials but also a Prompt Bank that stores style-aware prompt tokens plus key embeddings and associated MoE-LoRA experts (Wu et al., 5 Jul 2025). In AutoPrunedRetriever, the store is a persistent codebook of entities, relations, and edge sequences representing questions, answers, and facts (Wang et al., 4 Feb 2026).

This plurality of store types reflects a broader shift in RAG design. Standard systems typically store document embeddings and retrieve by semantic similarity alone. Recent work instead treats the store as a richer computational object: it may encode routing structure, prompt-conditioning structure, symbolic reasoning traces, or typed retrieval state. EfficientGraph-RAG makes this explicit by defining retrieval state as

$s_q = (p_q,\ \pi_q,\ E_q,\ z_q,\ U_q),$

where the stored artifacts include plans, traversal paths, verified evidence, verification signals, and reusable intermediate artifacts (Niu et al., 25 May 2026).

Store form	Stored objects	Representative systems
Vector datastore	Passage chunks pre-encoded into dense vectors	mRAG over Wikipedia (Chirkova et al., 2024)
Prompt/expert store	Key–prompt pairs and MoE-LoRA experts	Uni-RAG Prompt Bank (Wu et al., 5 Jul 2025)
Multi-store memory	STM, Sum, LTM, Epi as independent indices	Cost-sensitive store routing (Gaikwad, 8 Mar 2026)
Symbolic reasoning store	Entity, relation, edge-ID codebook and runs	AutoPrunedRetriever (Wang et al., 4 Feb 2026)
Heterogeneous retrieval plane	Milvus, Neo4j, Elasticsearch, MySQL	HetaRAG (Yan et al., 12 Sep 2025)

2. Store architectures and knowledge representation

A central distinction among retrieval-augmented stores lies in what they store as the primary retrieval object. In Uni-RAG, the Prompt Bank is defined as

$$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$

where $k_i \in \mathbb{R}^d$ is a learnable key vector and $P_i$ is a base prompt token or vector of tokens in the encoder’s embedding space (Wu et al., 5 Jul 2025). The key idea is that the store does not contain only documents; it contains style-aware prompt tokens and expert adapters that alter how the encoder interprets a query.

AutoPrunedRetriever adopts a different representation. Its persistent codebook is

$$\mathcal{C} = (E,R,\mathsf{M},\mathsf{Q},\mathsf{A},\mathsf{F},E_{\mathrm{emb}},R_{\mathrm{emb}}),$$

where $E$ and $R$ are entity and relation dictionaries, $\mathsf{M}$ is a sparse set of unique edges, and $\mathsf{Q}, \mathsf{A}, \mathsf{F}$ are sequences of edge IDs representing questions, answers, and facts (Wang et al., 4 Feb 2026). This store is symbolic rather than passage-centric, and retrieval operates over edge sequences and small coherent runs instead of raw text blocks.

Hybrid domain-specific systems combine structured and unstructured stores. SMART-SLIC pairs a Neo4j knowledge graph with a Milvus vector store for malware analysis and anomaly detection, using the KG for structured metadata and the VS for dense retrieval over documents and paragraph chunks (Barron et al., 2024). The legal Smart-SLIC variant combines a vector store, a knowledge graph, and a hierarchical NMF topic structure, with 190,283 unique nodes and 16,927,700 edges in the reported graph (Barron et al., 27 Feb 2025). A higher-education accreditation system similarly combines a Neo4j vector index with a Neo4j property graph containing AACSB standards, institutional documents, and LLM-extracted semantic entities and relations (Edwards, 2024).

Heterogeneous-store designs generalize this pattern further. HetaRAG proposes a retrieval plane that unifies vector indices, knowledge graphs, full-text engines, and structured databases, with Milvus, Neo4j, Elasticsearch, and MySQL serving complementary retrieval roles (Yan et al., 12 Sep 2025). This suggests that “store” is best understood as a family of persisted retrieval substrates rather than a single backend abstraction.

3. Retrieval, routing, and graph-aware access

Retrieval-augmented stores differ not only in representation but also in how queries are routed into them. In multi-store memory architectures, routing is made explicit as a subset-selection problem. For a query $q$, a routing policy $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$0 chooses

$$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$1

where $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$2, and the optimal cost-sensitive policy is formulated as

$$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$3

(Gaikwad, 8 Mar 2026). This formulation treats store selection itself as a first-class retrieval decision.

A related but simpler routing strategy appears in Context-Augmented Retrieval, where a query classifier maps a question to a domain label and retrieval is restricted to the corresponding domain-specific vector store and BM25 index (Ganesh et al., 2024). This classification-first design uses separate vector databases per domain and reports retrieval-time reductions of $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$4, $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$5, $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$6, and $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$7 across the reported query categories, with minimal impact on answer quality (Ganesh et al., 2024).

Style-aware systems route queries through learned prototype spaces. Uni-RAG first maps a multi-modal query $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$8 to a prototype $$\text{Prompt\_Bank} = \{(k_1, P_1), \dots, (k_N, P_N)\},$$9, matches that prototype against Prompt Bank keys by cosine similarity, retrieves top-$k_i \in \mathbb{R}^d$0 prompts, adapts them with MoE-LoRA, and only then performs document retrieval (Wu et al., 5 Jul 2025). This creates a two-stage retrieval process: query-to-prompt/expert retrieval followed by query-to-document retrieval.

Graph-aware reranking adds yet another routing layer. GraphER keeps the base vector or hybrid index unchanged, enriches objects with graph-relevant metadata during indexing, and then constructs a graph over the retrieved candidate set at query time (Miao et al., 26 Mar 2026). Its Graph Cohesive Smoothing update is

$k_i \in \mathbb{R}^d$1

with final scores $k_i \in \mathbb{R}^d$2 after convergence (Miao et al., 26 Mar 2026). The method is retriever-agnostic and reported latency overhead for reranking 200 candidates is about 0.49s for GCS and about 0.55s for GAT (Miao et al., 26 Mar 2026).

4. Optimization, efficiency, and state management

A major line of work asks how retrieval-augmented stores can be trained or organized efficiently. Uni-RAG freezes the feature encoder, tokenizer, and foundation model, and trains only the Prompt Bank, router, and MoE-LoRA parameters using a triplet loss plus a key-alignment term (Wu et al., 5 Jul 2025). This makes the prompt/expert store the trainable locus of adaptation while preserving a frozen retrieval backbone.

RA-DIT addresses a different layer of the stack by jointly adapting the LM and the retriever. It defines retriever probabilities over top-$k_i \in \mathbb{R}^d$3 chunks as

$k_i \in \mathbb{R}^d$4

and trains the retriever to match an LM-supervised retrieval distribution through a KL objective (Lin et al., 2023). This ties store retrieval more directly to what increases the LM’s answer likelihood, rather than to generic retrieval relevance alone.

Systems aimed at storage efficiency compress the store itself. BTR stores binarized intermediate token representations, requiring only 127GB of disk space for encoding 3 billion tokens in Wikipedia, accelerating state-of-the-art inference by up to 4x and reducing storage by over 100x while maintaining over 95% task performance (Cao et al., 2023). CompactDS addresses efficiency at web scale by filtering and indexing 1,851.8M passages and 380.5B words, then combining in-memory ANN retrieval and on-disk exact search to obtain subsecond latency on a single-node (Lyu et al., 2 Jul 2025).

Hyperparameter studies show that backend and chunking choices materially affect store behavior. One analysis reports that Chroma processes queries 13% faster, whereas Faiss yields higher retrieval precision; naive fixed-length chunking with small windows and minimal overlap outperforms semantic segmentation while remaining the quickest option; and re-ranking increases runtime by roughly a factor of 5 (Ammar et al., 13 May 2025). In biomedical QA over PubMed, retrieving 50 documents with BM25 before reranking with MedCPT optimally balances accuracy (0.90), recall (0.90), and response time (1.91s), while BM25 retrieval time remains stable at 82ms (Stuhlmann et al., 12 May 2025).

Structured state management pushes efficiency beyond static indexing. EfficientGraph-RAG stores reusable artifacts in a Shared Memory Pool

$k_i \in \mathbb{R}^d$5

with global, task, and agent levels, and reports cross-query cache hit rates ranging from 3.77% to 23.18% (Niu et al., 25 May 2026). This suggests that retrieval-augmented stores can also function as persistence layers for verified intermediate state, not only for raw evidence.

5. Applications and empirical evidence

Educational systems illustrate how store design can mediate both retrieval and explanation. Uni-RAG handles text, images, low-resolution images, sketches, art, audio, and multi-style multi-query inputs, and on SER reports Text→Image R@1 of 84.1, Sketch→Image R@1 of 85.1, and Audio→Image R@1 of 53.7 for Uni-RAG; allowing text+sketch raises T→I R@1 from 84.1 to 88.7 (Wu et al., 5 Jul 2025). The store here is explicitly pedagogical: style-aware prompt/expert retrieval is part of what makes the downstream explanation stage effective.

Multilingual settings show that the store interface includes language control, not only retrieval. The multilingual RAG baseline uses BGE-m3 to retrieve top-$k_i \in \mathbb{R}^d$6 candidates, reranks to $k_i \in \mathbb{R}^d$7, and stores Wikipedia passages in 13 languages (Chirkova et al., 2024). On XOR-TyDi QA, Arabic improves from 52.9 with an English store to 65.5 with a user-language store and 66.8 with an all-language store; Finnish improves from 45.2 to 58.9 and 59.1 under the same configurations (Chirkova et al., 2024). The same work shows that prompt engineering is needed to keep generation in the user language, which implies that the store’s language distribution interacts directly with the generator’s control interface.

Domain-specific systems further demonstrate that retrieval-augmented stores can be engineered around institutional or scientific corpora. SMART-SLIC reports an attempt rate of 100% and about 97% accuracy on 200 research questions when paired with its KG and VS, compared with about 20% accuracy on attempted document questions without RAG (Barron et al., 2024). The accreditation-reporting system reports mean RAGAs answer correctness of 0.787 and AACSB-query answer relevancy of 1.000, with the manually structured AACSB side performing markedly better than institution and hybrid queries on several retrieval-grounding metrics (Edwards, 2024).

Task-specific stores are also effective outside scholarly QA. In e-commerce product title translation, a bilingual Lucene/BM25 store of titles, bullet points, and descriptions supplies few-shot bilingual examples to a prompted Mixtral-8x7B-Instruct model. The reported chrF gains reach up to 15.3% for language pairs where the LLM has limited proficiency (Zhang et al., 2024). This indicates that retrieval-augmented stores can act as reusable domain-specific example libraries rather than only as evidence repositories.

6. Limitations, misconceptions, and emerging directions

A recurrent misconception is that retrieval-augmented stores are merely vector databases. Current systems contradict that view. Prompt Banks store style-aware prompts and expert parameters (Wu et al., 5 Jul 2025); multi-store memory systems treat routing among specialized stores as a decision problem (Gaikwad, 8 Mar 2026); symbolic graph stores persist reasoning subgraphs (Wang et al., 4 Feb 2026); and heterogeneous systems explicitly combine vector, graph, lexical, and relational backends (Yan et al., 12 Sep 2025).

A second misconception is that retrieving more context is always beneficial. Cost-sensitive store routing shows the opposite: on downstream question answering, oracle routing achieves 86.7% accuracy with 299 tokens, whereas uniform retrieval achieves 81.3% accuracy with 787 tokens (Gaikwad, 8 Mar 2026). The result is especially sharp in long-context settings, where irrelevant or stale context can degrade both efficiency and answer quality.

A third misconception is that semantic similarity alone is sufficient. GraphER argues that semantic search often fails when relevant evidence is scattered across multiple sources, and its graph-based reranking improves perfect recall on table retrieval, multi-hop QA, and chunked document retrieval benchmarks without requiring a knowledge graph (Miao et al., 26 Mar 2026). HetaRAG extends the same critique to monolithic vector-only systems and treats vector search, knowledge graphs, full-text search, and structured databases as complementary retrieval paradigms (Yan et al., 12 Sep 2025).

Open problems remain substantial. Uni-RAG notes weak audio handling, lack of explicit generation-aware training, and open questions about scaling Prompt Bank size and MoE routing (Wu et al., 5 Jul 2025). AutoPrunedRetriever depends on triple extraction quality and is evaluated only on text-only, single-turn QA (Wang et al., 4 Feb 2026). The multilingual RAG baseline identifies frequent code-switching in non-Latin alphabet languages, occasional fluency errors, wrong reading of provided documents, or irrelevant retrieval as central remaining issues (Chirkova et al., 2024). Heterogeneous-store systems still lack fully learned fusion and routing policies across all backends (Yan et al., 12 Sep 2025).

Emerging work increasingly treats the store as structured, stateful, and reusable. EfficientGraph-RAG formalizes retrieval state and couples a typed tree, role-specialized agents, and hierarchy-aware memory access in one framework (Niu et al., 25 May 2026). A more speculative direction replaces vector storage with a Field Memory Layer that stores knowledge as distributed holographic traces and retrieves it via phase interference; this proposal is presented as a cognitive RAG architecture and a white-paper proposal grounded in related complex-valued neural models rather than as a mature production design (Saklakov, 14 Nov 2025). Taken together, these lines of work suggest that the future of retrieval-augmented stores lies less in enlarging a single index and more in explicit organization: typed memory, adaptive routing, structured state reuse, and domain-specific representations that encode not only content, but also how that content should be retrieved and used.