Vector-Based Agentic RAG

• Vector-based agentic RAG is a paradigm where autonomous LLM agents orchestrate iterative dense retrieval through a sequential decision process.
• It integrates advanced embedding methods and similarity metrics with dynamic actions like query reformulation and self-reflection to improve retrieval precision.
• Empirical studies show that agentic RAG optimizes token cost and response quality while mitigating issues like hallucination and embedding drift.

Searching arXiv for the papers on arXiv and closely related agentic/vector RAG work to ground the article in current literature. Vector-based agentic retrieval-augmented generation (RAG) denotes a class of systems in which an LLM-driven agent orchestrates dense retrieval over external knowledge represented as vectors, rather than relying on a fixed retrieve-then-generate pipeline. In this paradigm, queries and evidence are embedded in $\mathbb R^d$, retrieved by similarity search or hybrid scoring, and then incorporated into generation through planning, tool use, reflection, memory updates, or multi-agent coordination. Relative to traditional single-shot RAG, the defining change is that retrieval becomes a controllable action inside a sequential loop: the system decides whether to retrieve, what to retrieve, how to reformulate the query, which auxiliary tools to invoke, and when to terminate (Singh et al., 15 Jan 2025, Mishra et al., 7 Mar 2026).

1. Formal model and defining characteristics

Traditional RAG is described as a linear three-stage process: retrieve relevant context, augment or summarize that context, and generate an answer with an LLM. Agentic RAG extends this by inserting one or more autonomous agents backed by an LLM reasoner, memory modules, planning routines, reflection or self-critique, and a tool-invocation interface. In consequence, retrieval is no longer a single compulsory step; it becomes one possible action among others such as reasoning, external tool use, and stopping with a final answer (Singh et al., 15 Jan 2025).

A formalization given in the systematization literature models agentic RAG as a finite-horizon partially observable Markov decision process. The state space is

$S = S_{\mathrm{env}} \times M,$

where $S_{\mathrm{env}}$ is the latent knowledge state of the corpus and $M=M_t$ is the agent’s working memory. The action space is

$$A = A_{\mathrm{ret}} \cup A_{\mathrm{reason}} \cup A_{\mathrm{tool}} \cup \{\mathrm{STOP}\},$$

with retrieval actions such as $\mathrm{Retrieve\_vec}(q')$ and $\mathrm{Retrieve\_sparse}(q')$, reasoning actions, tool invocations, and explicit termination. The reward decomposes into per-step cost and terminal task fidelity:

$$R(s_t,a_t)=-C(a_t)\quad \text{for } t<T,\qquad R(s_T,\mathrm{STOP}) = R_{\mathrm{task}}(y,y^*).$$

The control policy $\pi_\theta(a_t\mid M_t)$ is implemented by an LLM that consumes the current memory and selects the next action (Mishra et al., 7 Mar 2026).

Within that formalization, vector retrieval is an atomic retrieval action. Let $f:\mathrm{Text}\rightarrow\mathbb R^d$ be an embedding function. Then the $S = S_{\mathrm{env}} \times M,$0-NN retrieval operator is defined as

$S = S_{\mathrm{env}} \times M,$1

For dense-only systems, this operator is often instantiated with cosine similarity or dot product; the key property is that the agent chooses when this operator should be applied and how its outputs should update memory (Mishra et al., 7 Mar 2026).

A practical consequence of this definition is that vector-based agentic RAG is not reducible to “RAG with a vector database.” The vector store is only one module inside a larger sequential decision process. This distinction is explicit in work comparing naive RAG, workflow-RAG, and agentic architectures, where the core difference is the autonomy of retrieval control rather than the presence of embeddings alone (Du et al., 3 Feb 2026).

2. Retrieval substrate: embeddings, indexes, and scoring functions

The vector substrate of agentic RAG begins with an embedding function that maps text, images, or multimodal inputs into a shared or task-specific latent space. In text settings, examples include OpenAI text-embedding-ada-002 for financial filings, all-MiniLM-L6-v2 with dimensionality $S = S_{\mathrm{env}} \times M,$2 for fintech, and Qwen3-Embedding-0.6B for semantic search in hierarchical agentic retrieval. In multimodal settings, SAR-RAG defines $S = S_{\mathrm{env}} \times M,$3 over SAR imagery using a Qwen2-VL vision encoder fine-tuned on SAR data, while mRAG uses EVA-CLIP_SF encoders for image and text representations (Lumer et al., 22 Nov 2025, Cook et al., 29 Oct 2025, Du et al., 3 Feb 2026, Ramirez et al., 4 Feb 2026, Hu et al., 29 May 2025).

The most common similarity function is cosine similarity. In the Boolean-agent formulation, with embeddings $S = S_{\mathrm{env}} \times M,$4 and document vectors $S = S_{\mathrm{env}} \times M,$5, naive RAG retrieves

$S = S_{\mathrm{env}} \times M,$6

where

$S = S_{\mathrm{env}} \times M,$7

A-RAG applies the same cosine form at sentence level, aggregates chunk relevance by $S = S_{\mathrm{env}} \times M,$8, and exposes the result as a callable semantic-search tool. mRAG uses $S = S_{\mathrm{env}} \times M,$9-normalized embeddings in a FAISS IndexFlatIP index, so top-$S_{\mathrm{env}}$0 retrieval is by largest inner product, which is equivalent to cosine ranking after normalization (Kenneweg et al., 2024, Du et al., 3 Feb 2026, Hu et al., 29 May 2025).

Indexing backends vary by application. Qdrant stores SAR embeddings and associated metadata such as class, dimensions, azimuth, and depression angle. Azure AI Search indexes SEC filing chunks and metadata including document name, page number, and section headings. ChromaDB backed by FAISS HNSW is used in fintech, while OpenSearch Serverless stores 1024-dimensional Titan Text Embedding Model V2 vectors in a keyword-versus-vector comparison study. These systems commonly support metadata-aware retrieval, approximate nearest-neighbor search, and hybrid sparse-dense retrieval (Ramirez et al., 4 Feb 2026, Lumer et al., 22 Nov 2025, Cook et al., 29 Oct 2025, Subramanian et al., 19 Dec 2025).

Although cosine similarity is the canonical baseline, some agentic systems explicitly generalize it. SmartVector replaces plain cosine retrieval with a convex combination of four interpretable signals:

$S_{\mathrm{env}}$1

with default weights $S_{\mathrm{env}}$2 and $S_{\mathrm{env}}$3. Temporal validity uses a half-life of $S_{\mathrm{env}}$4 days; confidence combines exponential decay, feedback reconsolidation, and logarithmic access reinforcement; relational importance grows like $S_{\mathrm{env}}$5. This formulation treats embeddings not as static points but as stateful retrieval objects with time, trust, and dependency structure (Xu, 22 Apr 2026).

3. Agentic orchestration patterns

A central design axis in vector-based agentic RAG is the interface by which the LLM controls retrieval. A-RAG exposes three retrieval tools at different granularities—keyword_search, semantic_search, and chunk_read—and lets the model choose among them in a ReAct-style loop. The system prompt instructs the model to work iteratively as “search → read → evaluate → (repeat) → answer,” and the model decides at each turn whether to issue a keyword search, semantic search, chunk read, or final answer. This differs from both single-shot RAG, which performs one fixed retrieval pass, and workflow-RAG, which follows a designer-specified control flow (Du et al., 3 Feb 2026).

A simpler control policy is the Boolean retrieval gate. Boolean-Agent RAG introduces a binary decision function $S_{\mathrm{env}}$6, or in its advanced form a function $S_{\mathrm{env}}$7 that decides whether an initial no-retrieval answer should be regenerated with retrieved context. The pipeline is: baseline answer $S_{\mathrm{env}}$8, decision $S_{\mathrm{env}}$9, then conditional retrieval and regeneration only if $M=M_t$0. This architecture does not make retrieval iterative, but it is agentic in the sense that retrieval is a model-mediated decision rather than a compulsory step (Kenneweg et al., 2024).

More elaborate agentic systems decompose retrieval management across specialized modules. In the fintech architecture, an Orchestrator Agent invokes eight sub-agents: Intent Classifier, Query Reformulator, Acronym Resolver, Retriever Manager, Sub-Query Generator, Re-Ranker, Summary Agent, and QA Agent. Query reformulation strips function words, normalizes case, expands known acronyms via a glossary, and injects domain synonyms. If the QA score is below threshold, the system generates 2–3 follow-up sub-queries from extracted keyphrases and re-enters retrieval. In the financial-domain evaluation over SEC filings, the LLM agent decides when to apply metadata filters, how to rewrite the question into one or more retrieval queries, and whether to invoke a cross-encoder reranker or a small-to-big context expander (Cook et al., 29 Oct 2025, Lumer et al., 22 Nov 2025).

Agentic orchestration also appears in multimodal systems. SAR-RAG is organized as a sequential tool-using agent that embeds an input SAR image, semantically searches a vector database, retrieves associated metadata and exemplar embeddings, and then conditions a multimodal LLM on the query image, retrieved exemplars, and optional question to generate target recognition outputs. mRAG goes further by integrating retrieval, listwise reranking, generation, and self-reflection into a loop that asks, for each candidate document, whether it contains answer-bearing facts and whether the tentative answer faithfully derives from that document; the process stops at the first self-validated answer (Ramirez et al., 4 Feb 2026, Hu et al., 29 May 2025).

These patterns show that “agentic” does not designate a single architecture. It includes at least retrieval gating, hierarchical tool selection, iterative query reformulation, modular multi-agent orchestration, and self-reflective evidence validation. A plausible implication is that the main architectural variable is the degree of autonomy granted to the policy over retrieval trajectory design, rather than any specific retriever or vector index.

4. Multimodal and domain-specific instantiations

Vector-based agentic RAG has been instantiated in domains where evidence is heterogeneous, long-form, or weakly aligned with the query surface form. In synthetic aperture radar automatic target recognition, SAR-RAG combines a domain-adapted Qwen2-VL vision encoder, a Qdrant vector store, and a LoRA-tuned LLaVA-Next v1.6 Mistral-7B. Each retrieved record is formatted as a textual slot of the form “Exemplar $M=M_t$1: Class = $M=M_t$2; Dimensions $M=M_t$3 = $M=M_t$4; Azimuth = $M=M_t$5. [Image_i]”, and the MLLM fuses visual and retrieved textual streams through cross-attention to produce class labels, descriptive attributes, and numeric dimensions. The training objective combines generation and classification losses as $M=M_t$6 (Ramirez et al., 4 Feb 2026).

In multimodal visual question answering more broadly, mRAG studies the design space of retrieval, reranking, and generation for large vision-LLMs. Query embeddings may use image only or image plus caption, while knowledge-base candidates may combine image and passage text. The study reports that the best retrieval modality configuration is image-to-image+text ($M=M_t$7), that listwise reranking mitigates “lost in the middle,” and that providing only the top-1 document yields better end-to-end response accuracy than providing $M=M_t$8, even when larger $M=M_t$9 improves raw retrieval accuracy. The unified agentic variant dynamically suppresses irrelevant context by evaluating candidates one at a time (Hu et al., 29 May 2025).

In finance and fintech, vector-based agentic RAG addresses long documents, dense terminology, and domain-specific metadata. The SEC-filings architecture chunks documents into overlapping 512-token windows, embeds them with text-embedding-ada-002, and indexes both embeddings and metadata in Azure AI Search. Query-time retrieval uses a hybrid engine that combines semantic vector similarity with lexical BM25-style matching over metadata fields, after which optional cross-encoder reranking and small-to-big chunk expansion supply more precise and more complete evidence. The fintech system, by contrast, uses dense retrieval over 30K+ chunks with all-MiniLM-L6-v2 embeddings and ChromaDB, then relies on acronym resolution, sub-query decomposition, and cross-encoder reranking to compensate for specialized jargon and acronym ambiguity (Lumer et al., 22 Nov 2025, Cook et al., 29 Oct 2025).

A contrasting domain is cyber threat intelligence, where relevant evidence is often distributed across multiple text fragments and documents and depends on explicit relations among threat actors, malware, and vulnerabilities. In that setting, vector RAG is evaluated alongside graph-based retrieval, an agentic graph-query repair variant, and a hybrid graph-text system. The results indicate that graph grounding improves performance on structured factual queries and that the hybrid graph-text approach improves answer quality by up to 35 percent on multi-hop questions compared to vector RAG while remaining more reliable than graph-only systems. This establishes an important boundary condition: dense vector retrieval is not uniformly dominant in relation-heavy reasoning tasks (Hamzic et al., 13 Apr 2026).

5. Empirical behavior and reported performance

Across benchmark studies, vector-based agentic RAG typically improves answer quality, retrieval precision, or context efficiency relative to static baselines, though the magnitude depends strongly on domain and task structure.

In open-domain multi-hop question answering, A-RAG reports GPT-5-mini LLM-Evaluation Accuracy of 74.1 on MuSiQue, 94.5 on HotpotQA, 89.7 on 2Wiki, 93.1 on Med., and 85.3 on Novel for A-RAG (Full), outperforming Direct Answer, Naive RAG, GraphRAG, LinearRAG, and A-RAG (Naive). The same study reports that increasing max iterations from 5 to 20 boosts GPT-5-mini by approximately 8 points LLM-Acc, that increasing chain-of-thought reasoning effort yields up to +25 points, and that A-RAG (Full) retrieves 2,737 tokens on HotpotQA and 2,930 on 2Wiki versus 5,358 and 5,506 for Naive RAG (Du et al., 3 Feb 2026).

In SAR automatic target recognition and VQA, retrieval quality and downstream performance both improve when the vector memory bank is attached to the MLLM. With $$A = A_{\mathrm{ret}} \cup A_{\mathrm{reason}} \cup A_{\mathrm{tool}} \cup \{\mathrm{STOP}\},$$0, SAR-RAG reports Accuracy@1 = 77.72%, Precision@5 = 74.39%, AnyCorrect@5 = 93.54%, AllCorrect@3 = 61.58%, and AllCorrect@5 = 53.54%. ATR classification accuracy rises from 99.04% to 99.24% for vehicle type and from 98.47% to 98.79% for descriptive qualities, while mounted weapon detection remains at 100.0%. The gains are larger on numerical tasks: weight MAE improves from 0.530 to 0.428, and vehicle-dimension MAE improves from 0.3328 to 0.2639. The study attributes this especially to fewer rare hallucination outliers in numerical answers (Ramirez et al., 4 Feb 2026).

In temporally versioned retrieval, SmartVector substantially changes the behavior of dense retrieval by adding temporal, confidence-weighted, and relational signals. On a synthetic benchmark of 258 vectors and 138 queries, the held-out test results are: plain cosine RAG at 31.2% overall accuracy, stale-rate 35.0%, ECE 0.470; cosine+temporal at 66.7%, stale 6.7%, ECE 0.259; SmartVector default at 61.7%, stale 13.3%, ECE 0.244; and SmartVector tuned at 64.8%. The study also reports a 77% reduction in embedding calls for single-word updates and finds that removing the temporal signal collapses accuracy to 36.2% (Xu, 22 Apr 2026).

In financial question answering over SEC filings, vector-based agentic RAG achieves a 68% win rate over a hierarchical node-based system with end-to-end latency 5.2 s versus 5.98 s. Cross-encoder reranking improves MRR@5 from 0.160 to 0.750 and Recall@5 from 0.50 to 1.00 at the optimal setting $$A = A_{\mathrm{ret}} \cup A_{\mathrm{reason}} \cup A_{\mathrm{tool}} \cup \{\mathrm{STOP}\},$$1, with latency rising from 0.22 s to 2.02 s. Small-to-big chunk retrieval achieves a 65% win rate over baseline chunking with only 0.2 s additional latency (Lumer et al., 22 Nov 2025).

In fintech, the agentic system improves strict Hit@5 from 54.12% to 62.35%, Adjusted Hit@5 from 58.82% to 69.41%, and semantic accuracy from 6.35 to 7.04, while latency rises from 0.79 s to 5.02 s. On a human-curated benchmark, procedural-question coverage reaches 100.0% for the agentic system versus 57.14% for the baseline (Cook et al., 29 Oct 2025).

Selective retrieval can also reduce unnecessary retrieval operations. On post-cutoff and random Wikipedia-derived evaluation sets, Advanced Boolean Agent RAG triggers retrieval for 214 of 256 fresh questions and 138 of 300 random questions, while keeping truthfulness and relevance within roughly 0.1–0.2 points of Naïve RAG. This supports the view that agentic control can optimize not only answer quality but also retrieval frequency and token cost (Kenneweg et al., 2024).

6. Limitations, misconceptions, and research directions

A common misconception is that vector search alone resolves the central failure modes of RAG. Multiple studies reject that conclusion. SmartVector argues that static embeddings are context-free artifacts with no notion of creation time, trustworthiness, or dependency structure, and shows that pure cosine retrieval performs poorly on versioned queries. The SoK further identifies embedding drift, hallucination propagation, memory poisoning, retrieval misalignment, Boolean and logical gaps, and cost acceleration in autonomous loops as systemic risks. Proposed mitigations include hybrid sparse checks, retrieval confidence thresholds, provenance tagging, bounded horizons, explicit abstention logic, and budget-aware orchestration (Xu, 22 Apr 2026, Mishra et al., 7 Mar 2026).

A second misconception is that more retrieved context is always better. In mRAG, providing only the top-1 document yields better end-to-end response accuracy than providing $$A = A_{\mathrm{ret}} \cup A_{\mathrm{reason}} \cup A_{\mathrm{tool}} \cup \{\mathrm{STOP}\},$$2, because large vision-LLMs overweight earlier context and are disturbed by irrelevant documents. SAR-RAG likewise reports an ablation in which AllCorrect@3 at 61.58% exceeds AllCorrect@5 at 53.54%, indicating a trade-off between retrieval depth and exemplar precision (Hu et al., 29 May 2025, Ramirez et al., 4 Feb 2026).

A third misconception is that vector databases are mandatory for high-performing agentic retrieval. The keyword-search comparison study reports that tool-based keyword search implementations within an agentic framework can attain over 90% of the performance metrics compared to traditional RAG systems without using a standing vector database, with average answer-correctness attainment of 91.48% across five non-financial datasets and a +8.5 percentage-point improvement over RAG on FinanceBench. This does not imply that vector retrieval is obsolete; the same study states that dense vector retrieval may still outperform pure keyword search when queries require paraphrase matching, abstract conceptual links, or low lexical overlap (Subramanian et al., 19 Dec 2025).

A fourth limitation concerns relation-heavy reasoning. In CTI, graph grounding improves structured factual retrieval, and hybrid graph-text retrieval improves answer quality by up to 35 percent on multi-hop questions compared to vector RAG. The system’s agentic graph-query repair loop reduces brittle query failures, but latency increases substantially relative to vector-only RAG. This suggests that the appropriate baseline for vector-based agentic RAG is not always a stronger vector system; in some domains it is a hybrid architecture that combines dense retrieval with graph or structured-query tools (Hamzic et al., 13 Apr 2026).

Current research directions reflect these constraints. The SoK identifies stable adaptive retrieval, cost-aware orchestration, formal trajectory evaluation, and oversight mechanisms as central problems. SmartVector points toward embeddings with explicit lifecycles, temporal metadata, confidence decay, and dependency edges. This suggests a shift from viewing vectors as inert semantic surrogates to treating them as managed memory objects inside autonomous retrieval systems. In that sense, the evolution of vector-based agentic RAG is not only about better retrievers or larger agents, but about more explicit control over when dense similarity is appropriate, how retrieved evidence should be validated, and how memory should evolve over time (Mishra et al., 7 Mar 2026, Xu, 22 Apr 2026).