Milvus: Vector Database for ANN Retrieval
- Milvus is a vector database optimized for storing embedding vectors and executing approximate nearest-neighbor (ANN) searches in diverse applications.
- It leverages multiple indexing methods like HNSW and IVF_FLAT while integrating with relational databases and search engines to manage metadata.
- Milvus supports continuous updates and real-time query execution, achieving low latency and scalable performance in complex retrieval systems.
Milvus is a vector database and vector-search backbone used to store embedding vectors and serve nearest-neighbor retrieval in contemporary systems for image retrieval, video search, product deduplication, recommendation, retrieval-augmented generation, and scientific search (Erfanian et al., 29 Mar 2026, Kulunk et al., 19 Sep 2025, Prajapati, 24 Nov 2025). Across these systems, Milvus is rarely the entire application stack. Instead, it is typically the dense-retrieval layer beneath higher-level orchestration, while relational databases, search engines, graph engines, or lakehouse storage retain metadata, text, graph structure, or temporal history (Luu et al., 15 Dec 2025, Li et al., 11 Mar 2026).
1. Architectural position
Milvus is commonly deployed as the vector-serving substrate inside multi-component systems. NeedleDB places PostgreSQL beside Milvus, with PostgreSQL tracking directories and image metadata while Milvus stores per-image embedding vectors and serves approximate nearest-neighbor search during query execution (Erfanian et al., 29 Mar 2026). Interactive video-retrieval systems use a similar split: one system stores BEiT-3 keyframe embeddings in Milvus, OCR text in Elasticsearch, and metadata such as image paths and video boundaries in MongoDB, all joined through a unified keyframe_id (Luu et al., 15 Dec 2025). LiveVectorLake pushes this separation further by assigning Milvus only the current-state “hot tier,” while Delta Lake over Parquet retains full version history and point-in-time state (Prajapati, 24 Nov 2025).
In HPC-oriented analysis, Milvus is described as a microservice-based system that explicitly separates compute and storage. The paper identifies proxies, streaming nodes, and data/query nodes, with proxies handling routing and final aggregation, streaming nodes maintaining open segments in memory and flushing them to durable storage, and data/query nodes loading segments for indexing and query execution (Ockerman et al., 8 Jun 2026). This differs from worker-centric systems and makes storage behavior, WAL paths, and segment lifecycle operationally central.
Comparison papers situate Milvus as the canonical “pure vector database.” TigerVector uses it as the specialized vector-search baseline for pure ANN workloads, while HMGI treats Milvus as representative of systems that excel at semantic similarity search but do not natively provide deep graph traversal or joint graph-vector execution (Liu et al., 20 Jan 2025, Chandra et al., 11 Oct 2025). This suggests that Milvus is best understood as an optimized semantic-retrieval substrate rather than a universal data-management layer.
2. Stored objects, schemas, and index configurations
What Milvus stores varies by workload, but the dominant pattern is that it holds dense vectors plus lightweight identifiers, while richer metadata remains elsewhere. In NeedleDB, Milvus stores one image vector per image per embedder, and the paper says that adding a new vision embedder only requires appending an entry to embedders.json, after which the system “automatically creates the corresponding Milvus collection and loads the model.” This strongly implies one Milvus collection per embedder, which matches the fact that supported embedders may have different vector dimensions (Erfanian et al., 29 Mar 2026). In the e-commerce deduplication system, Milvus indexes compact 128-dimensional vectors for candidate generation, with the most defensible reading being that text embeddings are the indexed objects used for first-stage retrieval (Kulunk et al., 19 Sep 2025). In interactive video retrieval, Milvus stores keyframe embeddings as conceptual pairs of {keyframe_id, embedding_vector} (Luu et al., 15 Dec 2025). LiveVectorLake provides one of the most explicit schemas, with a "chunks" collection containing chunk_id, a 384-dimensional FLOAT_VECTOR, doc_id, position, valid_from, status, and content (Prajapati, 24 Nov 2025).
Several papers specify concrete index choices. NeedleDB uses HNSW with and (Erfanian et al., 29 Mar 2026). LiveVectorLake also uses HNSW, but with , , and cosine distance (Prajapati, 24 Nov 2025). The e-commerce deduplication system evaluated IVF_FLAT and HNSW, explicitly excluded quantized indexes to avoid recall loss, and selected IVF_FLAT under its memory constraints (Kulunk et al., 19 Sep 2025). PentaRAG instead configures Milvus with a flat index and cosine distance on 1024-dimensional float32 vectors to maintain “100 % recall” (Syarubany et al., 18 Jun 2025). RAGPerf’s support matrix reports that Milvus exposes HNSW, IVF, ScaNN, and DiskANN, and supports both CPU and GPU devices (Li et al., 11 Mar 2026).
| Workload | Objects stored in Milvus | Reported index/search setup |
|---|---|---|
| NeedleDB | Per-image vectors, one vector per image per embedder | HNSW, , |
| Product deduplication | Compact 128-dimensional vectors for candidate generation | IVF_FLAT selected under memory constraints |
| LiveVectorLake | Active chunks plus lightweight metadata | HNSW, , , cosine |
| PentaRAG | Semantic-cache queries, adaptive-memory passages, main corpus passages | Flat index, cosine, 1024-D float32 |
Across these systems, Milvus collections are usually narrow and vector-centric. This suggests a recurring design principle: scalar-rich, temporally rich, or relational state is often externalized, while Milvus holds the retrieval-critical embedding representation and minimal keys needed for joins.
3. Query execution and retrieval semantics
Milvus executes nearest-neighbor retrieval, but the semantics of a full query are usually defined by application-side logic layered above the vector search. NeedleDB converts a natural-language query into one or more synthetic guide images, embeds each guide image with each configured vision model, then issues ANN searches against Milvus and fuses the resulting ranked lists with weighted reciprocal-rank fusion; its anomaly-detection module can filter low-quality guide images before aggregation (Erfanian et al., 29 Mar 2026). In the Turkish e-commerce system, Milvus performs first-stage candidate generation, after which a separate decider model consumes two text embeddings and two image embeddings for pairwise duplicate classification (Kulunk et al., 19 Sep 2025).
Video-retrieval systems demonstrate two additional patterns. One system encodes text or image queries into the BEiT-3 space and uses Milvus for both text-to-image semantic retrieval and image-to-image keyframe similarity, with cosine similarity as the retrieval objective (Luu et al., 15 Dec 2025). Vortex issues two independent Milvus searches per query, one in the CLIP space and one in the SigLIP2 space, then merges the ranked lists with Reciprocal Rank Fusion; Rocchio-updated feedback vectors are re-issued to Milvus for iterative refinement (Nguyen et al., 18 Jun 2026). MERVIN uses three Milvus-backed retrieval stores—keyframes, transcripts, and summaries—and then combines them procedurally through filtering, prioritization, and user-driven iterative refinement (Pham-Nguyen et al., 15 May 2026).
Retrieval-augmented inference systems use Milvus as non-parametric memory rather than only as a document retriever. SPARK-IL stores fused spectral embeddings together with ground-truth labels and generator identifiers; at inference it retrieves the nearest neighbors via cosine similarity and predicts by majority voting over the retrieved labels (Eutamene et al., 4 Apr 2026). PentaRAG uses Milvus for a semantic cache, adaptive knowledge memory, and the main retrieval layer; the semantic cache accepts a hit when cosine similarity exceeds an 85% threshold, while the main retrieval layer fetches the top-3 passages and asynchronously inserts the top-10 into adaptive knowledge memory (Syarubany et al., 18 Jun 2025). In these systems, Milvus is the local-evidence layer, while answer generation, reranking, voting, or fusion remain outside the database.
4. Updates, synchronization, and temporal behavior
A major theme in the recent literature is that Milvus is increasingly treated as a mutable serving index rather than a write-once artifact. NeedleDB explicitly maintains synchronization between filesystem state, PostgreSQL, and Milvus: a background FileWatcherService detects added, modified, and deleted files; new images are queued for embedding and insertion; deleted images are removed from both PostgreSQL and Milvus; modified images are re-embedded; and a periodic consistency checker reconciles state after offline changes (Erfanian et al., 29 Mar 2026). The e-commerce deduplication paper likewise describes Milvus as the dedicated vector database layer responsible for storing embeddings, supporting continuous insert/update workflows, and executing ANN search (Kulunk et al., 19 Sep 2025).
LiveVectorLake makes temporal asymmetry explicit. Only active chunks are written into Milvus; historical state is stored in Delta Lake. The cross-tier protocol is Delta-first: write to Delta Lake, then write to Milvus, and on Milvus failure mark the Delta record uncommitted and reconcile later. The paper characterizes the result as eventual consistency with bounded staleness of “<1 second” (Prajapati, 24 Nov 2025). Point-in-time retrieval does not execute against Milvus at all in that design; Milvus is reserved for current-state retrieval, while temporal queries are answered from Delta Lake snapshots (Prajapati, 24 Nov 2025).
Incremental-learning systems use Milvus as cumulative exemplar memory. SPARK-IL adds new embeddings after each incremental training phase, expanding the retrieval knowledge base to cover newly encountered generation techniques (Eutamene et al., 4 Apr 2026). PentaRAG writes every answered query synchronously to both its exact cache and its Milvus-backed semantic cache, and it grows adaptive knowledge memory online from retrieved passages (Syarubany et al., 18 Jun 2025). These designs indicate a shift from static vector collections toward continuously refreshed stores whose update semantics matter operationally.
5. Application domains and empirical behavior
Milvus-backed systems span retrieval settings with markedly different scales and latency targets. NeedleDB reports that on a 100K-image dataset, the entire end-to-end query time in its Fast configuration is 0.203 s, broken down as 0.136 s for guide-image generation and 0.053 s for “multi-embedder search and retrieval,” while also reporting sub-second query latency more generally and up to 93% relative MAP improvement on LVIS hard queries (Erfanian et al., 29 Mar 2026). The Turkish e-commerce system uses Milvus to support candidate generation over catalogs “exceeding 200 million items with just 100GB of system RAM consumption,” and reports a macro-average F1 of 0.90 versus 0.83 for a third-party solution (Kulunk et al., 19 Sep 2025).
Milvus also appears in temporal and versioned knowledge systems. LiveVectorLake reports current-state retrieval latencies of p50 65 ms, p95 110 ms, and p99 145 ms for hot-tier Milvus queries, while historical queries routed to Delta Lake take p50 1,200 ms, p95 1,890 ms, and p99 2,100 ms (Prajapati, 24 Nov 2025). PentaRAG reports mean latencies of 0.0423 s for its Milvus-backed semantic cache, 8.97 s for adaptive knowledge memory, and 9.6 s for naïve RAG, with the broader system reducing average GPU time to 0.24814 s per query through routing to faster paths (Syarubany et al., 18 Jun 2025).
Benchmarking papers show that Milvus performance is highly index- and deployment-dependent. In RAGPerf, FLAT limits end-to-end throughput to 0.69 QPS, ANN alternatives reach 1.68 to 1.81 QPS, HNSW requires over 100 GB of host memory and more than 1200 seconds of build time, and IVF_PQ is presented as the best balance of throughput, build time, and memory efficiency within that study (Li et al., 11 Mar 2026). In HPC evaluation, Milvus attains Recall@10 of 0.876 on Yandex-T2I-10M and 0.982 on Pes2o-VE-10M, peaks at 47,236 vectors/s for single-worker multiclient insertion and 61,344 vectors/s for multiworker insertion at 16 proxies and streaming nodes, and shows cloud-to-HPC gains averaging 12.98× lower latency and 5.21× higher QPS on the reported smaller benchmarks (Ockerman et al., 8 Jun 2026).
6. Trade-offs, limitations, and research directions
The most detailed system study of filtered search argues that Milvus achieves “superior recall stability through hybrid approximate/exact execution.” In that account, Milvus primarily operates as “Pre-filtering + ANNS,” uses a Dual-Pool graph traversal so invalid nodes do not consume valid-result budget, and adaptively falls back to exact filtered search when filtering becomes highly selective; the paper also notes a threshold of 93% filtered vectors in the code path and describes the resulting robustness as imposing a “latency floor” (Amanbayev et al., 11 Feb 2026). The same study recommends considering IVFFlat rather than assuming HNSW is always optimal for low-selectivity queries, and explicitly argues that Milvus should not be thought of as merely “FAISS plus filters” (Amanbayev et al., 11 Feb 2026).
Other papers expose different trade-offs. RAGPerf reports that Milvus may use more than 30 GB of host memory in one IVF_PQ experiment where LanceDB uses approximately 10 GB, because Milvus loads the full index into memory upon collection initialization; it also reports that GPU ANN provided only 0 throughput gain while consuming 70 GB of GPU memory for GPU_CAGRA in the tested RAG setting (Li et al., 11 Mar 2026). The HPC study finds that Milvus often reaches peak query performance with only 32 virtual cores, that multiworker insertion throughput decreases beyond 16 proxies and streaming nodes, that write amplification can reach 2.8× on one reported Pes2o subset, and that the authors were unable to complete full distributed query evaluation on Pes2o-VE because Lustre-backed deployments and MinIO-on-Lustre configurations produced failures (Ockerman et al., 8 Jun 2026).
Milvus is also framed by comparison papers as strong on vector-only retrieval but limited for deeply relational workloads. TigerVector presents Milvus as the specialized vector-database baseline for pure ANN, while HMGI treats Milvus-like systems as strong at standalone vector similarity search but weaker for relationship-heavy hybrid queries that require native graph traversal and graph-vector co-execution (Liu et al., 20 Jan 2025, Chandra et al., 11 Oct 2025). This suggests that Milvus is most mature as a dense-retrieval substrate and that future work lies either in richer hybrid execution inside the engine or in clearer composition with graph, lakehouse, and metadata systems around it.
A final recurring limitation is documentary rather than algorithmic: many application papers are explicit about using Milvus but sparse about collection schemas, shard layouts, scalar filtering, or search-time hyperparameters. KGSR-ADS explicitly notes that it lacks a “deep dive on vector database (milvus)’s optimization,” and several multimodal systems omit the Milvus collection-level details needed for exact reproduction (Wang et al., 25 Dec 2025, Pham-Nguyen et al., 15 May 2026). An important practical implication is that Milvus is often the operational core of these systems, but not yet always the best-specified component in the published artifact.