---
title: 'Milvus: Vector Database for ANN Retrieval'
url: https://www.emergentmind.com/topics/milvus
type: topic
---

# Milvus: Vector Database for ANN Retrieval

Milvus is a vector database and vector-search backbone used to store embedding vectors and serve nearest-neighbor retrieval in contemporary systems for image retrieval, video search, product deduplication, recommendation, retrieval-augmented generation, and scientific search [2603.27464] [2509.15858] [2601.05270]. Across these systems, Milvus is rarely the entire application stack. Instead, it is typically the dense-retrieval layer beneath higher-level orchestration, while relational databases, search engines, graph engines, or lakehouse storage retain metadata, text, graph structure, or temporal history [2512.13169] [2603.10765].

## 1. Architectural position

Milvus is commonly deployed as the vector-serving substrate inside multi-component systems. NeedleDB places PostgreSQL beside Milvus, with PostgreSQL tracking directories and image metadata while Milvus stores per-image embedding vectors and serves approximate nearest-neighbor search during query execution [2603.27464]. Interactive video-retrieval systems use a similar split: one system stores BEiT-3 keyframe embeddings in Milvus, OCR text in Elasticsearch, and metadata such as image paths and video boundaries in MongoDB, all joined through a unified `keyframe_id` [2512.13169]. LiveVectorLake pushes this separation further by assigning Milvus only the current-state “hot tier,” while Delta Lake over Parquet retains full version history and point-in-time state [2601.05270].

In HPC-oriented analysis, Milvus is described as a microservice-based system that explicitly separates compute and storage. The paper identifies proxies, streaming nodes, and data/query nodes, with proxies handling routing and final aggregation, streaming nodes maintaining open segments in memory and flushing them to durable storage, and data/query nodes loading segments for indexing and query execution [2606.08950]. This differs from worker-centric systems and makes storage behavior, WAL paths, and segment lifecycle operationally central.

Comparison papers situate Milvus as the canonical “pure vector database.” TigerVector uses it as the specialized vector-search baseline for pure ANN workloads, while HMGI treats Milvus as representative of systems that excel at semantic similarity search but do not natively provide deep graph traversal or joint graph-vector execution [2501.11216] [2510.10123]. This suggests that Milvus is best understood as an optimized semantic-retrieval substrate rather than a universal data-management layer.

## 2. Stored objects, schemas, and index configurations

What Milvus stores varies by workload, but the dominant pattern is that it holds dense vectors plus lightweight identifiers, while richer metadata remains elsewhere. In NeedleDB, Milvus stores one image vector per image per embedder, and the paper says that adding a new vision embedder only requires appending an entry to `embedders.json`, after which the system “automatically creates the corresponding Milvus collection and loads the model.” This strongly implies one Milvus collection per embedder, which matches the fact that supported embedders may have different vector dimensions [2603.27464]. In the e-commerce deduplication system, Milvus indexes compact 128-dimensional vectors for candidate generation, with the most defensible reading being that text embeddings are the indexed objects used for first-stage retrieval [2509.15858]. In interactive video retrieval, Milvus stores keyframe embeddings as conceptual pairs of `{keyframe_id, embedding_vector}` [2512.13169]. LiveVectorLake provides one of the most explicit schemas, with a `"chunks"` collection containing `chunk_id`, a 384-dimensional `FLOAT_VECTOR`, `doc_id`, `position`, `valid_from`, `status`, and `content` [2601.05270].

Several papers specify concrete index choices. NeedleDB uses HNSW with \(M = 48\) and \(\mathit{efConstruction} = 200\) [2603.27464]. LiveVectorLake also uses HNSW, but with \(M = 16\), \(\mathit{efConstruction} = 200\), and cosine distance [2601.05270]. The e-commerce deduplication system evaluated IVF_FLAT and HNSW, explicitly excluded quantized indexes to avoid recall loss, and selected IVF_FLAT under its memory constraints [2509.15858]. PentaRAG instead configures Milvus with a flat index and cosine distance on 1024-dimensional float32 vectors to maintain “100 % recall” [2506.21593]. RAGPerf’s support matrix reports that Milvus exposes HNSW, IVF, ScaNN, and DiskANN, and supports both CPU and GPU devices [2603.10765].

| Workload | Objects stored in Milvus | Reported index/search setup |
|---|---|---|
| NeedleDB | Per-image vectors, one vector per image per embedder | HNSW, \(M=48\), \(\mathit{efConstruction}=200\) |
| Product deduplication | Compact 128-dimensional vectors for candidate generation | IVF_FLAT selected under memory constraints |
| LiveVectorLake | Active chunks plus lightweight metadata | HNSW, \(M=16\), \(\mathit{efConstruction}=200\), cosine |
| PentaRAG | Semantic-cache queries, adaptive-memory passages, main corpus passages | Flat index, cosine, 1024-D float32 |

Across these systems, Milvus collections are usually narrow and vector-centric. This suggests a recurring design principle: scalar-rich, temporally rich, or relational state is often externalized, while Milvus holds the retrieval-critical embedding representation and minimal keys needed for joins.

## 3. Query execution and retrieval semantics

Milvus executes nearest-neighbor retrieval, but the semantics of a full query are usually defined by application-side logic layered above the vector search. NeedleDB converts a natural-language query into one or more synthetic guide images, embeds each guide image with each configured vision model, then issues \(m \times l\) ANN searches against Milvus and fuses the resulting ranked lists with weighted reciprocal-rank fusion; its anomaly-detection module can filter low-quality guide images before aggregation [2603.27464]. In the Turkish e-commerce system, Milvus performs first-stage candidate generation, after which a separate decider model consumes two text embeddings and two image embeddings for pairwise duplicate classification [2509.15858].

Video-retrieval systems demonstrate two additional patterns. One system encodes text or image queries into the BEiT-3 space and uses Milvus for both text-to-image semantic retrieval and image-to-image keyframe similarity, with cosine similarity as the retrieval objective [2512.13169]. Vortex issues two independent Milvus searches per query, one in the CLIP space and one in the SigLIP2 space, then merges the ranked lists with Reciprocal Rank Fusion; Rocchio-updated feedback vectors are re-issued to Milvus for iterative refinement [2606.19682]. MERVIN uses three Milvus-backed retrieval stores—keyframes, transcripts, and summaries—and then combines them procedurally through filtering, prioritization, and user-driven iterative refinement [2605.16120].

Retrieval-augmented inference systems use Milvus as non-parametric memory rather than only as a document retriever. SPARK-IL stores fused spectral embeddings together with ground-truth labels and generator identifiers; at inference it retrieves the \(K\) nearest neighbors via cosine similarity and predicts by majority voting over the retrieved labels [2604.03833]. PentaRAG uses Milvus for a semantic cache, adaptive knowledge memory, and the main retrieval layer; the semantic cache accepts a hit when cosine similarity exceeds an 85% threshold, while the main retrieval layer fetches the top-3 passages and asynchronously inserts the top-10 into adaptive knowledge memory [2506.21593]. In these systems, Milvus is the local-evidence layer, while answer generation, reranking, voting, or fusion remain outside the database.

## 4. Updates, synchronization, and temporal behavior

A major theme in the recent literature is that Milvus is increasingly treated as a mutable serving index rather than a write-once artifact. NeedleDB explicitly maintains synchronization between filesystem state, PostgreSQL, and Milvus: a background `FileWatcherService` detects added, modified, and deleted files; new images are queued for embedding and insertion; deleted images are removed from both PostgreSQL and Milvus; modified images are re-embedded; and a periodic consistency checker reconciles state after offline changes [2603.27464]. The e-commerce deduplication paper likewise describes Milvus as the dedicated vector database layer responsible for storing embeddings, supporting continuous insert/update workflows, and executing ANN search [2509.15858].

LiveVectorLake makes temporal asymmetry explicit. Only active chunks are written into Milvus; historical state is stored in Delta Lake. The cross-tier protocol is Delta-first: write to Delta Lake, then write to Milvus, and on Milvus failure mark the Delta record uncommitted and reconcile later. The paper characterizes the result as eventual consistency with bounded staleness of “<1 second” [2601.05270]. Point-in-time retrieval does not execute against Milvus at all in that design; Milvus is reserved for current-state retrieval, while temporal queries are answered from Delta Lake snapshots [2601.05270].

Incremental-learning systems use Milvus as cumulative exemplar memory. SPARK-IL adds new embeddings after each incremental training phase, expanding the retrieval knowledge base to cover newly encountered generation techniques [2604.03833]. PentaRAG writes every answered query synchronously to both its exact cache and its Milvus-backed semantic cache, and it grows adaptive knowledge memory online from retrieved passages [2506.21593]. These designs indicate a shift from static vector collections toward continuously refreshed stores whose update semantics matter operationally.

## 5. Application domains and empirical behavior

Milvus-backed systems span retrieval settings with markedly different scales and latency targets. NeedleDB reports that on a 100K-image dataset, the entire end-to-end query time in its Fast configuration is 0.203 s, broken down as 0.136 s for guide-image generation and 0.053 s for “multi-embedder search and retrieval,” while also reporting sub-second query latency more generally and up to 93% relative MAP improvement on LVIS hard queries [2603.27464]. The Turkish e-commerce system uses Milvus to support candidate generation over catalogs “exceeding 200 million items with just 100GB of system RAM consumption,” and reports a macro-average F1 of 0.90 versus 0.83 for a third-party solution [2509.15858].

Milvus also appears in temporal and versioned knowledge systems. LiveVectorLake reports current-state retrieval latencies of p50 65 ms, p95 110 ms, and p99 145 ms for hot-tier Milvus queries, while historical queries routed to Delta Lake take p50 1,200 ms, p95 1,890 ms, and p99 2,100 ms [2601.05270]. PentaRAG reports mean latencies of 0.0423 s for its Milvus-backed semantic cache, 8.97 s for adaptive knowledge memory, and 9.6 s for naïve RAG, with the broader system reducing average GPU time to 0.24814 s per query through routing to faster paths [2506.21593].

Benchmarking papers show that Milvus performance is highly index- and deployment-dependent. In RAGPerf, FLAT limits end-to-end throughput to 0.69 QPS, ANN alternatives reach 1.68 to 1.81 QPS, HNSW requires over 100 GB of host memory and more than 1200 seconds of build time, and IVF_PQ is presented as the best balance of throughput, build time, and memory efficiency within that study [2603.10765]. In HPC evaluation, Milvus attains Recall@10 of 0.876 on Yandex-T2I-10M and 0.982 on Pes2o-VE-10M, peaks at 47,236 vectors/s for single-worker multiclient insertion and 61,344 vectors/s for multiworker insertion at 16 proxies and streaming nodes, and shows cloud-to-HPC gains averaging 12.98× lower latency and 5.21× higher QPS on the reported smaller benchmarks [2606.08950].

## 6. Trade-offs, limitations, and research directions

The most detailed system study of filtered search argues that Milvus achieves “superior recall stability through hybrid approximate/exact execution.” In that account, Milvus primarily operates as “Pre-filtering + ANNS,” uses a Dual-Pool graph traversal so invalid nodes do not consume valid-result budget, and adaptively falls back to exact filtered search when filtering becomes highly selective; the paper also notes a threshold of 93% filtered vectors in the code path and describes the resulting robustness as imposing a “latency floor” [2602.11443]. The same study recommends considering IVFFlat rather than assuming HNSW is always optimal for low-selectivity queries, and explicitly argues that Milvus should not be thought of as merely “FAISS plus filters” [2602.11443].

Other papers expose different trade-offs. RAGPerf reports that Milvus may use more than 30 GB of host memory in one IVF_PQ experiment where LanceDB uses approximately 10 GB, because Milvus loads the full index into memory upon collection initialization; it also reports that GPU ANN provided only \(1.04\times\) throughput gain while consuming 70 GB of GPU memory for GPU_CAGRA in the tested RAG setting [2603.10765]. The HPC study finds that Milvus often reaches peak query performance with only 32 virtual cores, that multiworker insertion throughput decreases beyond 16 proxies and streaming nodes, that write amplification can reach 2.8× on one reported Pes2o subset, and that the authors were unable to complete full distributed query evaluation on Pes2o-VE because Lustre-backed deployments and MinIO-on-Lustre configurations produced failures [2606.08950].

Milvus is also framed by comparison papers as strong on vector-only retrieval but limited for deeply relational workloads. TigerVector presents Milvus as the specialized vector-database baseline for pure ANN, while HMGI treats Milvus-like systems as strong at standalone vector similarity search but weaker for relationship-heavy hybrid queries that require native graph traversal and graph-vector co-execution [2501.11216] [2510.10123]. This suggests that Milvus is most mature as a dense-retrieval substrate and that future work lies either in richer hybrid execution inside the engine or in clearer composition with graph, lakehouse, and metadata systems around it.

A final recurring limitation is documentary rather than algorithmic: many application papers are explicit about using Milvus but sparse about collection schemas, shard layouts, scalar filtering, or search-time hyperparameters. KGSR-ADS explicitly notes that it lacks a “deep dive on vector database (milvus)’s optimization,” and several multimodal systems omit the Milvus collection-level details needed for exact reproduction [2601.00833] [2605.16120]. An important practical implication is that Milvus is often the operational core of these systems, but not yet always the best-specified component in the published artifact.

Source: https://www.emergentmind.com/topics/milvus