---
title: Milvus HNSW & IVF ANN Indexing
url: https://www.emergentmind.com/topics/milvus-hnsw-ivf
type: topic
---

# Milvus HNSW & IVF ANN Indexing

Milvus HNSW and IVF-Flat refer to two leading approximate nearest neighbor (ANN) vector indexing backends implemented within the Milvus (v2.x) open-source vector database. Both serve as high-throughput, scalable sublinear-time solutions for dense retrieval in large embedding-based search systems. These indexes support a range of text embedding models and chunking strategies. Their comparative behavior, parameterization, and impact on search quality are documented in "Evaluating Embedding Models and Pipeline Optimization for AI Search Quality" [2511.22240].

## 1. Principles and Data Structures

### HNSW (Hierarchical Navigable Small World)

HNSW is a multi-layer proximity graph in which each layer forms a sparse "small-world" graph. Higher layers possess progressively fewer nodes with long-range links, and only the bottom layer contains all data points with their local links. Query-time search begins at the entry point in the top layer and follows a greedy procedure: at each level ℓ, neighbors are traversed to the node closest to the query, descending layer by layer. Upon reaching the lowest layer, a best-first search with a priority queue (of tunable size efSearch) is used to gather approximate K-nearest neighbors.

Key HNSW parameters include:
- $M$: Number of bi-directional links per node (controls index sparsity/accuracy trade-off).
- efConstruction: Size of the dynamic candidate list during index build (effects graph quality vs. build cost).
- efSearch: Candidate list size during query (controls recall/latency trade-off).

### IVF-Flat (Inverted File with Flat Quantization)

IVF-Flat partitions the vector space into $nlist$ coarse Voronoi cells (clusters) using a quantizer. Each vector is assigned to its nearest centroid and stored verbatim in its respective cell. Querying involves assigning a query to its nearest centroids and scanning all vectors in the top-$nprobe$ cells with exact distance computation to identify top-K matches.

Key IVF-Flat parameters:
- nlist: Number of Voronoi partitions.
- nprobe: Number of clusters scanned per query, governing recall and latency.

## 2. Experimental Environment and Embedding Models

Experiments were conducted on Milvus (v2.x), deployed across multi-CPU, high-memory infrastructure within Cisco's internal compute cluster; embedding inference leveraged GPUs (NVIDIA A100/V100) for large-scale models. The evaluation dataset consists of 11,975 manually validated query-chunk pairs, derived from US City Council transcripts, with questions synthesized by an 8×7B Mistral LLM.

Embedding models and their dimensions included:
- all-mpnet-base-v2 (768-d)
- BGE-base-en-v1.5 (768-d)
- BGE-large-en-v1.5 (1024-d)
- GTE-base-en-v1.5 (768-d)
- GTE-large-en-v1.5 (1024-d)
- Qwen3-Embedding-0.6B (1024-d)
- Qwen3-Embedding-4B (2560-d)
- Qwen3-Embedding-8B (4096-d)

Chunking strategies assessed:
1. Fixed 2,000-character length (baseline)
2. Fixed 512-character length (fine-grained)
3. Semantic chunking aligned with discourse boundaries

The pipeline: transcripts are preprocessed, embeddings generated per chunk, then indexed in Milvus for ANN retrieval.

## 3. Evaluation Metrics

Search quality and scalability were evaluated using the following metrics:

- **Top-K Accuracy:** $Top@K = \dfrac{\text{Number of queries with ground-truth chunk in top-}K}{\text{Total number of queries}}$
- **Normalized Discounted Cumulative Gain (NDCG):**
  $$
  DCG@K = \sum_{i=1}^K \frac{2^{rel_i} - 1}{\log_2(i+1)},\quad NDCG@K = \frac{DCG@K}{IDCG@K}
  $$
- **Latency:** Average query response time (ms) for top-K results.
- **Memory Usage:** Total RAM consumed (graph pointers for HNSW; cluster assignments for IVF).

## 4. Comparative Performance Analysis

The quantitative impact of index type, model dimension, and chunking on retrieval accuracy is summarized below for selected scenarios:

| Embedding/Chunking          | Index Type      | Acc@3 | NDCG@3 | Acc@5 | NDCG@5 |
|-----------------------------|-----------------|-------|--------|-------|--------|
| GTE-large, 2K-char chunk    | HNSW            | 0.412 | 0.356  | 0.462 | 0.378  |
| GTE-large, 512-char chunk   | HNSW            | 0.460 | 0.415  | 0.500 | 0.431  |
| GTE-large, 512-char chunk   | IVF-Flat        | 0.427 | 0.387  | 0.472 | 0.414  |
| Qwen3-8B (4096-d), 2K chunk | HNSW            | 0.571 | 0.516  | 0.662 | 0.583  |

Key findings:
- Reducing chunk size from 2,000 to 512 characters increases Acc@3 by approximately 4.8 percentage points (pp) under HNSW.
- On 512-character chunks, HNSW outperforms IVF-Flat by ~3.3pp in Acc@3.
- IVF-Flat typically has slightly lower recall but consumes less memory and exhibits higher latency per candidate scan.
- For Qwen3-8B (4096-d), HNSW achieves the highest observed accuracy (Acc@3 = 0.571); IVF-Flat performance for this setting is unreported but, by analogy, is expected to trail by 3–5pp.

*This suggests that for given compute resources and memory budgets, HNSW offers the best recall/accuracy, but IVF-Flat may suit extreme scale or memory-constrained environments.*

## 5. Tuning and Optimization Strategies

Parameter tuning recommendations from empirical results:

### HNSW
- $M$: Increasing from 32 to 48 provides minor recall gains, with ~50% increase in graph edge count.
- efConstruction: Set $\gg M$ (128–256) to enhance graph quality at a linear increase in build time.
- efSearch: $\sim 4 \times K$ (e.g., 128 for $K\leq10$) yields $>95\%$ recall; further increases show diminishing returns.

### IVF-Flat
- nlist: Optimal scaling is $nlist \approx \sqrt{N}$ (e.g., $N \approx 1$M yields $nlist \approx 1000$).
- nprobe: nprobe = 8 reaches 90–95% recall; nprobe = 16–32 recovers most of HNSW recall at roughly linear latency cost.

### Chunking and Embedding Dimension
- Finer-grained fixed chunks (512 characters) increase Top-K accuracy by $\sim5$pp; semantic chunking achieves similar gains and retains discourse coherence.
- Higher-dimensional embeddings (e.g., Qwen3-8B, 4096-d) deliver better accuracy but incur $\sim4\times$ storage and $2$–$3\times$ higher query latency compared to 1024-d models.

### Neural Re-ranking
- Two-stage pipelines (GTE-large retrieval, BGE cross-encoder re-ranking) boost Acc@3 from 0.412 to 0.506 for 2,000-character chunks (+9.4pp).
- Reranking bridges much of the accuracy gap for legacy or lower-grade indexes without requiring re-indexing.

## 6. Best Practices and Use-Case Fit

Principal recommendations, as established by the experimental results:

1. Where resources permit, HNSW with $M=32$–$48$, efConstruction $\geq$128, efSearch $=128$–$256$ is favored.
2. For massive-scale or memory-limited deployments, IVF-Flat ($nlist \approx \sqrt{N}$, $nprobe$ tuned to SLA) is advantageous.
3. Fixed 512-character or semantic chunking is preferred to reduce retrieval noise.
4. Lightweight cross-encoder reranking maximizes Top-K precision with minimal extra latency.

* A plausible implication is that two-stage retrieval architectures (approximate search plus neural reranking) represent a robust configuration for text-centric vector search pipelines at scale [2511.22240].

Source: https://www.emergentmind.com/topics/milvus-hnsw-ivf