Hierarchical NSW: Efficient ANN Graph

• Hierarchical NSW is a layered graph-based data structure that efficiently supports approximate nearest neighbor search in high-dimensional spaces.
• It employs a greedy, best-first search and layered descent strategy to balance recall, latency, and memory usage in billion-scale applications.
• Widely integrated into libraries like Faiss and Milvus, HNSW underpins applications such as image retrieval, document search, and real-time analytics.

Hierarchical Navigable Small World (HNSW) graphs are a state-of-the-art data structure for efficient approximate nearest neighbor (ANN) search in high-dimensional spaces. HNSW combines a multi-layer graph approach with small-world connectivity to achieve logarithmic search times, robust recall, and efficient memory usage. It is a core algorithm found in leading open-source ANN libraries, supporting billion-scale vector retrieval applications.

1. Overview and Definition

HNSW constructs a hierarchy of proximity graphs, each corresponding to a different level in the structure. Let $\mathcal{X} = \{\mathbf{x}_i\}_{i=1}^{N}$ denote a dataset in $\mathbb{R}^D$. The HNSW structure connects points into a layered graph $G_l$, where $l$ indexes levels $0,\dots,L_{\mathrm{max}}$. The top layer contains fewer nodes with long-range links; lower layers are denser with local connections. Each layer is a variant of a small-world graph, featuring both short-range (local) links and random long-range (express) links, enabling rapid graph traversal.

Insertion starts from the top layer and greedily navigates using an ef_search parameter (number of candidates), progressively descending levels and updating neighbor lists using proximity heuristics. At query time, the algorithm begins at the top layer, employing a greedy best-first search and backtracking to lower layers, ultimately yielding a set of candidate nearest neighbors in logarithmic time.

2. Key Algorithmic Principles

HNSW is built on two foundational elements:

• Hierarchical Graph Construction: Each data point $\mathbf{x}_i$ is assigned a level $l_i \sim \mathrm{Geometric}(p)$, then inserted only into layers $0,\dots,l_i$. At insertion, ef_construction controls width of candidate lists, ensuring high probability of proximity links.
• Small World Connectivity: At each layer, each vertex maintains $M$ neighbors—chosen not only as the closest but also to maximize the diversity and reachability, as in Navigable Small Worlds (NSW). This balances search efficiency and accuracy.

The search process is guided by multiple invariants:

• Local navigation is guaranteed by dense neighborhood links ($M$ per node),
• Global connectivity is ensured by random long-range links at higher layers,
• Layered descent focuses search in increasingly refined subgraphs.

3. Construction and Complexity

Let $N$ be the number of data points, $D$ the dimensionality, $M$ the number of neighbors per node, and $L_{\mathrm{max}}$ the maximum layer. Constructing the HNSW involves the following main steps:

1. Assign Levels: Each point $\mathbf{x}_i$ gets a level $l_i$ following $\mathrm{Geometric}(p)$, so the expected level depth decays exponentially.
2. Insertion:

For each level $l = L_{\mathrm{max}}, \dots, 0$: - Perform best-first search from an entry point, updating candidate lists with ef_construction width, - Establish bidirectional links to $M$ nearest candidates considering both distances and link diversity, - Move down a layer, repeat process.

1. Memory and Computational Complexity:
   • Construction is $O(N \cdot D \cdot M \cdot \log N)$ in practice.
   • Each search requires $O(\log N)$ steps (due to hierarchical descent), each step costing $O(MD)$ for distance computation to candidate nodes.

These parameters are tunable, balancing recall, latency, and memory cost. Increasing $M$ and ef_construction improves recall at higher RAM and construction costs.

4. Search Procedure and Query Efficiency

A query $\mathbf{q} \in \mathbb{R}^{D}$ is executed as follows:

1. Start at the Top Layer: Greedy search selects entry point with minimal distance to $\mathbf{q}$.
2. Layer-wise Descent: At each layer, ef_search neighbors are considered. If no closer neighbor is found, descend a layer; continue refinement.
3. Candidate Extraction: At base layer, heap of candidates is maintained, rooted in best-first traversal. The process terminates after ef_search candidates have been reached, yielding the final ANN set.

This procedure typically achieves recall $\ge$ 0.95 on standard benchmarks at runtimes $O(\log N)$ per query for billion-scale datasets. Empirically, HNSW search outperforms other graph-based and tree-based ANN approaches in both accuracy and query latency for large vector databases.

5. Relation to PQ-based Graph Search

HNSW serves as a versatile, general-purpose graph backbone that can be integrated with various vector compression schemes, most notably Product Quantization (PQ) and its variants:

• Integration in Large-scale ANN Systems: HNSW is routinely used for coarse (coarse-grained) search, providing rapid candidate generation; fine search is implemented via Product Quantization codebooks (e.g., PQ, OPQ, RPQ) for distance evaluations at scale (Matsui et al., 2022).
• Acceleration via SIMD PQ: When combined with low-bit PQ (e.g., 4-bit PQ), HNSW enables sub-millisecond query times on billion-scale databases and commodity ARM hardware (Matsui et al., 2022). Candidates from HNSW are reranked using precomputed asymmetric distances with PQ lookups, avoiding reconstruction.
• Memory-efficient Search: HNSW augments PQ by hosting only compressed codes (bytes per vector) and graph links, remaining feasible for datasets where full vectors are prohibitively large.

Example: In search pipelines such as IVF-HNSW-PQ (IVF=inverted index, HNSW for graph routing, PQ for code compression), candidates are retrieved by HNSW traversal and scored via PQ-table ADC, combining navigable graph and compressed table lookups (Matsui et al., 2022).

6. Applications and Practical Considerations

HNSW is employed in:

• Billion-scale image retrieval, large-scale document embedding search, recommender systems, speech ASR embedding matching, and graph-based similarity joins.
• Commercial libraries (e.g., Faiss, Milvus, NMSLIB) use HNSW as the backbone for hardware-accelerated, distributed, and parameter-free ANN service.
• Memory usage: storing $O(MN)$ links ($M \sim 16-64$), plus PQ-compressed codes ($\sim 32-128$ bits per vector) enables 1B vector indices to fit within standard RAM footprints ($\leq$32 GB).

Retrieval parameters must be tuned for operational constraints:

• ef_construction and ef_search: increase recall at cost of more memory and query time.
• $M$ and L_max: control graph sparsity, top-layer coverage, and descent efficiency.

Notably, HNSW’s logarithmic search scaling makes it suitable for online-serving and real-time analytics over rapidly growing or dynamic databases.

7. Future Directions and Extensions

Emerging research extends HNSW through:

• Learned routing and differentiable PQ codebooks, producing graphs whose topology adapts to PQ code distributions and link features (Yue et al., 2023).
• Joint training of subspace rotations (OPQ), residual quantization, or stacked quantization atop HNSW graphs for increased recall and lower quantization loss.
• SIMD and in-memory acceleration: optimizations to leverage vector hardware are essential for ultra-fast lookup and graph traversal (Matsui et al., 2022).
• Distributed and dynamic HNSW: on-the-fly graph updating for streaming, sliding-window search, or federated database architectures.

A plausible implication is that future large-scale search infrastructures will deploy HNSW in conjunction with compressed PQ codes, stacking learned or progressive quantization schemes atop hierarchical graph layers to further reduce latency and optimize recall.

---

References:

• ARM 4-BIT PQ: SIMD-based Acceleration for Approximate Nearest Neighbor Search on ARM (Matsui et al., 2022)
• Routing-Guided Learned Product Quantization for Graph-Based Approximate Nearest Neighbor Search (Yue et al., 2023)