PipeANN-Filter: An Efficient Filtered Vector Search System on SSD

Abstract: We propose PipeANN-Filter, an efficient filtered vector search system on SSD. Unlike existing systems that explore only valid vectors (i.e., those satisfying the attribute constraints) during search, PipeANN-Filter explores a superset of valid vectors, and performs attribute verification after getting the top-k closest result vectors. This allows PipeANN-Filter to leverage probabilistic data structures (e.g., Bloom filters) to identify the superset, trading off a small number of false-positive vector explorations for a massive reduction in SSD I/O for attribute reading. Evaluations show that PipeANN-Filter improves search latency and throughput compared to state-of-the-art systems. PipeANN-Filter is open-source at https://github.com/thustorage/PipeANN

• The paper introduces speculative filtering to overcome I/O bottlenecks by deferring attribute verification using probabilistic data structures.
• It details a hybrid filtering mechanism that integrates Bloom filters, compressed value buckets, and dynamic cost models to balance SSD I/O and filtering accuracy.
• Empirical results demonstrate up to 1.91× throughput improvement and 89.5% latency reduction, showcasing its scalability on billion-scale datasets.

PipeANN-Filter: An Efficient Filtered Vector Search System on SSD

Motivation and Core Contributions

PipeANN-Filter addresses the significant challenge of executing filtered approximate nearest neighbor search (ANNS) over high-dimensional vector datasets with rich attribute annotations stored on SSD. Existing systems for filtered ANNS suffer from severe I/O bottlenecks due to "strict filtering"—requiring that every explored vector satisfy the query’s attribute constraints, necessitating constant attribute reads from SSD. This impacts both throughput and latency, especially in large-scale datasets where attributes themselves can exceed vector storage in total size.

PipeANN-Filter introduces speculative filtering, which allows the search process to explore a superset of valid vectors with attribute verification deferred to the final candidate re-ranking phase. This paradigm enables the use of memory-efficient probabilistic data structures (e.g., Bloom filters) for rapid filtering, with a controlled false-positive rate. The speculative filtering framework creates a new trade-off: it tolerates some false-positive explorations to vastly reduce SSD I/O for attribute accesses. This not only enhances system performance but empirically improves search accuracy by increasing graph connectivity—false positives serve as bridge nodes, mitigating fragmentation induced by low selectivity filtering.

System Design and Implementation

PipeANN-Filter orchestrates several innovative architectural techniques:

• Speculative Filtering Model: The system defines an approximate attribute filter, is_member_approx, which returns true for vectors possibly satisfying constraints, guaranteeing no false negatives but allowing false positives. Exact verification (is_member) is performed only after the approximate search yields top candidates.
• Probabilistic Data Structures: Per-vector Bloom filters and compressed value buckets (for range predicates) enable efficient in-memory approximate filtering. For label constraints, inverted indexes are employed on SSD, with rare labels pre-scanned and frequent labels checked via Bloom filters. For range constraints, attributes are quantized and summarized using fine-grained quantiles.
• Hybrid Filter Mechanism Selection: PipeANN-Filter utilizes a query cost estimation model that considers selectivity and false-positive rate, balancing compute and SSD I/O overhead. It dynamically selects among speculative pre-filtering, in-filtering, and post-filtering for each query workload.
• Graph Layout Augmentation: The on-SSD graph index stores not only direct neighbors but also a sampled subset of 2-hop neighbors for each vector. This densifies the graph and increases connectivity, ensuring speculative in-filtering does not suffer from disconnected subgraphs even under low selectivity constraints.
• Flexible User Interface: Attribute constraints are expressed as Selector objects, supporting arbitrary attribute types, Boolean combinations (AND/OR), batching, and user-defined extensions.

  Figure 1: PipeANN-Filter system overview, showing the hybrid graph/attribute index, in-memory probabilistic filters, and dynamic filter selection based on query workload.

Theoretical and Practical Implications

Speculative filtering fundamentally relaxes the requirement for strict online filtering during graph traversal, which allows integration of probabilistic membership testing. This design offers several advantages:

• I/O Efficiency: By avoiding frequent SSD reads for attributes, the system achieves drastically higher throughput and lower latency compared to both pre-filtering and post-filtering approaches and surpasses dedicated systems like Filtered-DiskANN.
• Recall Improvement: False-positive exploration bridges disconnected subgraphs caused by low selectivity filtering, empirically raising recall even over systems with higher out-degree customized graphs.
• Generalizability: Unlike attribute-aware index approaches, speculative filtering is attribute-agnostic and supports arbitrary constraint types, handling hybrid queries and complex Boolean expressions efficiently.
• Scalability: PipeANN-Filter’s approach scales efficiently to billion-scale datasets, with memory overheads for probabilistic filters remaining manageable (3.5–28.9% of attribute index size for labels, 12.5% for range filtering).

Empirical Results

PipeANN-Filter demonstrates strong improvements across diverse datasets and workloads:

• Throughput and Latency: On million-scale datasets (YT5M, YFCC10M), PipeANN-Filter achieves up to 1.91$\times$ and 1.71$\times$ higher throughput than PipeANN-BaseFilter (a competitive baseline), and 32.3$\times$ higher throughput over Milvus (state-of-the-art pre-filtering vector DB). At 0.9 recall, it reduces latency by up to 89.5%.
• Large-scale Performance: On 100M-scale LAION100M, PipeANN-Filter outperforms Filtered-DiskANN (strict in-filtering) by 4.35$\times$ in throughput and achieves 77.6% lower latency, also attaining higher peak recall.
• False-positive Rate and Memory: Speculative in-filtering’s average false-positive rate is 23.6% (median 17.1%), with memory footprint for the filters strictly controlled. Even under extreme false-positive rates, performance superiority is maintained.

  Figure 2: Search throughput comparison on YT5M and YFCC10M, with PipeANN-Filter outperforming Milvus and PipeANN-BaseFilter across recall targets.

  

  Figure 3: Search latency across recall targets on YT5M and YFCC10M, where PipeANN-Filter demonstrates superior scaling compared to baseline systems.

  

  Figure 4: Search latency and throughput for single-label filtering on LAION100M, highlighting PipeANN-Filter’s advantage over strict in-filtering and pre-filtering baselines.

Analysis of Speculative Filtering Mechanisms

PipeANN-Filter’s cost estimation for mechanism selection integrates selectivity scaling and precision scaling, trading the size of candidate pools with expected valid results. The hybrid approach to probabilistic data structures delivers a fine balance between SSD I/O and filtering accuracy, with early termination strategies mitigating overestimation in cost models. Memory usage remains proportional to the dataset size and attributes, supporting scalability without sacrificing performance.

Empirically, speculative filtering on attribute-rich datasets enables consistently higher recall than strict filtering-only approaches, as explored bridge edges in the graph facilitate deeper traversal and prevent local optima entrapment.

Figure 5: Visual comparison of speculative filtering (approximate filtering superset + exact verification) versus strict filtering mechanisms (pre-, post-, in-filtering).

Related Work and Open Problems

PipeANN-Filter advances disk-resident ANNS beyond traditional graph-based and cluster-based vector search, circumventing the limitations inherent in pure attribute-aware indexes and strict attribute filtering. The system leverages design philosophies from classic storage systems (e.g., probabilistic filters for disk access reduction), showing their applicability to modern, large-scale vector search.

An open problem remains in further optimizing cost estimation—designing models adaptive to data clustering and early search termination effects in the presence of non-uniform attribute distributions. The integration of more advanced probabilistic structures (e.g., Cuckoo Filter, SuRF [sigmod18surf]) for range queries and deletions and hardware acceleration (e.g., SmartSSD, GPU offloading) presents promising avenues for future research.

Figure 6: I/O estimation accuracy for speculative in-filtering across workloads, showing robust scalability with some overestimation due to early termination.

Conclusion

PipeANN-Filter delivers a robust, high-performance solution for attribute-filtered ANNS at scale on SSD. By exploiting speculative filtering and memory-efficient probabilistic filters, the system achieves significantly better throughput, latency, and recall compared to state-of-the-art strict-filtering systems. Theoretical and empirical analysis confirms the superiority of speculative filtering as a generalizable paradigm, with the potential for broader impact in the design of future retrieval, recommendation, and search systems over large, attribute-rich vector datasets.

(2605.17992)