BM25S: Orders of magnitude faster lexical search via eager sparse scoring (2407.03618v1)

Abstract: We introduce BM25S, an efficient Python-based implementation of BM25 that only depends on Numpy and Scipy. BM25S achieves up to a 500x speedup compared to the most popular Python-based framework by eagerly computing BM25 scores during indexing and storing them into sparse matrices. It also achieves considerable speedups compared to highly optimized Java-based implementations, which are used by popular commercial products. Finally, BM25S reproduces the exact implementation of five BM25 variants based on Kamphuis et al. (2020) by extending eager scoring to non-sparse variants using a novel score shifting method. The code can be found at https://github.com/xhluca/bm25s

• The paper introduces BM25S, an algorithm that pre-computes lexical scores through eager indexing, achieving up to 500x speedup over traditional BM25 methods.
• It employs efficient sparse matrix representations and optimized tokenization techniques to significantly enhance query performance in Python-based environments.
• BM25S accurately reproduces multiple BM25 variants, offering a robust solution for both large-scale systems and edge-deployed search applications.

Overview of BM25S: Orders of Magnitude Faster Lexical Search via Eager Sparse Scoring

The paper entitled "BM25S: Orders of Magnitude Faster Lexical Search via Eager Sparse Scoring," authored by Xing Han Lù, introduces BM25S, a highly optimized lexical search algorithm. This implementation achieves significant performance improvements over conventional Python-based and Java-based BM25 implementations through a novel approach of eager sparse scoring, while maintaining the ability to reproduce results from five BM25 variants.

Background and Motivation

Traditional sparse lexical search algorithms, such as BM25 and its variants, are widely utilized due to their absence of training requirements, multilingual applicability, and optimized speed, particularly when implemented in Java-based systems like Lucene. However, Python-based implementations often lag in performance. This paper addresses this gap by presenting BM25S, which optimizes scoring through pre-computation and efficient sparse matrix utilization.

Implementation Insights

The core innovation in BM25S lies in its eager calculation of BM25 scores during indexing, resulting in substantial speedups during query execution. This method involves pre-computing potential scores for any query token and storing these in sparse matrices.

Key Features

1. Eager Index-Time Scoring: Pre-computation of term frequency and inverse document frequency values during the indexing phase. This allows for rapid summation operations during query time, sidestepping the need for recalculating scores.
2. Sparse Matrix Representation: Storage of scores in Compressed Sparse Column (CSC) format, enabling efficient slicing and summation operations. The matrix sparsity is leveraged to ensure minimal memory usage while retaining fast access times.
3. Tokenization: The implementation uses a tokenizer derived from Scikit-Learn's text splitting, Elastic's stopword list, and optionally includes a C-based Snowball stemmer. This approach improves both performance and memory efficiency compared to subword tokenizers.
4. Top-k Selection: Utilizes a partition-based approach for selecting the top-k relevant documents, ensuring average O(n) time complexity. This process is further optimized with optional multi-threading capabilities.

Numerical Results

The paper demonstrates the effectiveness of BM25S with benchmarks on the BEIR dataset. Key findings include:

• Throughput: BM25S achieves significantly higher throughput compared to existing implementations. For instance, on the ArguAna dataset, BM25S processes 573.91 queries per second (QPS) versus 2.00 QPS by Rank-BM25, representing a 500x speedup. This pattern of substantial speedups is consistent across multiple datasets.
• Tokenization Impact: The evaluation of different tokenization schemes revealed modest improvements in performance when stemming is applied, while the inclusion or exclusion of stopwords showed minimal overall impact but significant variations in specific cases.
• Variant Comparison: The paper also benchmarks BM25S against other BM25 variants and commercial offerings like Elasticsearch. It finds that while Elasticsearch achieves slightly higher average scores, BM25S remains competitive across various datasets and parameter settings.

Theoretical and Practical Implications

The theoretical contributions of this paper lie in demonstrating the effectiveness of pre-computation (eager scoring) within sparse matrices and extending this approach to various BM25 variants. This method not only accelerates query times but ensures mathematical accuracy across multiple BM25 implementations.

Practically, BM25S offers a significant performance boost for Python-based applications, making it suitable for scenarios requiring fast and efficient lexical searches, such as edge deployments and browser-based applications utilizing WebAssembly frameworks like Pyodide and Pyscript.

Future Directions

The proposed BM25S framework opens several avenues for future research and practical innovations:

• Integration in Large-Scale Systems: Exploring the deployment of BM25S in more extensive, distributed search systems.
• Extensions to Dense Retrieval: Investigating the potential of combining BM25S with dense retrieval models to leverage the benefits of both sparse and dense representations.
• Further Optimization: Continuing to refine the implementation for even faster performance, particularly focusing on memory efficiency and multi-threading.

Conclusion

BM25S presents a significant advancement in the domain of lexical search algorithms by enabling orders of magnitude faster performance in Python-based environments. Through systematic pre-computation and efficient use of sparse matrices, BM25S stands out as a highly optimized and mathematically robust implementation suitable for a wide range of applications. This research reaffirms the importance of algorithmic improvements in enhancing computational efficiency and sets the stage for further innovations in the field of information retrieval.