BM25S: Accelerated Sparse BM25 Retrieval

• BM25S is a Python-based information retrieval framework that precomputes term-document BM25 scores to enable rapid, vectorized query ranking.
• It leverages sparse matrix storage and BLAS-accelerated operations to achieve up to 500× speedups over conventional BM25 implementations.
• BM25S supports differential scoring for BM25 variants, ensuring accurate retrieval while maintaining memory efficiency during indexing and query-time aggregation.

BM25S (“BM25 via eager sparse scoring”) is a Python-based information retrieval framework that accelerates BM25 and its variants by precomputing all possible term-document BM25 scores at index time and storing them in a sparse matrix, thereby enabling orders-of-magnitude faster query-time ranked retrieval compared to conventional implementations. BM25S relies on NumPy, SciPy, and (optionally) libstemmer, and achieves up to 500× speedups over popular Python toolkits, while outperforming highly optimized Java-based solutions such as Elasticsearch by factors of 2–10× on many standard datasets. BM25S also generalizes to “non-occurrence” BM25 variants (such as BM25+ and BM25L) via a differential-score shifting method that preserves sparse storage and exact accuracy (Lù, 2024).

1. Classical BM25 Scoring

BM25 is a family of lexical ranking functions used in text retrieval. For a document collection $C$ of size $|C|$ and a query $Q = \{q_1, ..., q_{|Q|}\}$, classical BM25 assigns to each document $D \in C$ a score

$B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$

where the term-document score $S(t, D)$ is typically given (following Lucene) as:

$$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$

with the following standard components:

• $\mathrm{TF}(t, D)$: frequency of term $t$ in document $D$
• $|C|$0: number of tokens in $|C|$1
• $|C|$2: average document length
• $|C|$3, $|C|$4: tunable parameters (e.g., $|C|$5, $|C|$6)
• $|C|$7: term document frequency
• $|C|$8

Conventional BM25 implementations recompute $|C|$9 and $Q = \{q_1, ..., q_{|Q|}\}$0 (or look up) at query time and evaluate $Q = \{q_1, ..., q_{|Q|}\}$1 for every $Q = \{q_1, ..., q_{|Q|}\}$2 with $Q = \{q_1, ..., q_{|Q|}\}$3 (via inverted indexes).

2. Eager Sparse Scoring and Matrix Construction

BM25S departs from the traditional inverted-index paradigm by eagerly evaluating each nonzero $Q = \{q_1, ..., q_{|Q|}\}$4 during corpus indexing and storing the results as a sparse term-document matrix $Q = \{q_1, ..., q_{|Q|}\}$5, where $Q = \{q_1, ..., q_{|Q|}\}$6 is the vocabulary:

• Each unique word token (possibly after stemming and stopword removal) is mapped to an integer row index $Q = \{q_1, ..., q_{|Q|}\}$7.
• For document $Q = \{q_1, ..., q_{|Q|}\}$8 (column $Q = \{q_1, ..., q_{|Q|}\}$9), $D \in C$0 for every $D \in C$1.
• Terms not present in a document lead to $D \in C$2 and hence $D \in C$3; such entries are omitted (sparsity).

Matrix $D \in C$4 is stored in CSC (Compressed Sparse Column) format, optimizing for fast access to document-wise sums and efficient slicing by multiple term rows. During querying, for a query $D \in C$5 with corresponding row indices $D \in C$6, BM25S extracts the $D \in C$7 submatrix $D \in C$8. Summing across rows gives the score vector for all documents: $t$3 This operation leverages BLAS-accelerated sparse summation, yielding efficient query-time document ranking.

3. Extension to Non-Occurrence Variants: Differential Scoring

Variants of BM25 (such as BM25+, BM25L, and others) may assign a nonzero score even when $D \in C$9, i.e., when a term does not occur in a document. Let $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$0 denote the “non-occurrence score.” For example, BM25+ uses:

$B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$1

with $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$2 and $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$3.

BM25S defines the “differential score”:

$B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$4

For $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$5, $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$6. Thus, $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$7 is still sparse. The aggregate BM25 score is exactly recovered as:

$B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$8

BM25S stores only $B(Q, D) = \sum_{i=1}^{|Q|} S(q_i, D)$9 in the sparse matrix; a small 1D array of $S(t, D)$0 values (for each term) is maintained, and $S(t, D)$1 is computed once per query and added to the document scores. This approach generalizes to all BM25 variants covered in Kamphuis et al. (2020) without dense storage explosion.

4. Computational Complexity and Memory Analysis

BM25S exhibits the following complexity characteristics:

• Index-time: $S(t, D)$2 arithmetic operations to compute all $S(t, D)$3 and $S(t, D)$4; assembling the CSC sparse matrix with $S(t, D)$5 explicitly stored entries (each entry: 8 bytes for float + two 4-byte indices).
• Query-time: For a query of length $S(t, D)$6 and $S(t, D)$7 documents, extracting $S(t, D)$8 rows and summing $S(t, D)$9 sparse vectors takes $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$0, where $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$1 is the posting list length for $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$2. Top-$$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$3 selection is performed by $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$4 in expected $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$5 time.

Empirical usage shows $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$6 for typical queries, and fast C-based kernels dominate performance. For comparison:

• Naive Python implementations (e.g., Rank-BM25) recompute all terms per query tuple, with $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$7 Python-level operations.
• Java Lucene computes $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$8 at query time for each term–document match. BM25S shifts all per-occurrence computations to indexing, enabling high-throughput query-time ranking via vectorized linear algebra.

5. Empirical Benchmarks

BM25S was evaluated on 14 BEIR zero-shot benchmark datasets (e.g., ArguAna, Climate-FEVER, CQADupStack, DBPedia, FEVER, FiQA, HotpotQA, MS-MARCO, NFCorpus, NaturalQuestions, Quora, SciDocs, SciFact, TREC-COVID, Touche2020) on a single-threaded Intel Xeon (2.2 GHz, 30 GB RAM). Throughput is measured in queries per second (QPS):

Dataset	BM25S QPS	Elasticsearch QPS	BM25-PT QPS	Rank-BM25 QPS	Relative Speedup (BM25S/ES)
ArguAna	574	13.7	110.5	2.0	~287×
NFCorpus	1,196	45.8	256.7	224.7	~26×

On 10 of 14 datasets, BM25S achieves $$S(t, D) = \mathrm{IDF}(t, C) \cdot \frac{\mathrm{TF}(t, D)} {\mathrm{TF}(t, D) + k_1 \left(1 - b + b \frac{|D|}{L_{\mathrm{avg}}} \right)}$$9 speedup over Rank-BM25, peaking at $\mathrm{TF}(t, D)$0 (ArguAna). Against Java Elasticsearch, BM25S achieves $\mathrm{TF}(t, D)$1–$\mathrm{TF}(t, D)$2 higher QPS on most cases. NDCG@10 evaluation shows that adding a Snowball stemmer and English stopword removal can improve effectiveness from 38.4 to 39.7 average, confirming parity or slight superiority versus established toolkits (Lù, 2024).

6. Implementation Details and Recipes

• Tokenization & Vocabulary: Default uses Scikit-Learn’s regex r"(?u)\b\w\w+\b"; (optionally) C-based libstemmer. Each token is mapped to its integer vocabulary index.
• Index Construction:

$t$4

• Querying & Top-k Selection:

$t$5

• Non-occurrence Variants: Store $\mathrm{TF}(t, D)$3 in matrix, with a small $\mathrm{TF}(t, D)$4-length array for base scores; at query, $\mathrm{TF}(t, D)$5 scalar addition retrieves the global offset.
• Optional Accelerations: Employ JAX’s jax.lax.top_k for faster selection or wrap matrix operations in a thread pool for multi-threaded throughput.

7. Limitations and Deployment Considerations

• Index-Time Resource Usage: Each term occurrence is precomputed and stored as a float (vs. integer $\mathrm{TF}(t, D)$6 for classic inverted indexes). Large corpora (e.g., $\mathrm{TF}(t, D)$7 documents, $\mathrm{TF}(t, D)$8 vocabulary) remain sparse, but RAM requirements can reach tens of GB.
• Parameter Fixity: Parameters $\mathrm{TF}(t, D)$9 and $t$0 are fixed at index time. Modifying them requires index rebuilding, unlike Rank-BM25, which supports query-time parameter adjustment.
• Tokenizer Choice: The provided regex+stemmer combination offers a balance of speed and fidelity. Language-specific analyzers may require customization.
• Index Maintenance: BM25S is suited to mostly-static corpora. Document additions/deletions require partial or full reindexing; incremental updates are nontrivial.
• Non-Occurrence Overhead: BM25+ and related variants incur an extra $t$1 per-query addition, which remains negligible relative to main computation.

BM25S “rolls up” all expensive term-document scoring into an index-time matrix, transforming query-time retrieval to a small set of vectorized operations and top-$t$2 selection. Its speed, exactness for BM25 and variants, minimal dependencies, and ease-of-integration make it suitable for both research and production, from server deployments to browser-based Pyodide execution (Lù, 2024).