Pairing Clustered Inverted Indexes with kNN Graphs for Fast Approximate Retrieval over Learned Sparse Representations (2408.04443v1)

Abstract: Learned sparse representations form an effective and interpretable class of embeddings for text retrieval. While exact top-k retrieval over such embeddings faces efficiency challenges, a recent algorithm called Seismic has enabled remarkably fast, highly-accurate approximate retrieval. Seismic statically prunes inverted lists, organizes each list into geometrically-cohesive blocks, and augments each block with a summary vector. At query time, each inverted list associated with a query term is traversed one block at a time in an arbitrary order, with the inner product between the query and summaries determining if a block must be evaluated. When a block is deemed promising, its documents are fully evaluated with a forward index. Seismic is one to two orders of magnitude faster than state-of-the-art inverted index-based solutions and significantly outperforms the winning graph-based submissions to the BigANN 2023 Challenge. In this work, we speed up Seismic further by introducing two innovations to its query processing subroutine. First, we traverse blocks in order of importance, rather than arbitrarily. Second, we take the list of documents retrieved by Seismic and expand it to include the neighbors of each document using an offline k-regular nearest neighbor graph; the expanded list is then ranked to produce the final top-k set. Experiments on two public datasets show that our extension, named SeismicWave, can reach almost-exact accuracy levels and is up to 2.2x faster than Seismic.

• The paper introduces SeismicWave, which optimizes retrieval by pairing ordered block traversal with kNN graphs to improve approximate maximum inner product search over learned sparse representations.
• It uses a heuristic that prioritizes high-scoring document blocks, reducing computational waste and cutting query latency.
• Experiments on Ms Marco and Natural Questions reveal up to a 2.2× speedup with maintained or improved accuracy at fixed memory budgets.

Pairing Clustered Inverted Indexes with $\kappa$-NN Graphs for Fast Approximate Retrieval over Learned Sparse Representations

The paper "Pairing Clustered Inverted Indexes with $\kappa$-NN Graphs for Fast Approximate Retrieval over Learned Sparse Representations" authored by Sebastian Bruch, Franco Maria Nardini, Cosimo Rulli, and Rossano Venturini presents an enhancement to the Seismic algorithm to further increase the efficiency and accuracy of approximate Maximum Inner Product Search (MIPS) over learned sparse representations (LSR).

Introduction

Learned sparse retrieval (LSR) involves encoding inputs into sparse embeddings. These embeddings map to terms in a dictionary, with non-zero coordinates indicating semantic relevance. The goal of retrieval here is MIPS, which entails finding the top-$k$ vectors maximizing the inner product with a query vector. While LSR offers interpretability and generalization advantages over dense retrieval methods, it faces computational challenges in MIPS tasks, a gap that the Seismic algorithm addresses by leveraging inverted lists and forward indexes to achieve significant speed-ups over previous state-of-the-art solutions.

Methodology

This work introduces two key innovations to the Seismic algorithm, collectively termed SeismicWave, to further enhance retrieval speed and accuracy:

1. Ordered Block Traversal: Seismic originally processes blocks within an inverted list in an arbitrary order. SeismicWave improves upon this by sorting blocks based on their inner product with the query before evaluation. This heuristic leverages the empirical observation that querying a significant portion of retrieval time is spent on initial lists, allowing SeismicWave to effectively prioritize more promising blocks early in the query process. This ordering reduces the likelihood of evaluating less relevant blocks, thereby saving computational resources and reducing query latency.
2. $\kappa$-Regular Nearest Neighbor ($\kappa$-NN) Graph: SeismicWave also introduces a $\kappa$-NN graph to augment the retrieval set. After obtaining a preliminary set of top-$k$ documents via Seismic's procedure, SeismicWave expands this set by including $\kappa$ nearest neighbors for each document in the initial set. This exploits the clustering hypothesis, positing that closely related documents are often collectively relevant to similar queries. The expanded candidate set is then re-ranked to produce the final top-$k$ results, incorporating global structural information to enhance accuracy without incurring substantial additional computational costs.

Experimental Results

Experiments are conducted on two datasets: Ms Marco and Natural Questions (NQ), using Splade embeddings. Findings indicate that SeismicWave significantly improves on Seismic across multiple metrics. Specifically, at a fixed memory budget, SeismicWave achieves up to $2.2\times$ speedup with comparable or superior accuracy. Substantial reductions in latency are observed for both memory budget setups—$1.5\times$ and $2\times$ the forward index size—demonstrating the robustness and efficiency of the enhancements.

Implications

The contributions of SeismicWave have several implications for the future of approximate nearest neighbor search in LSR:

• Algorithmic Efficiency: The ordered block traversal in SeismicWave demonstrates the utility of incorporating heuristics based on empirical observations to optimize traversal strategies in retrieval algorithms.
• Leveraging Structural Information: The augmentation with $\kappa$-NN graphs underscores the potential gains from integrating global structural information into sparse retrieval frameworks, enhancing the balance between speed and accuracy.
• Extensions to Other Domains: While this work focuses on text retrieval over sparse representations, its methodologies can inspire similar enhancements in other domains where MIPS is crucial, such as recommendation systems and multimedia search.

Future Directions

Potential future work includes examining the impact of various compression techniques on the performance of SeismicWave, exploring dynamic adaptation of $\kappa$ in $\kappa$-NN graphs based on query characteristics, and applying the methodology to other domains with dense or hybrid sparse-dense representations. Additionally, further refinement of block traversal ordering heuristics could be studied to dynamically adjust based on query-specific patterns.

Conclusion

This paper provides a meaningful advancement in the approximate retrieval over learned sparse representations by effectively pairing clustered inverted indexes with $\kappa$-NN graphs. SeismicWave achieves notable improvements in efficiency and accuracy, setting a new benchmark for retrieval methodologies in LSR. These contributions underline the importance of integrating empirical heuristics and global structural information to enhance retrieval algorithms.