AdANNS: A Framework for Adaptive Semantic Search (2305.19435v2)

Abstract: Web-scale search systems learn an encoder to embed a given query which is then hooked into an approximate nearest neighbor search (ANNS) pipeline to retrieve similar data points. To accurately capture tail queries and data points, learned representations typically are rigid, high-dimensional vectors that are generally used as-is in the entire ANNS pipeline and can lead to computationally expensive retrieval. In this paper, we argue that instead of rigid representations, different stages of ANNS can leverage adaptive representations of varying capacities to achieve significantly better accuracy-compute trade-offs, i.e., stages of ANNS that can get away with more approximate computation should use a lower-capacity representation of the same data point. To this end, we introduce AdANNS, a novel ANNS design framework that explicitly leverages the flexibility of Matryoshka Representations. We demonstrate state-of-the-art accuracy-compute trade-offs using novel AdANNS-based key ANNS building blocks like search data structures (AdANNS-IVF) and quantization (AdANNS-OPQ). For example on ImageNet retrieval, AdANNS-IVF is up to 1.5% more accurate than the rigid representations-based IVF at the same compute budget; and matches accuracy while being up to 90x faster in wall-clock time. For Natural Questions, 32-byte AdANNS-OPQ matches the accuracy of the 64-byte OPQ baseline constructed using rigid representations -- same accuracy at half the cost! We further show that the gains from AdANNS translate to modern-day composite ANNS indices that combine search structures and quantization. Finally, we demonstrate that AdANNS can enable inference-time adaptivity for compute-aware search on ANNS indices built non-adaptively on matryoshka representations. Code is open-sourced at https://github.com/RAIVNLab/AdANNS.

• The paper introduces the AdANNS framework, which adaptively adjusts representation capacities in the ANNS pipeline to enhance accuracy-compute trade-offs.
• It demonstrates that adaptive IVF configurations can deliver up to 1.5% improved accuracy and 90 times faster processing compared to traditional methods.
• The framework’s efficient quantization approach shows that a 32-byte representation can match the accuracy of a 64-byte standard while reducing computational costs.

An Adaptive Framework for Semantic Search

The paper presents a novel approach to improve the accuracy-compute trade-off in semantic search systems by introducing the AdANNS framework, which leverages Matryoshka Representations (MR) for adaptive approximate nearest neighbor search (ANNS). By utilizing adaptive embeddings, the authors break away from the traditional use of rigid, high-dimensional representations, which are often computationally expensive. The main contribution of this work is the proposition that various stages of the ANNS pipeline can employ representations of varying capacities for enhanced efficiency, thereby optimizing the accuracy-compute trade-offs.

Key Contributions and Findings

1. Dance: Adaptive ANNS Framework:
   • The authors propose AdANNS, a framework utilizing Matryoshka Representations to improve search data structures and quantization methods, achieving better accuracy-compute trade-offs than existing solutions.
2. Advancements in ANNS Building Blocks:
   • AdANNS is implemented in two primary components: the search data structure (IVF) and distance computation method. The framework proposes {-IVF, which improves the traditional inverted file index by using adaptive representations. This approach was shown to be up to 1.5% more accurate than standard rigid representation-based methods and up to 90 times faster.
3. Efficient Quantization Techniques:
   • AdANNS introduces {-OPQ, which significantly outperforms the baseline OPQ by utilizing adaptive embedding dimensions. The paper demonstrates that a 32-byte {-OPQ can achieve the accuracy of a 64-byte OPQ while being twice as fast.
4. Generalization to Modern Composite Indices:
   • The authors extend AdANNS to composite indices such as IVFOPQ, showing that the approach not only delivers better results but also significantly reduces computational costs. Specifically, the integration of AdANNS with DiskANN provided similar accuracy at half the cost.
5. Empirical Validation:
   • Through extensive experimentation on datasets such as ImageNet-1K and Natural Questions, the authors demonstrate that adaptive Matryoshka Representations provide better trade-offs over rigid counterparts in both clustering and quantization tasks.

Implications and Future Directions

The paper significantly advances the ANNS field by demonstrating how learned representations can be adaptively leveraged at different stages of the pipeline to achieve optimal performance. The results suggest that adaptive representations are better aligned for search tasks than traditional rigid embeddings, offering a new avenue for developing more efficient search algorithms. Future work could explore further the integration of adaptive features in different types of ANNS indices and test scalability to even larger datasets and real-world applications.

By setting a new standard for the adoption of adaptive methods in semantic search, this paper paves the way for more efficient and cost-effective search systems in practice. The combination of theoretical insights with practical validation marks a noticeable shift towards more dynamic and flexible retrieval architectures that respond to varying resource constraints, ultimately reflecting broader trends in AI system design that prioritize adaptability and efficiency.