An Encoding--Searching Separation Perspective on Bi-Encoder Neural Search

Abstract: This paper reviews, analyzes, and proposes a new perspective on the bi-encoder architecture for neural search. While the bi-encoder architecture is widely used due to its simplicity and scalability at test time, it has some notable issues such as low performance on seen datasets and weak zero-shot performance on new datasets. In this paper, we analyze these issues and summarize two main critiques: the encoding information bottleneck problem and limitations of the basic assumption of embedding search. We then construct a thought experiment to logically analyze the encoding and searching operations and challenge the basic assumption of embedding search. Building on these observations, we propose a new perspective on the bi-encoder architecture called the \textit{encoding--searching separation} perspective, which conceptually and practically separates the encoding and searching operations. This new perspective is applied to explain the root cause of the identified issues and discuss ways to mitigate the problems. Finally, we discuss the implications of the ideas underlying the new perspective, the design surface that it exposes and the potential research directions arising from it.

• The paper introduces an encoding–searching separation framework that decouples encoding from searching to address neural search inefficiencies.
• It demonstrates that relaxing the encoding-for-search assumption and isolating information bottlenecks can enhance zero-shot and task-specific performance.
• The study proposes modular training strategies, including fixed encoding with trained search operations, to reduce overfitting and improve transfer learning.

An Encoding--Searching Separation Perspective on Bi-Encoder Neural Search

Introduction

This paper discusses a fundamental critique and restructuring of the bi-encoder architecture in the context of neural search. The bi-encoder model, which facilitates search tasks by encoding queries and items separately as embeddings and computing their similarity, suffers from notable limitations. These include reduced performance on seen datasets, suboptimal zero-shot performance, high training costs, and a propensity for overfitting. The document addresses these issues through a novel analytical framework: the encoding--searching separation perspective. This perspective introduces a conceptual and practical separation between encoding and searching operations, aiming for improved efficiency and effectiveness in neural search.

Critiques of the Bi-Encoder Architecture

Encoding Information Bottleneck

The encoding information bottleneck refers to the restrictive capacity of embeddings in the bi-encoder architecture, specifically imposed by the encoding process rather than the embeddings themselves. The paper argues that this bottleneck constrains the model's expressiveness and accuracy. Whereas previous attempts to widen the bottleneck have focused on increasing embedding size, the paper suggests utilizing larger encoders to allow more generic, task-agnostic encoding.

Limitations of Encoding-for-Search Assumption

The foundational assumption in embedding search is that relevance can be estimated by the similarity between embeddings. This implies a direct alignment between encoding and searching operations, necessitating embeddings to encapsulate information directly relevant to search tasks. The paper identifies this assumption as overly restrictive, particularly in multi-modal contexts where such alignment is unnatural.

New Perspective on Bi-Encoder Architecture

Thought Experiment

The paper conducts a thought experiment to analyze the roles of encoding and searching operations separately. It explores various scenarios to understand how search operations can be optimized independent of encoding specifics. Critical insights reveal that encoding-for-search is not always necessary, proposing instead a generic encoding coupled with specific searching operations. This enables more efficient alignment of useful features rather than entire embedding spaces, creating an "encoding gap" for better separation and control.

Encoding--Searching Separation

The encoding--searching separation framework allows freedom in designing the encoding and searching modules, moving beyond the traditional assumptions of bi-encoder architecture. It effectively separates the encoding process from the search task, providing control over information bottlenecks and allowing selective alignment of features pertinent to queries and items.

Discussion

Addressing Identified Issues

The encoding--searching separation perspective elucidates root causes of performance bottlenecks and suggests practical mitigations:

• Better Control Over Information Bottleneck: Localizing bottlenecks solely in the searching operation facilitates efficient task-specific tuning without overfitting.
• Relaxation of Encoding-for-Search Assumption: Creating an encoding gap allows a move away from rigid architectural assumptions and reduces fine-tuning requirements.

Implications and Directions for Future Research

Significant research proposals arising from this perspective include:

• Fixed Encoding, Trained Searching: Exploring the separation further by training only the searching operations on task-agnostic embeddings to enhance efficiency and effectiveness.
• Investigating Information Bottlenecks: Understanding bottleneck positioning and widening through improved architectural and training strategies.
• Designing Advanced Searching Operations: Developing sophisticated searching models beyond simple linear layers to handle complex search tasks.
• Transfer Learning Approaches: Implementing modular designs for smooth transitions between different search tasks, leveraging pre-trained architectures for generalization.

Conclusion

The encoding--searching separation perspective offers a promising reevaluation of bi-encoder architecture, highlighting intrinsic shortcomings and enabling strategic innovations. By distinguishing the roles of encoding and searching operations, the perspective not only addresses existing performance issues but also propels forward-thinking research protocols, opening pathways for enhanced search algorithms and architectures. Through these insights, the paper contributes valuable conceptual and practical advancements to the field of neural search.