Align then Train: Efficient Retrieval Adapter Learning

Abstract: Dense retrieval systems increasingly need to handle complex queries. In many realistic settings, users express intent through long instructions or task-specific descriptions, while target documents remain relatively simple and static. This asymmetry creates a retrieval mismatch: understanding queries may require strong reasoning and instruction-following, whereas efficient document indexing favors lightweight encoders. Existing retrieval systems often address this mismatch by directly improving the embedding model, but fine-tuning large embedding models to better follow such instructions is computationally expensive, memory-intensive, and operationally burdensome. To address this challenge, we propose Efficient Retrieval Adapter (ERA), a label-efficient framework that trains retrieval adapters in two stages: self-supervised alignment and supervised adaptation. Inspired by the pre-training and supervised fine-tuning stages of LLMs, ERA first aligns the embedding spaces of a large query embedder and a lightweight document embedder, and then uses limited labeled data to adapt the query-side representation, bridging both the representation gap between embedding models and the semantic gap between complex queries and simple documents without re-indexing the corpus. Experiments on the MAIR benchmark, spanning 126 retrieval tasks across 6 domains, show that ERA improves retrieval in low-label settings, outperforms methods that rely on larger amounts of labeled data, and effectively combines stronger query embedders with weaker document embedders across domains.

• The paper introduces a novel dual-stage ERA framework that efficiently bridges query-document representation gaps by aligning and fine-tuning query embeddings with minimal supervision.
• It employs a lightweight linear adapter with a self-supervised alignment stage followed by supervised adaptation using hard negative sampling, achieving up to 12% nDCG@10 improvement.
• The approach supports asymmetric encoder pairings and enhances scalability in low-resource, complex query environments without necessitating corpus re-indexing.

Efficient Retrieval Adapter Learning for Complex Query-Document Asymmetry

Motivation and Problem Formulation

Dense retrieval systems must increasingly handle queries containing nuanced instructions or task-specific intent, which contrasts with the relatively simple and static nature of target documents. This query-document complexity asymmetry causes a retrieval mismatch: the query side demands strong reasoning and instruction-following capabilities typically associated with large LLM-based embedders, while the document side requires lightweight embedding models for scalability and efficient indexing. Conventional retrieval paradigms treat queries and documents symmetrically—using identical models or adaptation strategies for both—which leads to inefficiencies or prohibitive operational and computational costs, especially as full model fine-tuning for complex queries is expensive and requires substantial labeled supervision.

The Efficient Retrieval Adapter (ERA) framework addresses this challenge by decoupling retrieval adaptation into two stages: a self-supervised alignment stage and a supervised adaptation stage. ERA employs a lightweight linear adapter to transform the query embedding space, thereby bridging both the representation gap across heterogeneous embedders and the semantic gap between complex queries and simple documents, without necessitating corpus re-indexing.

Figure 1: Overview of the Efficient Retrieval Adapter (ERA) framework, which aligns embedding spaces and adapts query representations in two explicit training stages.

ERA Framework and Training Paradigm

Asymmetric Adapter-Based Retrieval

Traditional dense retrieval systems operate with shared embedders for queries and documents, thus optimizing similarity scores in a symmetric embedding space. ERA generalizes this by supporting asymmetric retrieval: queries and documents can be encoded by entirely different models ($E_q$ and $E_d$, respectively). A linear adapter $\mathbf{W}$ transforms the query embedding, allowing for flexible heterogeneous encoder pairs, with the similarity computation $\text{sim}(E_q(q)\mathbf{W}, E_d(d))$ bridging the representation gap.

ERA requires no parameter access to base embedders and is label-efficient, functioning effectively with fewer than 100 labeled query-document pairs per task—substantially reducing annotation and computational overhead compared with extant baselines.

Self-supervised Alignment Stage

The alignment stage leverages unlabeled documents, projecting both query and document embedders onto identical inputs and optimizing $\text{sim}(E_q(d)\mathbf{W}, E_d(d))$ via cosine similarity. This step exploits transferable geometric structure among embedding spaces established in prior literature, reducing cross-model disparity and positioning the adapter for efficient downstream supervised adaptation.

Supervised Adaptation Stage

A subsequent supervised stage fine-tunes $\mathbf{W}$ using limited labeled pairs, optimizing contrastive objectives (e.g., InfoNCE, triplet loss) to further bridge semantic gaps. The approach employs hard negative sampling (TopK-PercPos), which selects challenging distractors based on post-alignment retrieval scores, outperforming naive top-k and random sampling strategies and offering robust performance regardless of negative sample count.

Figure 2: nDCG@10 comparison between zero-shot, ERA trained on all domains, and ERA trained on all domains except the target domain, demonstrating domain generality and out-of-domain robustness.

Experimental Analysis

Main Results and Robustness

ERA is evaluated on the MAIR benchmark (126 tasks, six domains), with five embedders spanning LLM and lightweight models. ERA consistently outperforms both zero-shot retrieval and adapter-based baselines across all domains and training label ratios. Notably, ERA delivers up to 12% nDCG@10 improvement without corpus re-indexing, with gains magnified for smaller embedders. Embedding Adapter baselines show significant performance degradation in low-resource settings, reaffirming ERA's strong label efficiency.

Figure 3: Performance profile of Qwen3-8B as query embedder versus Qwen3-0.6B as document embedder, illustrating ERA’s ability to facilitate asymmetric retrieval.

Asymmetric Retrieval and Cross-Model Pairings

ERA enables effective coupling of strong query embedders (e.g., Qwen3-8B) to weaker document embedders (e.g., BGE-M3, OpenAI-small), achieving superior retrieval accuracy and flexibility for real-world deployments. Asymmetry is essential for scenarios where queries require high-level reasoning, but indexing must remain lightweight. Extensive experiments confirm that ERA generalizes across different model families and scales naturally to diverse retrieval contexts.

Label Efficiency and Domain Generality

ERA's alignment stage drives label efficiency, allowing substantial performance gains with minimal supervision—performance remains robust with only 5% labeled queries (~500 examples) and is not sensitive to hyperparameter choice or negative sample count. Furthermore, a single general-domain adapter performs comparably with domain-specific alternatives, supporting practical large-scale deployment without domain-specific adaptation overhead.

Figure 4: nDCG@10 degradation at low train ratios, showing ERA’s superiority in label efficiency across asymmetric embedder pairings.

Figure 5: Robustness analysis for different negative sampling strategies, with TopK-PercPos outperforming naive and random sampling across train ratios.

Figure 6: ERA’s performance stability with varying numbers of negative samples, confirming negligible sensitivity.

Implications and Future Perspectives

ERA establishes a scalable, label-efficient paradigm for complex retrieval adaptation, enabling asymmetric retrieval pipelines in resource-constrained environments. The dual-stage approach—self-supervised alignment followed by lightweight supervised adaptation—systematically bridges both cross-model representation gaps and query-document semantic mismatches without base model updates or corpus re-indexing.

Practically, ERA facilitates flexible system architectures (e.g., coupling advanced instruction-following query encoders to static lightweight document indices), reducing operational complexity and cost. Theoretically, it validates transferability of cross-model embedding geometry for retrieval adaptation, posing opportunities for continued exploration into more expressive adapter designs, multimodal asymmetry, and further reduction in supervision via unsupervised or weakly-supervised methods.

Conclusion

ERA is an efficient retrieval architecture combining self-supervised alignment and supervised adaptation via lightweight linear adapters, systematically mitigating query-document complexity asymmetry and delivering strong performance with minimal labeled data. It achieves robust domain generality and supports heterogeneous encoder pairings without corpus re-indexing, thereby defining a flexible approach for modern retrieval systems and motivating future research in scalable, generalizable retrieval adaptation strategies.