---
title: Embedding-based Retrieval Fundamentals
url: https://www.emergentmind.com/topics/embedding-based-retrieval
type: topic
---

# Embedding-based Retrieval Fundamentals

Embedding-based retrieval (EBR) refers to retrieval systems that represent queries and items (such as documents, products, or entities) as dense vectors—embeddings—and recast the retrieval problem as a search for nearest neighbors in this embedding space. This approach provides a unified and highly scalable mechanism for a broad spectrum of information retrieval and recommendation tasks, ranging from web, product, and entity search to cross-modal applications. EBR has been widely adopted in large-scale commercial systems due to its effectiveness in bridging the semantic gap between queries and candidates and enabling sub-linear search over massive corpora.

## 1. Foundations: Architecture, Embedding Formation, and Scoring

Contemporary EBR systems almost universally adopt some variant of the dual-encoder (“two-tower”) paradigm, where separate neural towers encode the query and item independently [2006.11632][2202.06212]. These towers may be identically parameterized (as in the Siamese setting) or may differ, especially in cross-modal or multimodal configurations [2507.01066][2510.12014].

Given a user query $q$ and a candidate item $d$, the encoders produce $d$-dimensional vectors $f(q) \in \mathbb{R}^d$ and $g(d) \in \mathbb{R}^d$, and the retrieval score is typically the cosine similarity or dot product:

$$
S(q, d) = \frac{f(q)^\top g(d)}{\|f(q)\|\|g(d)\|}
$$

These representations may be further refined by combining multiple modalities (e.g., text, image, metadata as tokens in a multimodal transformer [2302.11052]), personalization signals (location, user history as in Etsy [2306.04833]), or knowledge graph features [1908.10554].

Item embeddings are pre-computed and indexed using fast Approximate Nearest Neighbor (ANN) libraries such as FAISS, HNSW, or product quantization variants [2006.11632][2408.04887]. At runtime, a query embedding is generated and the index is searched for high-scoring candidates.

## 2. Training Objectives, Loss Functions, and Optimization

EBR systems are predominantly trained using losses that push relevant query–item pairs together and irrelevant pairs apart, with several paradigmatic choices:

- **Contrastive InfoNCE (softmax) loss:** Used by dual encoders in Facebook, Bing, Taobao, and others [2006.11632][2404.05989][2210.09787]. For a batch of $N$ query–positive pairs, and in-batch negatives:
  $$
  L = -\sum_{i=1}^N \log \frac{\exp(S(q_i, d_i^+)/\tau)}{\sum_{j=1}^{N}\exp(S(q_i, d_j^-)/\tau)}
  $$
  with $\tau$ as a learned or fixed temperature.

- **Margin-based losses:** Pairwise hinge or triplet losses are sometimes used, though misalignment between margin loss and global top-K inference can degrade retrieval [2106.09297]. Softmax cross-entropy, matching serving conditions, increases both convergence and recall [2106.09297].

- **Listwise/Ranking and Multi-task objectives:** In high-value cases (ads, e-commerce), objectives may combine knowledge distillation from high-precision teachers, CTR/profitability, and click feedback [2202.06212][2408.04884].

Hard negative sampling (including in-batch, in-device, or ANN-mined negatives) is critical for effective training [2006.11632][2106.09297][2306.04833]. In product and recommendation systems, label noise due to implicit feedback is mitigated with temperature smoothing, careful loss weighting, or explicit human feedback distillation [2106.09297][2408.04884].

## 3. Advanced Variants, Filtering, and Calibration

### 3.1 Precision Filtering and Score Calibration

Dense embedding-based search, unlike lexical matching, can flood downstream ranking with low-relevance or “junk” items due to uncalibrated cosine scores. This is overcome using query-dependent calibration layers (e.g., the Cosine Adapter) that map raw similarities to absolute probabilities, enabling a global threshold $\tau$ to prune irrelevant results while controlling recall [2408.04887]. Similarly, sigmoid score transforms and segment-specific thresholds in social network search balance junk removal and recall [2304.09287].

### 3.2 Probabilistic Thresholding

Frequentist EBR methods using fixed $K$ exhibit over- or under-retrieval as the set of relevant items per query varies; probabilistic approaches (pEBR) instead fit, per-query, a score distribution $p(s \mid q)$ (e.g., ExpNCE, BetaNCE) that enables dynamic thresholds at target quantiles, providing both higher recall for head queries and improved precision for tails [2410.19349].

### 3.3 Multi-embedding and Granularity

For structured or long documents, as in the legal domain, multi-layer EBR generates embeddings at several granularities (e.g., document, section, paragraph, enumeration) and retrieves at the chunk level most semantically aligned with the query [2411.07739]. In web retrieval, multi-embedding frameworks select segments based on click frequencies to match diverse query intents [2210.09787].

## 4. Compression, Efficiency, and Scalability

Industrial-scale EBR requires extreme efficiency in memory and query latency. Binary EBR (BEBR) compresses float32 embeddings into multi-level binary codes using recurrent MLP-based binarization, compatible with existing ANN backends and offering 30–50% memory and 2x latency reductions with minor accuracy loss [2302.08714]. Product quantization and matching-oriented PQ (MoPQ) jointly optimized with the embedding model (as in Bing’s Uni-Retriever) increase recall, enabling billion-scale candidate pools [2202.06212].

Backward-compatible binarization and smooth deployment across embedding versions are supported via embedding-to-embedding training and auxiliary contrastive losses [2302.08714].

## 5. Theoretical Limitations, Expressivity, and Hybrid Models

Single-vector EBR systems are fundamentally limited in the number of retrievable top-$k$ subsets by embedding dimension $d$:
$$
M_{d, k}(N) \leq \sum_{i=0}^{d} \binom{N-1}{i}
$$
With increasing $N$ and small $d$, many possible top-$k$ sets cannot be realized, even for simple $k$ (e.g., $k=2$) [2508.21038]. This is empirically demonstrated by the LIMIT benchmark, where all SOTA single-vector EBR models fail to achieve full recall; only multi-vector or sparse hybrid models like BM25 or ModernColBERT overcome these constraints. This limitation becomes acute for instruction-following, logic-based, or attribute-combinatorial queries.

Hybrid systems, blending keyword and embedding-based retrieval, are deployed at scale (Facebook Group Search), with linear score fusion (weight $\alpha$ tuned via LLM evaluation and A/B) to maximize both precision and diversity [2509.13603]. Multi-vector representations and late-interaction models (e.g., ColBERT) or cross-encoder rerankers are active research areas for breaking the expressivity bottleneck [2508.21038].

## 6. Application-Specific Extensions

EBR frameworks have been extended to multi-task retrieval (per-cluster task adaptation via prefix-tuning [2302.02657]), personalized search (joint query-user encoders [2306.04833]), multimodal content moderation (vision, text fusion via supervised contrastive learning [2507.01066]), zero-shot retrieval via synthetic query generation [2009.10270], and retrieval-augmented generation over complex, hierarchical texts [2411.07739]. In content-based recommendation and image retrieval, embedding distillation from large teacher models (vLLMs) transfers fine-grained alignment into scalable, dual-encoder systems [2510.12014].

## 7. Practical Lessons, Empirical Findings, and Future Directions

Successful deployment hinges on careful negative mining, ever-fresh ANN tuning, score calibration, label and semi-positive mining, typo and query-robust augmentation, and dedicated handling of “integrity” errors (harmful/junky content) [2304.09287][2408.04887][2408.04884]. Human-in-the-loop feedback (as in Walmart and Que2Engage) and explicit multitask losses outperform naïve label or click-based optimization.

Despite the flexibility and scalability of EBR, known shortcomings remain: the single-vector paradigm is provably limited, and high recall on truly compositional or instruction-rich queries is unattainable without multi-vector, reranking, hybrid, or sparse methods [2508.21038][2509.13603]. Current research is focused on relaxing these constraints, integrating more expressive architectures, and further automating calibration and adaptation across domains and modalities.

Source: https://www.emergentmind.com/topics/embedding-based-retrieval