---
title: Comparative Retriever with CMC Framework
url: https://www.emergentmind.com/topics/comparative-retriever-cr
type: topic
---

# Comparative Retriever with CMC Framework

A Comparative Retriever (CR) implemented via the Comparing Multiple Candidates (CMC) framework is an advanced reranking architecture designed to simultaneously improve retrieval recall and top-1 ranking efficiency within classic retrieve-and-rerank paradigms. CMC enables joint, contextualized comparison of a query with multiple candidate embeddings through shallow self-attention, bridging the speed–accuracy gap between bi-encoder (BE) and cross-encoder (CE) methods by directly modeling their intermediate candidate sets with a scalable Transformer. Designed for deployability as either an intermediate reranker or a final-stage scoring mechanism, CMC supports high-throughput settings, outperforming traditional pipelines on a range of retrieval tasks in both accuracy and end-to-end latency [2405.12801].

## 1. Pipeline Integration and Function

Typical retrieval pipelines consist of a fast bi-encoder (BE) for large-scale candidate retrieval, followed by a slower, more expressive cross-encoder (CE) reranker for a limited candidate pool. The CMC layer is introduced between BE and CE:

- **Stage 1: Bi-Encoder Retrieval** The query $q$ and all candidate $c_i$ embeddings are processed with a BE, with Maximum Inner Product Search (MIPS) returning the top-$K$ candidates as $C_q = \{c_{q,1}, \dotsc, c_{q,K}\}$.
- **Stage 2: CMC Reranking** CMC ingests the single query $q$ and all $K$ candidate embeddings, jointly encoding them via shallow self-attention to compute contextualized similarity scores for all $K$ candidates in parallel.
- **Stage 3: Optional Cross-Encoder Final Rerank** The highest-scoring $K'$ ($K' < K$) candidates from CMC may be forwarded to a CE for final strict reranking.

This yields an enhanced pipeline: BE $\rightarrow$ CMC $\rightarrow$ CE, where CMC supplies a "virtually enhanced" retrieval stage, producing improved recall metrics (e.g., $\Delta$R@16$\approx$+4.8%-p, $\Delta$R@64$\approx$+3.5%-p on ZeSHEL) with negligible latency increase ($<$7%) and, as a standalone reranker, is both faster (11$\times$) and more effective on specific downstream tasks (+0.7%-p on Wikipedia EL, +3.3 MRR on DSTC7 dialogue ranking) relative to CE [2405.12801].

## 2. Mathematical Formulation

CMC is composed of several key formal elements:

### 2.1 Sentence-level Encodings

A query $q$ and each candidate $c_{q,j}$ are encoded as follows:
- Query: $\mathbf{x}_q = [{\tt [CLS]}, x_q^{1}, \dotsc, x_q^{m}]$
- Candidate: $\mathbf{x}_{c_{q,j}} = [{\tt [CLS]}, x_{c_{q,j}}^{1}, \dotsc, x_{c_{q,j}}^{n_j}]$
  
With two separate encoders $\mathrm{Enc}_{qry}$ and $\mathrm{Enc}_{can}$, extract single-vector summaries:
$$
\mathbf{h}_q^{\rm sent} = \mathrm{Enc}_{qry}(\mathbf{x}_q)_{[CLS]}, \qquad \mathbf{h}_{c_{q,j}}^{\rm sent} = \mathrm{Enc}_{can}(\mathbf{x}_{c_{q,j}})_{[CLS]}
$$

### 2.2 Multi-Candidate Self-Attention

Stack query and $K$ candidate embeddings:
$$
\mathbf{H}^0 = [ \mathbf{h}_q^{\rm sent};\, \mathbf{h}_{c_{q,1}}^{\rm sent};\, \dotsc;\, \mathbf{h}_{c_{q,K}}^{\rm sent} ] \in \mathbb{R}^{(K+1)\times d}
$$

Process $\mathbf{H}^0$ via $L$ Transformer encoder layers (no positional encoding, $L=2$, $H=12$, $d=768$):
$$
\mathbf{H}^{\rm CMC} = \mathrm{TransformerEncoder}_{L}(\mathbf{H}^0)
$$
with $\mathbf{H}^{\rm CMC} = [ \mathbf{h}_q^{\rm CMC};\, \mathbf{h}^{\rm CMC}_{c_{q,1}}; \dotsc; \mathbf{h}^{\rm CMC}_{c_{q,K}} ]$.

### 2.3 Scoring & Selection

Score each contextualized candidate against the contextualized query:
$$
s_j = (\mathbf{h}_q^{\rm CMC}) \cdot (\mathbf{h}_{c_{q,j}}^{\rm CMC})^\top,\quad 1\leq j\leq K
$$
Select top-1:$\hat c_q = \arg\max_j s_j$; for further reranking, pass top-$K'$ by $s_j$ downstream.

### 2.4 Training Loss

Employ a multi-class cross-entropy over $K$ candidates:
$$
p_j = \frac{\exp(s_j)}{\sum_{i=1}^{K} \exp(s_i)}, \qquad \mathcal{L}_{\rm CE} = -\sum_{j=1}^{K} y_j \log p_j
$$
where $y_j = \mathbf{1}_{[j=g]}$. Optionally, include a KL regularizer to bias toward the BE's retrieval distribution $r_j$:
$$
\mathcal{L} = \lambda_1 \mathcal{L}_{\rm CE} + \lambda_2 \sum_{j=1}^K p_j \log\frac{p_j}{r_j}, \qquad \lambda_1+\lambda_2=1
$$

## 3. Theoretical and Empirical Efficiency

CMC offers a favorable tradeoff between computational complexity and retrieval effectiveness:

| Approach  | Typical Cost per Query      | Latency (Empirical)           |
|:----------|:---------------------------|:------------------------------|
| BE        | $O(\mathrm{MIPS})$         | $570$ ms (K=512, ZeSHEL)      |
| CE        | $K\times O(p^2 d L_{\rm CE})$ | $260$ ms (K=64, ZeSHEL)       |
| CMC       | $O((K+1)^2 d L_{\rm CMC})$   | $37$ ms (K=512, ZeSHEL)       |

- **Complexity:** For moderate $K$ (e.g., $K=64$), $(K+1)^2=4225$ (CMC) versus $p^2=25,600$ for typical CE token lengths ($p=160$).
- **Empirical Throughput:** CMC can process up to $16,000$ candidates in a single batch, limited by GPU memory, unlike CE which scales poorly beyond a few dozen documents.
- **Latency:** On ZeSHEL, BE+CMC ($K=512$) requires $607$ ms ($\approx1.07\times$ BE alone); CMC-filtered 16 + CE takes $160$ ms ($\approx0.6\times$ CE@64).

## 4. Empirical Evaluation and Benchmarks

CMC achieves notable improvements across a spectrum of retrieval and ranking tasks:

### 4.1 ZeSHEL Retrieval (Recall@K)

- BE@64 Recall: 87.95%
- BE+CMC@64 Recall: 91.51% ($\Delta=+3.56$%-p)
- BE+CMC@16 Recall: 86.32% vs. BE@16: 81.52% ($\Delta=+4.80$%-p)

### 4.2 Downstream Accuracy

- **Wikipedia EL (AIDA/MSNBC/WNED-CWEB avg):**
  - CE: 80.2%-p
  - CMC: 80.9%-p ($+0.7$%-p)
- **DSTC7 Dialogue (MRR@10):**
  - CE: 73.2
  - CMC: 76.5 ($+3.3$ MRR)

All improvements are statistically significant at $p<0.01$.

### 4.3 Datasets and Hyperparameters

- **Entity Linking:** AIDA-CoNLL, WNED-CWEB, MSNBC, ZeSHEL ($10$ k–$100$ k entities)
- **Passage Ranking:** MS MARCO ($8.8$ M passages)
- **Dialogue:** DSTC7 track1 ($100$ candidates)

CMC configurations: BERT-base/large encoders (e.g., BLINK/CoCondenser), 2-layer/$12$-head/$768$-dim self-attention, max query&#8209;lengths $32$/$128$, max doc length $128$/$512$; learning rate in $\{1\times10^{-5}, 5\times10^{-6}, 2\times10^{-6}\}$, batch $4$–$8$, epochs $3$–$10$, with hard negatives from BE.

## 5. Engineering and Practical Deployment

For insertion into BE→CE pipelines:
1. **Offline corpus preparation:** Encode all items via $\mathrm{Enc}_{can}$, store $\mathbf{h}_c^{\rm sent}$ for lookup.
2. **Online per-query retrieval:**
    - Encode $q$ using $\mathrm{Enc}_{qry}$.
    - BE retrieves top-$K$ candidate IDs.
    - Retrieve precomputed $\{\mathbf{h}^{\rm sent}_{c_{q,j}}\}$.
    - Concatenate with $\mathbf{h}^{\rm sent}_q$ to form $\mathbf{H}^0$, process through CMC's Transformer.
    - Optionally invoke CE on CMC's top-$K'$ outputs.

Parameter guidance:
- $K$ in $[32,128]$ recommended; CMC robust up to $K=512$.
- 2 self-attention layers, 12 heads, are sufficient.
- Use skip connections per self-attention block for training stability.

## 6. Training and Inference Workflows

**Training pseudocode:**
```python
for each batch of queries {q_i}:
  for each q_i:
    C_i = BE.topK(q_i)                     # indices of hard negs + gold
    h_q = Enc_q(q_i)[CLS]
    H_c = [Enc_c(c)[CLS] for c in C_i]      # precompute offline
    H0 = concat( h_q, H_c )                 # shape=(K+1,d)
    H = TransformerEncoder(H0)              # L=2 layers
    scores = [ dot(H[0], H[j]) for j=1..K ]
    loss = CrossEntropyLoss(scores, gold_idx) \
           + λ KL(softmax(scores) ∥ BE_scores)
  backpropagate & update parameters
```

**Inference pseudocode:**
```python
input: query q
candidates = BE.topK(q)  # e.g., K=512
h_q = Enc_q(q)[CLS]
H_c = lookup( candidates )   # precomputed embeddings
H0 = concat( h_q, H_c )
H = TransformerEncoder(H0)
scores = [ dot(H[0], H[j]) for j=1..K ]
if final rerank:
  topKprime = argsort(scores)[-K'..]
  return CE.rerank(q, topKprime)
else:
  return argsort(scores)  # new top list
```

CMC thus unifies the scalability of BE retrieval with the contextual discrimination of cross-attention, offering a high-throughput comparative retriever suitable for modern large-vocabulary ranking tasks [2405.12801].

Source: https://www.emergentmind.com/topics/comparative-retriever-cr