Cross-Encoder Re-ranking in IR

• Cross-Encoder re-ranking is a retrieval method that jointly encodes queries and documents to capture fine-grained token interactions, significantly enhancing ranking precision.
• It supports various interaction schemes—pointwise, pairwise, and listwise—with architectural variants like MICE and Set-Encoder that optimize computational efficiency and robustness.
• Fine-tuning via contrastive learning and teacher distillation, along with effective hard negative sampling, enables adaptation to diverse retrieval and domain-specific tasks.

Cross-Encoder re-ranking is the standard second-stage retrieval setup in which a query and each candidate document are jointly encoded by a transformer reranker, and the model assigns a relevance score used to reorder the top results returned by a first-stage retriever. Because every query–document pair is encoded jointly, cross-encoders can model fine-grained token-level interactions, but they are computationally expensive and therefore are typically applied only to a candidate set such as the top-$k$ results from BM25, SPLADE, or ColBERTv2 rather than to the full corpus (Déjean et al., 2024, Pezzuti et al., 28 Mar 2025, Litschko et al., 2022). In contemporary retrieval research, they function both as strong practical baselines and as the principal comparison point for LLM-based listwise rerankers, late-interaction models, parameter-efficient multilingual transfer methods, and efficiency-oriented architectural variants (Déjean et al., 2024, Vast et al., 18 Feb 2026).

1. Canonical retrieve–then–rerank formulation

In the standard formulation, a first-stage retriever produces a candidate set

$\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$

for a query $q$, and the cross-encoder jointly encodes $(q,d_i)$ to compute a relevance score $s(q,d_i)$. Ranking is then induced by sorting candidates by that score. A common BERT-style realization is

$$s^{ce} = \mathit{CE}([CLS]\, q \,[SEP]\, d_i \,[SEP]),$$

or, equivalently, a linear scoring layer applied to the contextualized $[CLS]$ representation of the concatenated query–document sequence (Pezzuti et al., 28 Mar 2025, Litschko et al., 2022).

This pointwise scoring regime appears in several settings. In passage retrieval, SPLADE-v3 retrieves an initial ranked list and a cross-encoder reranks only the top $k$ candidates, with $k \in \{50,100,200\}$ in one comparative study; in cross-lingual retrieval, BM25 retrieves the top-100 candidates and reranking is applied only to that set; in zero-shot query-expansion experiments, strong rerankers score BM25’s top-1000 documents (Déjean et al., 2024, Goworek et al., 24 Nov 2025, Li et al., 2023). The common design principle is that joint encoding is reserved for a relatively small candidate pool because cross-encoder scoring is too expensive for exhaustive corpus ranking (Pezzuti et al., 28 Mar 2025).

The same formal pattern generalizes beyond ad hoc passage retrieval. In generation-based dialogue systems using sample-and-rank decoding, the Cross-Encoder baseline jointly encodes the dialogue context and each candidate response, produces scores $f(C,r_1),\ldots,f(C,r_M)$, and selects the highest-scoring response (Song et al., 2021). In bilingual lexicon induction, BLICEr uses a multilingual pretrained LLM fine-tuned in a cross-encoder manner to rerank translation candidates retrieved from a cross-lingual word embedding space, again applying the expensive model only to a small candidate set (Li et al., 2022). This suggests that “re-ranking” in the cross-encoder literature denotes a general computational pattern—fast candidate generation followed by expensive joint scoring—rather than a task-specific procedure.

2. Interaction patterns and architectural variants

The standard pointwise cross-encoder is only one member of a broader family. Existing cross-encoder models can be categorized as pointwise, pairwise, or listwise. Pointwise models score each query–passage pair independently; pairwise and listwise models permit passage interactions, which can improve effectiveness but often reduce efficiency and robustness to input-order permutations (Schlatt et al., 2024). In one comparative study, cross-encoders used for reranking effective SPLADE retrievers remain pointwise and introduce no special aggregation formula beyond scoring top-$\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$0 query–document pairs independently and sorting them by score (Déjean et al., 2024).

Listwise extensions attempt to recover inter-passage information without abandoning joint encoding. The Set-Encoder processes one sequence per passage, $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$1, with positional encodings restarted from zero for each passage, and enables inter-passage attention by allowing a sequence to attend to the $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$2 tokens of the other passages. This yields a permutation-invariant listwise cross-encoder that is as effective as state-of-the-art listwise models on TREC Deep Learning and TIREx while being more efficient and invariant to passage-order permutations (Schlatt et al., 2024).

A different line of work begins from the internal interaction structure of standard cross-encoders and removes attention pathways judged unnecessary or detrimental. MICE (“Minimal Interaction Cross-Encoders”) masks selected transfers involving $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$3, query, and document tokens, then derives a reranker with early separate contextualization of query and document, later directed interaction from document to query, and layer dropping. The resulting architecture is closer to a late-interaction model than to a fully bidirectional cross-encoder, but it is explicitly derived from a cross-encoder rather than designed independently (Vast et al., 18 Feb 2026).

Dialogue response selection provides another useful contrast. There, the Cross-Encoder separately encodes each context–candidate pair with full attention, while Uni-Encoder preserves full context–response attention but encodes the context only once and processes all candidates in one forward pass. In the paper’s unifying view, “Square Attention with one candidate at a time” reduces to Cross-Encoder behavior, making the classical cross-encoder a special case of a broader interaction-design space (Song et al., 2021). A plausible implication is that much of the modern architectural work on reranking can be read as attempts to preserve the discriminative benefits of joint encoding while reducing repeated computation or undesirable interaction pathways.

3. Fine-tuning objectives and supervision sources

State-of-the-art cross-encoders can be fine-tuned to be highly effective in passage re-ranking, but the typical fine-tuning process requires large amounts of manually labelled data, a contrastive learning objective, and a set of heuristically sampled negatives (Pezzuti et al., 28 Mar 2025). In one standard setup, each query has one relevant passage $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$4, hard negatives $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$5 sampled from the top of a first-stage ranking list, and a contrastive or softmax ranking objective such as Localized Contrastive Estimation. Hard negatives are drawn from strong candidate generators rather than from random corpus documents; examples in the literature include sampling from the top 200 or top 500 passages produced by ColBERTv2 (Pezzuti et al., 28 Mar 2025, Schlatt et al., 2024).

An alternative supervision regime is distillation from a stronger teacher ranker. Here the student cross-encoder is trained to mimic the ranking order produced by an LLM-based or otherwise stronger reranker, often using RankNet as the distillation loss. Sequential multi-stage fine-tuning can then be defined as contrastive learning followed by distillation ($\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$6) or distillation followed by contrastive learning ($\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$7) (Pezzuti et al., 28 Mar 2025). A systematic comparison found that single-stage contrastive fine-tuning is consistently better than single-stage distillation, and that the best multi-stage approach offers no statistically significant advantage over the best single-stage contrastive baseline, leading to the conclusion that standard contrastive fine-tuning remains sufficient in that setting (Pezzuti et al., 28 Mar 2025).

Distillation quality, however, depends heavily on the teacher-generated data. Rank-DistiLLM argues that prior gaps between LLM rerankers and distilled cross-encoders were largely due to weak first-stage candidate pools, insufficient ranking depth, and pairwise-only training data. Its Rank-DistiLLM dataset retrieves top-100 passages with BM25 or ColBERTv2, reranks them with RankZephyr, and studies truncated variants at depths 10, 25, 50, and 100. The strongest distilled models are obtained from ColBERTv2-then-RankZephyr rather than BM25-then-RankZephyr, and performance peaks at 50 passages per query rather than 100 (Schlatt et al., 2024). This suggests that “distillation” is not a monolithic recipe: the quality and depth of the candidate pool are part of the supervisory signal.

Several papers replace binary relevance labels with richer signals. In legal question answering, a lightweight cross-encoder based on ms-marco-MiniLM-L-6-v2 is trained with mean squared error to predict continuous C-LIME perturbation-based attribution scores rather than cited/uncited labels, because binary labels do not preserve the ranking information needed for passage selection (Elganayni et al., 2 Jun 2026). In interactive information-seeking retrieval, a MiniLM cross-encoder is meta-trained with MAML and then fine-tuned per query using only the feedback documents for that query, with BitFit restricting updates to bias terms so that only about 0.11% of parameters are trainable (Baumgärtner et al., 2022). These examples broaden the notion of cross-encoder supervision from manual relevance judgments to teacher rankings, attribution scores, and per-query feedback.

4. Effectiveness, generalization, and efficiency

Cross-encoders remain a strong empirical baseline even under comparison with modern LLM rerankers. In reranking SPLADE on MS MARCO/TREC Deep Learning, DeBERTa-v3 and ELECTRA are hard to distinguish in-domain, and reranking tends to smooth out differences among first-stage SPLADE retrievers. Out-of-domain, the picture is clearer: on BEIR and LoTTE, DeBERTa-v3 is consistently better than ELECTRA across retrievers and reranking depths, and increasing reranking depth is more consistently beneficial once ArguAna is excluded from BEIR-12 (Déjean et al., 2024).

Against LLM-based listwise rerankers, the evidence is nuanced rather than replacement-oriented. GPT-4 is competitive with DeBERTa-v3 and sometimes better, especially on some out-of-domain or held-out settings such as NovelEval, but traditional cross-encoders remain very competitive overall. GPT-3.5 Turbo is generally weaker than the cross-encoder baseline, and open LLM rerankers such as SOLAR, Yi-34B-Chat, and Llama-70B-chat are described as much less reliable than GPT-4 and often below the best cross-encoder (Déjean et al., 2024). Query expansion studies reach a related conclusion from another angle: naive expansion methods such as RM3, Query2Doc, or Query2Keyword can degrade strong rerankers like MonoT5 and RankT5, but carefully designed keyword generation and reciprocal-rank-weighted fusion can improve zero-shot generalization on BEIR and TREC Deep Learning 2019/2020 (Li et al., 2023).

The effectiveness–efficiency trade-off remains central. Cross-encoders are already slow relative to first-stage retrievers, but LLM rerankers are substantially slower and more expensive still; reranking 50 documents with GPT-4 or Llama-70B can take up to about a minute on an H100 GPU (Déjean et al., 2024). This has driven multiple acceleration efforts. ED2LM trains an encoder–decoder reranker in a document-to-query generation formulation and decomposes it into a decoder-only LLM at inference, achieving results comparable to more expensive cross-attention ranking approaches while being up to 6.8X faster (Hui et al., 2022). MICE decreases fourfold the inference latency compared to standard cross-encoders, matches late-interaction models like ColBERT in latency, retains most in-domain effectiveness, and often improves out-of-domain robustness (Vast et al., 18 Feb 2026). Rank-DistiLLM reports cross-encoders distilled from LLMs that achieve the effectiveness of the teacher LLMs while being up to 173 times faster and 24 times more memory efficient (Schlatt et al., 2024).

Interpretability-based analysis adds an unexpected perspective on these trade-offs. “Cross-Encoder Rediscovers a Semantic Variant of BM25” argues that a MiniLM reranker reconstructs a BM25-like circuit: Matching Heads compute soft semantic term frequency, Relevance Scoring Heads aggregate those signals, and a dominant low-rank direction of the embedding matrix stores IDF-like information (Lu et al., 7 Feb 2025). A plausible implication is that part of cross-encoder effectiveness comes not from an opaque departure from classical IR, but from a semantic extension of its core heuristics.

5. Integrating lexical, feedback, and multidimensional signals

A recurrent theme in the literature is that cross-encoders benefit when external ranking signals are exposed at the input level rather than fused only after scoring. One direct example is CE$\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$8, which injects the BM25 score as text between query and passage: $\mathcal{R}_q^k=\{d_1,\ldots,d_k\}$9 Using global Min-Max normalization and integer encoding, this method improves all four tested cross-encoders—BERT-base, BERT-large, DistilBERT, and MiniLM—on MS MARCO dev and TREC DL 2019/2020, and it improves exact-matching capabilities over both BM25 and the standard query–passage cross-encoder (Askari et al., 2023).

A related but more semantic approach is to enrich the document with a textual relevance statement. In Consumer Health Search, candidate documents retrieved by BM25 are enhanced with statements such as “Credibility score of the document is X,” and the cross-encoder scores the query together with this enhanced document: $q$0 The resulting $q$1 statistically outperforms both aggregation-based multidimensional methods and score-injection baselines, indicating that cross-encoders can use natural-language statements to incorporate non-topical relevance dimensions such as credibility (Upadhyay et al., 2023).

Relevance feedback produces a different form of augmentation. In information-seeking retrieval, a MiniLM cross-encoder initialized from MS MARCO is fine-tuned per query on the top $q$2 relevant and top $q$3 non-relevant judged documents from BM25, optionally after MAML meta-training. The strongest system is a fusion of CE MAML + Query FT with BM25-QE via Reciprocal Rank Fusion,

$q$4

with $q$5, yielding an average nDCG@20 of 0.4973 versus 0.4427 for BM25-QE alone and the reported 5.2 nDCG@20 headline improvement (Baumgärtner et al., 2022). This indicates that cross-encoder reranking is not limited to static relevance estimation; it can also serve as a per-query adaptive component in interactive search.

6. Multilingual and specialized deployments, and mechanistic interpretation

Cross-encoder reranking has become a standard experimental intervention in cross-lingual retrieval. In one CLIR study, BM25 retrieves the top-100 candidates, the judged relevant document is injected at rank 100 if necessary so reranking always sees a gold-containing candidate set, and XLM-R-base or Nomic is fine-tuned as a pointwise “BERT-style” cross-encoder with binary cross-entropy over sigmoid scores (Goworek et al., 24 Nov 2025). Reranking helps most when BM25 is very weak—especially for cross-script or lexically distant pairs such as en–zh, ar–ru, and ja–en—and its effectiveness depends strongly on hard-negative quality. On mMARCO, for example, XLM-R trained on hard negatives reaches 83.4 Recall@10 and 62.5 nDCG@100, whereas easy-negative training can fail to surpass BM25 (Goworek et al., 24 Nov 2025).

Parameter-efficient multilingual reranking offers a different solution to data scarcity. Using mBERT as the base encoder, one paper composes Ranking Adapters or Sparse Fine-Tuning Masks trained on English relevance judgments with Language Adapters or Language Masks trained by masked language modeling, then applies the resulting cross-encoder to multilingual and cross-lingual reranking. These modular rerankers outperform full fine-tuning in zero-shot transfer, with gains particularly pronounced for low-resource languages such as Kyrgyz, Uyghur, Somali, and Swahili (Litschko et al., 2022). BLICEr applies an analogous post-hoc reranking idea to bilingual lexicon induction by combining a CLWE score with a multilingual cross-encoder score over a small candidate set, improving performance in 351 out of 352 setups (Li et al., 2022).

Specialized domains show that the same retrieve–then–rerank pattern can support objectives other than generic topical relevance. In legal QA on AQuAECHR, semantic similarity is reported as effectively uncorrelated with perturbation-based attribution scores, and within the retriever candidate pool similarity ranking performs worse than random selection at surfacing gold citation paragraphs. Training a lightweight cross-encoder on continuous C-LIME attribution targets shifts the mean gold rank from 37.4 under GTR similarity ranking to 11.1 for Mistral and 12.0 for Llama, while substantially improving citation faithfulness and NLI-based citation similarity (Elganayni et al., 2 Jun 2026). In Vietnamese reranking, ViRanker adapts BGE-M3 with Rotary Position Encoding, Blockwise Parallel Transformer, an MLP scoring head, and hybrid hard-negative sampling, achieving NDCG@10 = 0.7302 and MRR@10 = 0.7107 on MMARCO-VI (Dang et al., 11 Sep 2025). These cases indicate that cross-encoder reranking can be retargeted toward citation utility, language-specific syntax, or other domain constraints when the supervision signal is aligned with the downstream use.

Mechanistic analysis has begun to explain why cross-encoders are effective across these varied settings. The MiniLM study cited above identifies Relevance Scoring Heads 10.1, 10.4, 10.7, and 10.10; Matching Heads such as 0.8, 1.7, 2.1, and 8.8; and a top singular direction $q$6 of the embedding matrix that correlates with IDF at about $q$7. A linear reconstruction built from Matching Scores and $q$8 achieves Pearson correlation $q$9 with cross-encoder relevance scores on an MS MARCO-based dataset, and across 12 IR datasets the reported median alignment with cross-encoder rankings reaches 88.4% NDCG@10 (Lu et al., 7 Feb 2025). This suggests that cross-encoder re-ranking may often be understood not as a rejection of classical lexical ranking, but as a semantic generalization of BM25-style matching, saturation, document-length normalization, and term-importance weighting.