Identify the best candidate generators for reranking pipelines

Identify the best candidate-generating retrievers for reranking pipelines by evaluating retrieval recall at larger candidate-set cutoffs, such as the top 100 or top 1000 documents, rather than only at the small cutoffs used for standalone ranking metrics.

Background

The evaluation primarily uses small cutoffs, which reward retrievers for placing relevant documents near the top of the ranking. In a multistage retrieval pipeline, however, a reranker typically receives a much larger candidate set, so the relevant criterion is how many relevant documents are retrieved within the top 100 or top 1000. The paper therefore leaves unresolved which retrievers are most suitable as candidate generators for reranking systems.

References

Our results therefore show which retrievers rank well on their own, but identifying the best candidate generators for a reranking pipeline would require evaluating at these larger cutoffs, which we leave to future work.

— A Systematic Multi-Domain Evaluation of Document Retrievers  (2609.29455 - Velev et al., 24 Sep 2026) in Section 5, “Implications”