Scaling Generative Search to Millions of Passages

Develop scalable generative search methods—particularly numeric ID-based approaches such as Differentiable Search Index (DSI)—that can operate effectively on corpora containing millions of passages, achieving competitive retrieval performance and efficiency relative to dual-encoder baselines.

Background

Within the discussion of numeric ID identifiers for generative search, the survey highlights significant challenges related to generalization, corpus updates, and scaling to large corpora. Numeric ID-based approaches require the LLM to memorize document–ID associations, which becomes increasingly difficult as corpus size grows.

The cited scaling study reports that while generative search can be competitive on small corpora, extending to millions of passages remains unresolved, underscoring a core limitation of current identifier and training strategies in large-scale retrieval settings.

References

It is found that while generative search is competitive with state-of-the-art dual encoders on small corpora, scaling to millions of passages remains an important and unsolved challenge.

— A Survey of Generative Search and Recommendation in the Era of Large Language Models  (2404.16924 - Li et al., 2024) in Section 4.2, Document Identifiers (Numeric ID)

Each is a property of the protocol rather than a measured effect; we did not vary corpus size, so we do not quantify how retrieval quality would change at larger scale.

— The Embedder's Dilemma: LLMs Are Better, but at What Cost?  (2608.12875 - Assadi et al., 13 Aug 2026) in Appendix, Section "Extended Limitations," paragraph (iii) "Small-corpus retrieval"