Scaling Generative Search to Millions of Passages

Develop scalable generative search methods—particularly numeric ID-based approaches such as Differentiable Search Index (DSI)—that can operate effectively on corpora containing millions of passages, achieving competitive retrieval performance and efficiency relative to dual-encoder baselines.

Background

Within the discussion of numeric ID identifiers for generative search, the survey highlights significant challenges related to generalization, corpus updates, and scaling to large corpora. Numeric ID-based approaches require the LLM to memorize document–ID associations, which becomes increasingly difficult as corpus size grows.

The cited scaling study reports that while generative search can be competitive on small corpora, extending to millions of passages remains unresolved, underscoring a core limitation of current identifier and training strategies in large-scale retrieval settings.

References

It is found that while generative search is competitive with state-of-the-art dual encoders on small corpora, scaling to millions of passages remains an important and unsolved challenge.

— A Survey of Generative Search and Recommendation in the Era of Large Language Models  (2404.16924 - Li et al., 2024) in Section 4.2, Document Identifiers (Numeric ID)

Each is a property of the protocol rather than a measured effect; we did not vary corpus size, so we do not quantify how retrieval quality would change at larger scale.

— The Embedder's Dilemma: LLMs Are Better, but at What Cost?  (2608.12875 - Assadi et al., 13 Aug 2026) in Appendix, Section "Extended Limitations," paragraph (iii) "Small-corpus retrieval"

We do not claim unqualified scalability: our evaluation covers a limited range, and the carousel's linear cost in $N_{db}$ makes $>105$ records prohibitive. Unlike ORAM, the carousel uses a single in-enclave scan and outperforms ORAM-based constructions for private membership tests. Thus, TME's carousel is a cheaper oblivious option for smaller scales. Larger databases may require hybrid designs, such as oblivious cluster-level scans, which we leave as future work.

— Trusted Model Environment for Private Semantic Computations  (2609.30032 - Duddu et al., 24 Sep 2026) in Section 5, Evaluation—Private Semantic Database Retrieval; Takeaway