Assess SelRoute’s advantage over fact-key augmentation on benchmarks favoring fact keys

Determine whether the performance advantage of SelRoute over Contriever with LLM-generated fact-key augmentation, observed on the LongMemEval_M benchmark, persists on other retrieval benchmarks where fact-key augmentation provides greater benefit.

Background

SelRoute is proposed as a query-type-aware routing framework that outperforms several baselines on LongMemEval_M, including Contriever augmented with LLM-generated fact keys. Fact keys append LLM-derived factual summaries to each session at index time, potentially providing broad factual coverage at inference cost. SelRoute instead uses deterministic vocabulary enrichment and routing without LLMs at query time.

While SelRoute shows a performance advantage on LongMemEval_M, the authors explicitly note uncertainty about whether this advantage would hold on datasets where fact-key augmentation is particularly beneficial. Establishing this would clarify the conditions under which routing versus LLM-based index augmentation is preferable.