Solve multi-element factual retrieval from model parameters

Develop methods that enable large language models to retrieve complete, accurate multi-element factual lists from their parameters while avoiding omissions and hallucinated additions.

Background

The benchmark evaluates whether models can retrieve several qualifying facts, apply filtering constraints, and organize the results into flat or hierarchical lists. The reported results show that even the strongest standard-inference systems achieve relatively low LLM-Judge-F1 scores, with complex questions involving sub-attributes performing substantially worse.

The authors characterize this capability as unresolved because supplying the source Wikipedia page produces large performance gains, indicating that parametric recall—not merely reasoning or formatting—is the principal bottleneck.

References

Multi-element factual retrieval remains largely unsolved: the best standard-inference model scores below 37\% LLM-Judge-F1, with the top tier (GPT-5.4, Claude Opus variants, Gemini 2.5 Flash, Nova Premier) clustering in a narrow band that suggests a shared ceiling on parametric retrieval; the majority of models fall below 20\%.

— ListQA: A Benchmark for Evaluating List-Formatted Factual Knowledge Retrieval in Large Language Models  (2610.19165 - Kulkarni et al., 13 Oct 2026) in Section 5, Experiments, subsection “The Knowledge Bottleneck”