Systematic Exploration of Admissible Hypothesis Sets by LLMs under Underdetermination

Determine whether large language models can systematically explore the full set of admissible explanatory hypotheses consistent with a fixed set of observations in controlled underdetermination settings, by generating multiple non-redundant, valid hypotheses rather than converging on a small subset or a single answer.

Background

The paper motivates evaluation beyond single-answer correctness because many scientific inference problems are underdetermined: multiple mechanistically distinct hypotheses can explain the same observations. In such settings, an effective system should enumerate and explore the complete admissible hypothesis set rather than stopping at one correct solution.

To address this gap, the authors introduce HypoSpace, treating LLMs as samplers of finite hypothesis sets and measuring Validity (appropriateness), Uniqueness (originality), and Recovery (coverage). The stated open question concerns whether current LLMs can systematically explore and cover admissible explanation spaces under controlled underdetermination, which HypoSpace is designed to diagnose.

References

Yet contemporary LLM evaluations largely reward one-shot correctness~\citep{shojaee2025llm,koblischke2025gravity,shojaee2024llm,wang2024mmlu,coignion2024performance,hendrycks2020measuring}, leaving open whether models can systematically explore sets of valid explanations under controlled underdetermination.

— HypoSpace: Evaluating LLM Creativity as Set-Valued Hypothesis Generators under Underdetermination  (2510.15614 - Chen et al., 17 Oct 2025) in Section 1 (Introduction), Page 1

\citet{salimi2026wiring}, synthesizing more than sixty studies across eleven model families, report a similar pattern and identify ``limited mechanistic understanding of abductive processes'' as an open problem.

— LLMs Don't Pay for the Jump  (2608.14397 - Balani et al., 14 Aug 2026) in Section 6, Empirical Signatures of Decoupling