Can LLMs Perform the Creative and Challenging Parts of Research?

Determine whether large language models can take on the creative and challenging parts of the scientific research process, beyond supporting tasks such as literature retrieval, code generation, and corpus analysis, by rigorously assessing capabilities like independently proposing, framing, and advancing novel research ideas to an expert standard.

Background

The paper notes that, although LLMs have shown strong performance on many research-adjacent tasks (e.g., solving math problems, assisting in proof writing, retrieving related work, and generating code), their ability to handle the more creative and difficult stages of scientific work is unresolved. This motivates the authors’ large-scale, expert-blinded comparison of AI- and human-generated research ideas to probe one key facet of creativity: research ideation.

The study provides evidence that AI-generated ideas can be rated as more novel than expert ideas, though questions remain about broader creative capabilities beyond ideation, and how such capabilities translate to executed research outcomes.

References

While these are useful applications that can potentially increase the productivity of researchers, it remains an open question whether LLMs can take on the more creative and challenging parts of the research process.

— Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers  (2409.04109 - Si et al., 2024) in Section 1 (Introduction)

Co-Scientist mitigates this via novelty objectives and diversified temperature sampling, yet the extent to which true conceptual novelty can be achieved remains an open question.

— Accelerating Scientific Research with Gemini in the Real-World  (2608.26701 - Schmidgall et al., 27 Aug 2026) in Ethical considerations, Section 2.4, paragraph “Diversity of scientific inquiry”

As LLM capabilities advance and their role in AI for Science expands, it remains an open problem whether they can genuinely discover scientific laws and how this ability should be evaluated.

— Can LLMs Discover Scientific Laws in Real and Parallel Worlds?  (2609.01552 - Huang et al., 1 Sep 2026) in Abstract, page 1