Can LLMs Perform the Creative and Challenging Parts of Research?

Determine whether large language models can take on the creative and challenging parts of the scientific research process, beyond supporting tasks such as literature retrieval, code generation, and corpus analysis, by rigorously assessing capabilities like independently proposing, framing, and advancing novel research ideas to an expert standard.

Background

The paper notes that, although LLMs have shown strong performance on many research-adjacent tasks (e.g., solving math problems, assisting in proof writing, retrieving related work, and generating code), their ability to handle the more creative and difficult stages of scientific work is unresolved. This motivates the authors’ large-scale, expert-blinded comparison of AI- and human-generated research ideas to probe one key facet of creativity: research ideation.

The paper provides evidence that AI-generated ideas can be rated as more novel than expert ideas, though questions remain about broader creative capabilities beyond ideation, and how such capabilities translate to executed research outcomes.