Can LLM generate interesting mathematical research problems?

Abstract: This paper is the second one in a series of work on the mathematical creativity of LLM. In the first paper, the authors proposed three criteria for evaluating the mathematical creativity of LLM and constructed a benchmark dataset to measure it. This paper further explores the mathematical creativity of LLM, with a focus on investigating whether LLM can generate valuable and cutting-edge mathematical research problems. We develop an agent to generate unknown problems and produced 665 research problems in differential geometry. Through human verification, we find that many of these mathematical problems are unknown to experts and possess unique research value.

• The paper demonstrates that GPT-5 architectures can autonomously generate novel, research-level mathematical problems in differential geometry.
• It employs the DeepMath-generate framework, using a generator and evaluator in an iterative feedback loop to refine problem quality and ensure originality.
• The results reveal robust, nontrivial outputs while highlighting current limitations that necessitate further prompt engineering and domain-specific enhancements.

Can LLMs Generate Interesting Mathematical Research Problems? An In-Depth Analysis

Motivation and Problem Setting

The paper "Can LLM generate interesting mathematical research problems?" (2603.18813) systematically investigates the capacity of LLMs, specifically the GPT-5 family, to autonomously propose valuable and novel mathematical research problems in advanced domains—in this instance, differential geometry. This extends beyond common evaluations of mathematical reasoning or problem-solving, targeting the more elusive property of creativity in mathematics: the ability to formulate questions that might meaningfully advance the field.

The analysis is grounded on three axes of mathematical creativity, initially established in (Chen et al., 13 May 2025): (1) generation of new concepts, (2) invention of new methods, and (3) creation of new mathematical objects. The current paper aims at one core aspect of creativity: the formulation of original research questions that are unknown in the literature, nontrivial, and potentially capable of generating new research directions.

System Architecture: DeepMath-generate

The framework developed, DeepMath-generate, comprises two main components: a generator and an evaluator. Both are instantiated as agents using GPT-5 and operate over a tightly controlled prompt protocol:

• Generator: Receives high-level knowledge points, then must produce a single, original, research-level mathematical problem that avoids trivial reformulations of known results and is accompanied by a justification as to why it is a "good" problem, paying heed to qualities such as profound insight, cross-disciplinary relevance, and simplicity of statement.
• Evaluator: Critically assesses output against rigorous criteria, checking for restatement of existing theorems, research depth, logical soundness, and the quality of justification.

The evaluation pipeline is iterative: if the generator’s problem fails to meet any criterion, the evaluator issues explicit feedback, prompting the generator to revise until validation succeeds. This method ensures deviations (e.g., triviality, ill-posedness, duplication of existing literature) are systematically filtered out, resulting in a set of research problems robust to expert-level scrutiny.

Prompts are constructed to encourage the distillation of the “essence” of a good mathematical question—simple in formulation, profound in implication, and, if possible, bridging disparate areas of theory.

Experimental Protocol and Results

Applying DeepMath-generate to 200 subfields in differential geometry, the authors produced an extensive list (665) of research problems. Each problem was subjected to expert human verification to ascertain three properties:

1. Unknown Status: Problems are not apparent reformulations of existing theorems or exercises in the field.
2. Research Value: Each offers nontrivial challenges with the potential to inspire further work.
3. Logical Soundness: Formal well-posedness and consistency.

Two illustrative examples focus on the topology and geometry of exotic spheres and nonnegative sectional curvature:

• Problem 1 asks for which pairs $(\Sigma^k, r)$ (where $\Sigma^k$ is an exotic $k$-sphere, $k \geq 7$, and $r$ a positive integer) there exist rank-$r$ vector bundles admitting complete metrics of nonnegative curvature with soul diffeomorphic to $\Sigma^k$. This tightly intertwines differential topology and Riemannian geometry, open even to field experts.
• Problem 2 probes whether the moduli space of nonnegatively curved metrics on an exotic sphere can be topologically distinguished from the moduli space for the standard sphere, directly connecting geometric analysis, smoothing theory, and moduli space topology.

These problems are not extracts of current theorems, are precise, and their resolution would have substantial theoretical implications, confirming that the LLM-driven pipeline can, in some instances, achieve true mathematical creativity as formally defined by the authors.

Limitations and Observations

Despite promising outcomes—the generation of numerous previously unknown, research-grade problems—the paper notes a qualitative ceiling in the current LLM output. The generated problems, though robust and nontrivial, do not reach the transformational depth or elegant simplicity exemplified by historical conjectures such as the Poincaré Conjecture or the Riemann Hypothesis. This points to the limitations of both LLM pretraining and system prompt engineering for the highest tiers of creativity, suggesting that further improvements (including prompt refinement, domain-specific pretraining, or reinforcement learning augmentation) would be necessary to cross this threshold.

The role of prompt specificity is emphasized: instructing the LLM with precise, context-rich information increases the likelihood of creative, yet sound, output.

Theoretical and Practical Implications

From a theoretical perspective, these findings substantiate the claim that LLMs can be leveraged not only as assistants for deductive tasks in mathematics but also as active collaborators capable of contributing research-level questions. This realization shifts the perceived function of LLMs in mathematical pipelines—from passive answerers to sources of new lines of inquiry.

Practically, curating large corpora of high-quality open research questions, automatically filtered for redundancy and logical flaws, presents immediate value for mathematical research communities. Cross-field collaborations could also be catalyzed by LLMs’ capacity to identify problem statements bridging previously disconnected domains.

The methodology also has meta-research implications—future LLM-based agents might refine their creativity through RLHF or even curriculum-based autonomy, iterating towards deeper mathematical insight.

Conclusion

The paper demonstrates that, when embedded in a well-designed iterative agent architecture and governed by domain-appropriate prompts, advanced LLMs such as GPT-5 possess a tangible (albeit bounded) ability to generate unknown, research-level mathematical problems with clear value to the field. The results indicate the emergence of a new paradigm for collaboration between mathematicians and AI, particularly in problem discovery—a core aspect of mathematical creativity. Advancement beyond the current ceiling would depend on further improvements in both system design and underlying model capabilities.

For more comprehensive technical discussion of agentic LLM frameworks, refer also to (Luo et al., 27 Mar 2025).