Designing reasoning-enhancing methods to maximize originality and diversity

Develop and evaluate frameworks for reinforcement learning of language models, chain-of-thought prompting, and scaling test-time compute that explicitly optimize for originality relative to the training set and diversity across multiple responses in open-ended tasks, rather than only enhancing the quality of single examples.

Background

The authors note that methods such as RL, chain-of-thought prompting, and scaling test-time compute are geared toward improving the quality of a single output, which does not directly address originality and diversity across multiple outputs.

They explicitly state that it is unclear how to design these methods to maximize originality and diversity in the open-ended setting, highlighting a methodological gap.

References

It is unclear how to design them to maximize originality against a training set, and diversity over multiple responses.

— Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction  (2504.15266 - Nagarajan et al., 21 Apr 2025) in Section 6 (Discussion), Subsection: Effects of reasoning-enhancing methods

We conjecture that the increased reasoning diversity introduced by MTS enables the model to generate multiple complementary solution paths, thereby improving the robustness of majority-vote decoding under larger inference budgets.

— Reactivating Test-Time Scaling for Plane Geometry Problem Solving  (2608.30156 - Kang et al., 31 Aug 2026) in Appendix, Section: Test-Time Scaling on Three Geometry Benchmarks

Second, designing thinking traces suitable for creative ideation is an open-problem. In our work, we tried one approach that was hindsight generation. While we instruct the teacher model to propose negatives, verify etc. there may be instances where there is no ‘search’ for candidates beyond what the model generated originally; we also tried distilling forward-thinking traces but it showed the same trend (Appendix E). We leave the problem of designing synthetic or appropriate thinking traces for creative/open-ended problems to future work.

— Training Reasoning Models to Generate Diverse Scientific Ideas  (2610.22324 - Wadhwa et al., 15 Oct 2026) in Section 5, page 6

Since our objective targets diversity within each set and does not directly optimize the spread of idea types across instances, improving population-level diversity is an open direction for future work.

— Training Reasoning Models to Generate Diverse Scientific Ideas  (2610.22324 - Wadhwa et al., 15 Oct 2026) in Section 5, page 7