Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis

Abstract: Single-step retrosynthesis is a central component of computer-aided synthesis planning, yet its intrinsically one-to-many nature is poorly captured by single-answer evaluation and benchmarking protocols. To address this, we introduce Top-K prompting as a robust training and inference paradigm to better capture diverse, plausible reaction predictions. We compile CREED-CCV-2+USPTO-XL, an ultra-large-scale dataset of ~45.6 million verified reactions to train the C3LM (Chemistry Constraint-Consistent LLM). By integrating fine-tuning with ChemCensor-based and novelty-oriented rewards, our model achieves state-of-the-art performance on the OOD URSA-expert-2026 benchmark. Further analysis of reaction uniqueness shows that LLMs and conventional models explore complementary reaction spaces, motivating ensemble-based retrosynthesis systems. Overall, our results establish Top-K, plausibility-aware training as a practical new direction for robust future LLM-based synthesis planning.

• The paper introduces Top-K prompting and trains C3LM models on 45.6 million verified reactions, improving diverse single-step retrosynthesis beyond conventional approaches.
• The best 2.6B-parameter model uses ChemCensor plausibility and novelty rewards to achieve gold-medal performance across all Av. PT-Top-K CC metrics on the OOD URSA-expert-2026 benchmark.
• The results show that LLMs and conventional retrosynthesis tools generate complementary reaction spaces, supporting ensemble systems while highlighting risks from benchmark leakage and proxy-based evaluation.

Overview

This paper addresses a persistent weakness in LLM-based single-step retrosynthesis (SSRS): the one-to-many nature of the task, where a target molecule admits multiple plausible disconnections, is poorly served by single-answer prompting and evaluation. The authors introduce Top-$K$ prompting as both an inference and training paradigm, compile CREED-CCV-2+USPTO-XL, an ultra-large training set of ~45.6M verified reactions, and train a new generation of C3LM (Chemistry Constraint–Consistent LLM) models that, after reinforcement fine-tuning (RFT) with ChemCensor-based and novelty rewards, surpass conventional SSRS models on the out-of-distribution URSA-expert-2026 benchmark (2608.18940).

Top-K prompting as a benchmarking and training paradigm

The paper formalizes two prompting modes built on 15 natural-language templates adapted from MolInstructions. In Top-1 mode, the model produces one reactant set per prompt; in Top-$K$ mode, prompts are suffixed with "Give me 15 different answers," and three sampled responses per target are averaged. The transition from Top-1 to Top-$K$ yields large gains for nearly all foundation LLMs on the diversity-sensitive Av. PT-Top-10 metric, and it reshuffles rankings: Grok-4.1 leads in Top-1 mode, while Gemini 3.1 Pro becomes the strongest LLM baseline under Top-$K$. Only GPT 5.2 degrades under Top-$K$ on Av. PT-Top-10. On this basis, the authors argue that Top-$K$ should be the default protocol for SSRS benchmarking of LLMs.

The same principle applies to training: retraining C3LM-LFM2-CREED-CCV+USPTO in Top-$K$ mode rather than Top-1 improves Max/@3/@5/@10 by +0.30/+0.62/+0.70/+0.60 on URSA-expert-2026 — a greater than 2.5-fold improvement on Av. PT-Top-10 — lifting the SFT model to parity with top proprietary LLMs such as Gemini 3.1 Pro and GPT 5.5.

Data scaling with CREED-CCV-2+USPTO-XL

CREED-CCV-2 merges ChEMBL v34 compounds enumerated through the Virtual Synthesis Engine with reactions from CREED-CCV, deduplicated and verified by ChemCensor v1.1.1 (~2.9M products, ~36M reactions). USPTO-XL augments USPTO products with VSE-enumerated, ChemCensor-verified reactants (~859K products, ~10.6M reactions), addressing USPTO's limitation of roughly one reaction per product. Scaling training data more than sixfold adds further gains of +0.12/+0.13/+0.17/+0.29 across Max/@3/@5/@10.

Reinforcement fine-tuning with plausibility and novelty rewards

Starting from the LFM2 2.6B checkpoint, the best model is trained with GRPO using six reward components: thinking-format validity, SMILES validity, exact-$k$ answer count, uniqueness within the group, ChemCensor plausibility score, and novelty (plausible reactants outside the exhaustive training-set reactant lists). CC-RFT alone contributes +0.04/+0.07/+0.06/+0.02; adding the novelty reward contributes another +0.08/+0.06/+0.08/+0.08. The resulting C3LM-LFM2-RFT-CC-NR achieves gold-medal performance on all Av. PT-Top-K CC metrics on URSA-expert-2026 (Max 2.16; @3 1.94; @5 1.73; @10 1.37), exceeding the best conventional models (LocalRetro, MHNreact, RetroKNN).

On USPTO-50K-test-mini, C3LM-RFT-CC-NR ranks second on Av. PT-Top-10 (1.85 vs. MHNreact's 1.90), which the authors interpret as evidence of genuine chemical generalization rather than memorization. Conversely, near-perfect Av. PT-Max CC values (~4.8–4.9) by conventional models are flagged as potential evidence of leakage between USPTO-50K benchmarks and these models' training data — a caution against that metric on this benchmark.

Complementarity of LLM and conventional reaction spaces

An intersection analysis against MHNreact shows that most LLMs produce fewer unique plausible reactions than MHNreact (negative $\Delta$), but only the two strongest C3LM variants exceed it (+0.4 and +0.3 reactions per target on average). Every LLM nonetheless proposes some plausible reactions absent from MHNreact's output, and pooled predictions from all 17 GP LLMs collectively exceed all eight conventional models on frontier metrics (e.g., Av. PT-Top-10 of 1.65 vs. 1.45 on URSA-expert-2026). The union of all 30 benchmarked models defines the detectable frontier (Av. PT-Top-10 = 1.81), leaving +0.44 headroom over the best individual model — motivating ensemble-based retrosynthesis systems combining LLMs and conventional tools.

Limitations

The authors acknowledge several constraints. First, optimizing with ChemCensor-based rewards creates partial circularity with ChemCensor-based evaluation, defended as a deliberate design choice for practitioner utility. Second, ChemCensor does not model practical synthesis parameters (conditions, solvents, purification), and its reference datasets are limited to patent-derived reaction spaces. Third, chemical diversity is assessed only via exact SMILES matching, without reaction-class or mechanistic comparison. Finally, the template-based virtual synthesis engine used to build the training data may bias coverage toward known chemical patterns and omit novel transformations.

Conclusion

This work establishes Top-$K$ prompting and plausibility-aware RFT as effective levers for LLM-based SSRS, demonstrating that a 2.6B-parameter domain-tuned model can outperform both frontier proprietary LLMs and established conventional baselines on an OOD expert-curated benchmark. The demonstrated complementarity between LLM and conventional reaction spaces, together with the substantial remaining gap to the pooled-model frontier, frames the central open question: how far can plausibility-aware training close that gap before the proxy metric itself becomes the binding constraint?