MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry

Abstract: LLMs have shown promise for molecular property prediction, but their ability to reason over chemical structures remains limited, as molecular representations such as SMILES differ substantially from the natural language on which LLMs are primarily trained. To bridge this semantic and chemical knowledge gap, we propose MolE-RAG, a training-free, molecule-centric retrieval-augmented generation framework for LLM-based molecular property prediction. MolE-RAG augments each prediction with three complementary sources of inference-time context: retrieved chemistry literature, molecule-specific information including compound synonyms, identifiers, functional group annotations, and physicochemical descriptors, and structurally similar molecules retrieved from the training set. We evaluate MolE-RAG across nine molecular property prediction tasks using proprietary, chemistry-specialized, and open-source LLMs. Across general-purpose LLMs, MolE-RAG improves ROC-AUC by up to 28 percentage points on classification tasks and reduces regression RMSE by up to 67% relative to a SMILES-only baseline. We further find that the utility of each context source varies across models and tasks, with different models benefiting most from textual retrieval, molecular context, or structural retrieval. These results suggest that molecule-centric retrieval can improve LLM-based molecular property prediction without model fine-tuning while providing a flexible framework for integrating heterogeneous chemical knowledge at inference time.

• The paper introduces the MolE-RAG framework that integrates textual retrieval, molecule-specific context, and structure-based retrieval to improve molecular property prediction.
• It demonstrates significant improvements in ROC-AUC and RMSE across multiple MoleculeNet tasks compared to SMILES-only approaches.
• The study highlights that inference-time context augmentation can narrow performance gaps between LLMs and specialized GNNs, suggesting a versatile plug-and-play solution for chemistry applications.

MolE-RAG: Enabling Structure-Enhanced Retrieval-Augmented Generation for Molecular Property Prediction

Motivation and Background

LLMs exhibit limited competency in molecular property prediction due to their weak inductive bias for non-natural language molecular representations such as SMILES and insufficient exposure to scientific literature during pretraining. Existing approaches to enhance LLMs in chemistry—through model finetuning or instruction tuning—show partial effectiveness, but do not bridge the semantic gap between structured molecular data and textual scientific knowledge, nor do they leverage external evidence at inference-time.

The MolE-RAG framework addresses these limitations by integrating structure-enhanced retrieval-augmented generation (RAG) specifically adapted to chemistry, augmenting LLM predictions with retrieved chemical literature, molecule-specific annotations, and structurally similar labeled molecules. The formulation is training-free and designed for direct applicability to off-the-shelf LLMs.

MolE-RAG Framework

The core innovation of MolE-RAG lies in aggregating three complementary forms of task-adaptive, inference-time context for molecular property prediction:

1. Textual Retrieval: Uses LLM-augmented hybrid queries for passage retrieval from the ChemRAG corpus, blending task instructions, LLM-generated domain keywords, and filtered molecule names. This promotes maximal lexical overlap with chemistry texts.
2. Molecule-Specific Context: Injects compound identifiers (names, synonyms, IUPAC), functional group annotations (via AccFG), and a compact, task-adaptive set of highly correlated RDKit descriptors as natural language blocks in the prompt. Descriptor selection maximizes relevance, focusing on properties most predictive for each target.
3. Structure-Based Retrieval: Retrieves labeled, structurally similar molecules from the training set using molecular fingerprints tuned per task (e.g., ECFP2, Atom Pair, Topological Torsion, FCFP2). Molecules are ranked by Tanimoto similarity, ensuring informative in-context demonstrations.

The prompt architecture places these context sources in a fixed sequence: task instruction, retrieved texts, molecule context, structure neighbors, and the query molecule, preserving interpretability and maximizing LLM access to domain knowledge.

Figure 1: The MolE-RAG pipeline illustrating the integration of text retrieval, molecular context, and structure-based retrieval as modular, composable context sources.

Figure 2: The MolE-RAG framework shown on the BBBP task, demonstrating context augmentation for a molecular property prediction.

Experimental Evaluation

Datasets and Protocol

MolE-RAG is benchmarked on nine standard MoleculeNet tasks, spanning both classification (BBBP, BACE, ClinTox, HIV, Tox21, SIDER) and regression (ESOL, FreeSolv, Lipophilicity). Scaffold splits are used for train/validation/test separation. Structure-based retrieval pools are built from the training split, and model and fingerprint hyperparameters are selected on the validation set.

Model Classes

A spectrum of LLMs is assessed:

• Proprietary: GPT-4o-mini, GPT-5.4-nano
• General-purpose open-source: Llama-3.2-3B, Mistral-7B, Qwen3-4B
• Chemistry-specialized: ChemDFM-v2.0-14B

All models operate in a strict zero-shot setting (temperature 0), isolating context augmentation effects.

Baseline comparators include supervised/self-supervised GNNs (MGCN, SchNet, GROVER, MolCLR family).

Main Results

Classification Tasks

MolE-RAG drives consistent and sometimes dramatic improvements for general-purpose LLMs over SMILES-only baselines. For instance, Mistral-7B ROC-AUC rises from 46.1 to 74.6 (BBBP) and 48.6 to 76.8 (BACE), while Qwen3-4B increases from 53.0 to 80.1 (BBBP) and 53.3 to 73.5 (HIV)—up to 28 ROC-AUC points gain in some settings. The proprietary and chemistry-focused LLMs also benefit, but open models particularly close the gap with proprietary baseline performance.

Noteworthy, the best MolE-RAG-augmented LLMs approach supervised GNNs and, in several tasks, outperform GNNs trained via self-supervision (MolCLR family). However, on tasks like ClinTox, graph methods still yield higher accuracy—LLMs maintain an average performance gap when explicit graph-level structure is most relevant.

Regression Tasks

MolE-RAG reduces regression RMSE by margins as large as 67% (e.g., Mistral-7B on FreeSolv: 12.585 → 4.128), and makes open LLMs competitive with, or exceeding, GNNs in most regression tasks. Best-augmented general-purpose LLMs avoid large numeric failures seen in SMILES-only settings and often yield RMSEs within reach of the strongest MolCLR models.

Context Source Ablations

Ablations demonstrate that each context source contributes differently:

• Textual retrieval is only effective with LLM-augmented queries—naive SMILES-based retrieval is ineffective or detrimental due to poor lexical match with text.
• Molecule context (identifiers, functional groups, descriptors) stabilizes numerics and focuses the LLM on property-relevant features.
• Structure-based retrieval is the most potent single augmentation, yielding up to 20-point gains in ROC-AUC over both zero-shot and random few-shot in-context learning.

No single fingerprint representation is optimal; task-adaptive selection produces the best results, aligning with known heterogeneity of structure–activity relationships across molecular properties.

Implications and Future Directions

MolE-RAG empirically validates the hypothesis that model performance in molecular property prediction is bottlenecked less by parameter scale than by task-relevant context availability. The results demonstrate that inference-time augmentation, if flexibly curated, can enable competitive performance in a plug-and-play, training-free fashion—substantially lowering the barrier for impactful chemistry applications of LLMs.

This framework facilitates future extensions, such as:

• Integration with dense retrievers and chain-of-thought reasoning to further enhance LLM interpretability and accuracy.
• Automated selection and fusion of context modalities (text, structure, descriptors) tailored to downstream property classes.
• Deployment as an interface layer, enabling downstream use of foundation models without domain-specific finetuning.

The modular design and strong empirical results suggest broader applicability in scientific domains where structured, non-linguistic data must be linked to domain textual knowledge.

Conclusion

MolE-RAG establishes that LLMs, when augmented at inference time with structurally and contextually curated evidence, can approach the predictive accuracy of GNNs in molecular property prediction and are particularly strong in regression settings. The study highlights how targeted retrieval from chemical literature, molecule-centric descriptors, and structure-based examples synergize to compensate for inherent LLM weaknesses in chemistry. With further improvements in retriever quality and prompt engineering, MolE-RAG could become a general-purpose context augmentation paradigm for scientific LLMs operating over non-textual data domains (2606.05693).