MolSpectLLM: Multimodal Molecular Model
- MolSpectLLM is a 7B-parameter molecular foundation model that unifies spectroscopy, SMILES, and 3D geometry as coequal textual modalities.
- It generates and interprets spectra, converts between SMILES, IUPAC, and 3D structure, and enhances molecular elucidation through joint reasoning over multiple data types.
- The model leverages multimodal pretraining and a text-first strategy but faces challenges in stereochemical resolution, mixture handling, and data noise.
MolSpectLLM is a 7B-parameter molecular foundation model that treats spectroscopy, symbolic molecular representations, and 3D geometry as first-class, unified modalities within a single LLM. Built by pretraining and fine-tuning Qwen2.5‑7B on chemical text, SMILES, experimental and simulated spectra, and 3D structures encoded as standardized textual descriptions, it is designed to interpret and generate spectra, perform spectra-to-structure molecular elucidation, generate 3D structures from SMILES or spectra, and handle routine molecular tasks such as question answering and name conversion (Shen et al., 26 Sep 2025).
1. Conceptual scope and problem setting
MolSpectLLM addresses a limitation that is explicit in much prior molecular language modeling: most molecular LLMs and foundation models operate purely on SMILES or IUPAC strings, ignore 3D geometry, and ignore experimental spectroscopy, even though stereochemistry, spatial conformation, and experimental validation are central to practical chemistry (Shen et al., 26 Sep 2025). The model is therefore framed not as a new transformer architecture, but as a domain-specialized, multimodal adaptation of a general decoder-only backbone to joint molecular reasoning across text, spectra, and structure.
Its capabilities are organized around four task families. First, it performs spectrum analysis and generation, including SMILES→Spectra for 13C NMR, 1H NMR, IR, and MS, and Spectra→SMILES for structure inference from combinations of NMR, IR, and MS. Second, it supports molecular elucidation more broadly, including Spectra→SMILES, molecular question answering, and SMILES↔IUPAC conversion. Third, it generates 3D structures from SMILES or IUPAC names by emitting atom identities, bond connectivity, and Cartesian coordinates. Fourth, it supports a chained spectra→SMILES→3D workflow, linking experimental measurements to spatial structure within one autoregressive model (Shen et al., 26 Sep 2025).
This design places MolSpectLLM at the intersection of several lines of work. Spectrum-aware representation learning had already begun to appear in encoder-style pretraining frameworks such as MolSpectra, which aligns 3D molecular encoders with UV–Vis, IR, and Raman spectra (Wang et al., 22 Feb 2025), while broader multimodal molecular LLMs such as Mol-LLM focused on SELFIES-plus-graph generalism rather than spectroscopy or 3D generation (Lee et al., 5 Feb 2025). MolSpectLLM differs by making spectra, molecular strings, and 3D structure coequal textual modalities inside a single generative model (Shen et al., 26 Sep 2025).
2. Modalities, textualization, and corpus construction
MolSpectLLM works entirely in text space. Every modality is converted into a structured textual representation and then tokenized. Symbolic molecular inputs include canonicalized SMILES, IUPAC names, molecular formulae, and basic properties. Spectral inputs include 1H NMR peak lists with chemical shift, multiplicity, coupling constants , integration, solvent, and spectrometer frequency; 13C NMR shifts; IR, Raman, and UV peak lists after smoothing and noise removal; and MS peak lists with , normalized intensity, and acquisition tags. Structural inputs include atom types, formal charges, aromaticity, bond connectivity and order, Cartesian coordinates , ring systems, partial charges, and pharmacophore-like features, all serialized into a textual molecular description block (Shen et al., 26 Sep 2025).
The central representational decision is to avoid raw vectors and instead convert spectra to structured text. The paper emphasizes that this removes sparsity and noise while preserving chemistry. Representative forms include tagged blocks such as \<1H_NMR>(400 MHz, CDCl3) δ 7.26 (s, 1H) ... </1H_NMR>, <IR>(500~4000) 1715(0.85) 1600(0.42) ... </IR>, and <ms_positive CE=20eV> m/z 91:100 77:65 ... </ms_positive>, produced after modality-specific preprocessing including peak picking, smoothing, noise thresholding, and metadata inclusion (Shen et al., 26 Sep 2025). This text-first strategy contrasts with spectrum encoders such as SpecFormer in MolSpectra, which tokenize spectra into patches and learn through masked patch reconstruction (Wang et al., 22 Feb 2025).
The training corpus combines several sources. The text corpus contains approximately 10 million chemistry papers. Molecular identity and 3D information come from PubChem, including SMILES, IUPAC names, formulae, metadata, and 2D/3D SDF-derived descriptors parsed by RDKit. Simulated spectra and accurate 3D structures come from QM9S, comprising about 130k small organic molecules with DFT-optimized geometries and simulated IR, Raman, and UV-Vis spectra. Additional multimodal spectroscopic supervision comes from the Alberts et al. “Unraveling molecular structure” dataset, with synthetic 1H, 13C, HSQC NMR, IR, and MS/MS spectra for about 790k molecules. Experimental NMR at scale comes from NMRBank, with approximately 225k compounds and chemical shifts extracted from more than 5.7 million publications via NMRExtractor. After filtering duplicates and low-SNR spectra, the training set comprises about 5M molecules and about 0.2M spectra (Shen et al., 26 Sep 2025).
Within the broader data landscape, this corpus aligns with a growing shift toward spectrum-rich molecular resources. QMe14S, for example, provides 186,102 small organic molecules across 14 elements and 47 functional groups, with harmonic IR, Raman, and NMR spectra, as well as tensorial quantities such as polarizability and Hessians (Yuan et al., 31 Jan 2025). This suggests that MolSpectLLM is part of a larger movement in which spectral data are becoming trainable first-class molecular signals rather than downstream evaluation artifacts.
3. Architecture and learning procedure
MolSpectLLM is a decoder-only transformer based on Qwen2.5‑7B. It retains the base model’s multi-head self-attention, rotary position embeddings, and autoregressive next-token objective, but specializes the token stream so that SMILES, spectra, and 3D descriptions are all processed by the same embedding and positional pipeline (Shen et al., 26 Sep 2025). The paper is explicit that no new encoder modules or SE(3)-equivariant GNNs are introduced. Joint representation is achieved through tags and standardized formats rather than architectural branching.
This choice has two consequences. First, cross-modal reasoning is learned entirely through token co-occurrence and long-range autoregressive dependencies. Spectral peaks, functional groups, stereochemical markers, bond patterns, and 3D motifs become mutually predictive within a single language-model state space. Second, generation is unified across tasks: the same model can be prompted with spectral text and asked to emit a SMILES string, or prompted with a SMILES string and asked to emit a 3D structure block (Shen et al., 26 Sep 2025).
Training proceeds in three phases. The first is molecular specialization pretraining, consisting of one epoch over the unified molecular textual description dataset. The second is multi-task mixed supervised fine-tuning over three epochs of instruction-formatted data covering molecule QA, SMILES→3D and IUPAC→3D, SMILES↔IUPAC conversion, Spectra→SMILES, and SMILES→Spectrum. The third is instruction-following SFT via LoRA, motivated by the observation that full-parameter fine-tuning can degrade alignment and safety; LoRA is therefore applied on template-aligned examples to improve prompt formatting, answer formatting, and instruction adherence while freezing the base weights (Shen et al., 26 Sep 2025).
The reported training setup uses a single node with 8× NVIDIA A800 GPUs, maximum sequence length 4096, effective batch size 32, and learning rate with a cosine schedule and warmup (Shen et al., 26 Sep 2025). The paper notes that all training is end-to-end within the LLM, with no separate spectra or 3D encoders.
4. Spectral reasoning and molecular elucidation
A central benchmark family concerns spectrum generation and spectra-to-structure reasoning. For SMILES→Spectra, MolSpectLLM predicts textual 13C NMR, 1H NMR, IR, and MS outputs. For Spectra→SMILES, it receives one or more spectral blocks and autoregressively emits a SMILES string. Evaluation includes token-level accuracy, sequence-level accuracy after canonicalization, and structural similarity via RDKFingerprint, topological torsion, and atom-pair Tanimoto similarity (Shen et al., 26 Sep 2025).
On spectrum-generation benchmarks, MolSpectLLM reports an average accuracy of approximately 0.53 across NMR, IR, and MS. For 13C NMR it achieves and ; for 1H NMR it achieves Jaccard , , and peak-level ; for IR it reaches cosine similarity ; and for MS cosine similarity 0 (Shen et al., 26 Sep 2025).
On Spectra-to-SMILES, MolSpectLLM achieves sequence accuracy 15.50%, token accuracy 41.65%, RDK fingerprint similarity 0.458, torsion similarity 0.359, and atom-pair similarity 0.460. The paper contrasts these numbers with large general-purpose LLM baselines such as GPT‑5, o3, and KIMI‑K2, which obtain sequence accuracy between 0 and 1.5%, token accuracy around 15–20%, and fingerprint similarity around 0.20–0.25 (Shen et al., 26 Sep 2025). The intended interpretation is not that spectra-to-structure is solved—indeed the paper stresses that the task is fundamentally ill-posed—but that large-scale multimodal pretraining on textualized spectra materially improves molecular elucidation.
| Task | Metric | Result |
|---|---|---|
| Spectrum generation | Average accuracy across NMR, IR, and MS | ~0.53 |
| Spectra-to-SMILES | Sequence / token accuracy | 15.50% / 41.65% |
| SMILES-to-3D | SDF Validity / FP similarity | 89.68% / 0.582 |
| IUPAC-to-3D | SDF Validity / FP similarity | 82.78% / 0.705 |
The model’s handling of spectral reasoning is noteworthy because it depends on standardized symbolic descriptions rather than explicit quantum-mechanical descriptors or dedicated spectral networks. Earlier UV/Vis work had shown that purely geometrical descriptors were insufficient and that electronic descriptors such as orbital energy differences, transition dipole moments, and charge-transfer distances were important for accurate learning (Armas-Morejón et al., 2022). MolSpectLLM does not incorporate those descriptors directly; instead, it learns statistical correspondences among chemical text, spectra, and structure. This suggests a different route to spectroscopic competence: not descriptor engineering, but multimodal language pretraining over chemistry-native textual abstractions.
5. 3D structure generation and end-to-end spectral pipelines
MolSpectLLM encodes 3D structures as text blocks containing atom indices, element types, Cartesian coordinates, and bond connectivity. Conceptually, the target consists of atoms 1, element types 2, coordinates 3, and bonds 4 with bond orders 5. The output can be parsed by RDKit and related tools into SDF or Mol objects (Shen et al., 26 Sep 2025).
Generation is supported from three input conditions. In SMILES→3D, connectivity is supplied explicitly by the molecular string. In IUPAC→3D, connectivity must first be inferred from the name. In spectra-driven structure generation, the pipeline proceeds as spectra→predicted SMILES→3D, all within the same LLM (Shen et al., 26 Sep 2025). The paper characterizes this as bridging spectral analysis, molecular elucidation, and molecular design.
Evaluation uses SDF Validity, Atom Clash, Bond Violation, and fingerprint similarity. On SMILES-to-3D, MolSpectLLM reports SDF Validity 89.68%, Atom Clash 2.880, Bond Violation 0.994, and fingerprint similarity 0.582. GPT‑5, by comparison, reports Validity 69.50% and FP similarity 0.314, while o3 reports Validity 45.50% and FP similarity 0.356 (Shen et al., 26 Sep 2025). On IUPAC-to-3D, MolSpectLLM reaches SDF Validity 82.78%, Atom Clash 3.012, Bond Violation 1.357, and FP similarity 0.705; GPT‑5 attains a slightly higher FP similarity of 0.813 but at lower validity and worse bond-length behavior (Shen et al., 26 Sep 2025).
The absence of explicit equivariant geometry modules is important. Contemporary 3D-aware models often rely on graph or equivariant architectures, as in MolSpectra’s TorchMD-Net-based 3D encoder aligned to UV–Vis, IR, and Raman spectra (Wang et al., 22 Feb 2025). MolSpectLLM instead learns to emit coordinates as text. This suggests that chemically usable 3D generation can emerge from autoregressive sequence modeling when trained on sufficiently standardized structure descriptions, though the paper does not claim perfect stereochemical resolution or mixture handling (Shen et al., 26 Sep 2025).
6. Position in the literature, applications, and limitations
Within the molecular foundation-model literature, MolSpectLLM occupies a distinct position. Relative to SMILES-only or text-centric molecular LLMs, it explicitly integrates experimental spectroscopy and 3D geometry; relative to 3D-aware pretraining systems such as MolSpectra, it is generative and instruction-tuned rather than encoder-only; and relative to multimodal generalist molecular LLMs such as Mol-LLM, it centers spectroscopy and 3D generation rather than graph utilization for general molecular tasks (Shen et al., 26 Sep 2025, Wang et al., 22 Feb 2025, Lee et al., 5 Feb 2025). Text-augmentation approaches such as GPT‑MolBERTa showed that chemically meaningful prose can improve downstream molecular representation learning (Balaji et al., 2023); MolSpectLLM extends the broader text-serialization idea to spectral peak lists and explicit 3D descriptions.
Potential applications listed in the paper include drug discovery and medicinal chemistry, reaction prediction and mechanism elucidation, analytical chemistry workflows, structure elucidation at scale, and materials or physical chemistry contexts where IR, Raman, or UV spectra are relevant (Shen et al., 26 Sep 2025). Related work underscores why such integration matters. QMe14S provides the kind of broad IR/Raman/NMR supervision that improves spectral ML coverage across functional groups and elements (Yuan et al., 31 Jan 2025). Bond-based polarizability learning has shown that physics-informed Raman prediction can achieve strong agreement with experiment for flexible molecules (Sowa et al., 2024). Automated mixture analysis has shown that chemical relevance and structural context materially improve spectroscopic assignment in crowded settings (Fried et al., 2024). Approximate catalog-scale frameworks such as RASCALL further show the value of functional-group-level spectral priors when exact spectra are unavailable (Sousa-Silva et al., 2019).
The paper also delineates clear limitations. Full-domain supervised fine-tuning can degrade instruction alignment and safety, which motivates the additional LoRA alignment stage. General-purpose capabilities remain weaker than those of larger closed models such as GPT‑5 and o3. Training coverage is concentrated on small-to-medium organic molecules, with limited support likely for very large molecules, organometallics, and exotic functional groups. Spectra often underdetermine stereochemistry, and the model does not claim complete resolution of complex stereochemical ambiguity or mixtures. Finally, parts of the data are mined or simulated, so label noise, simulation error, and domain bias can propagate into model behavior (Shen et al., 26 Sep 2025).
The proposed future directions are correspondingly practical: scaling model size and training data, improving instruction alignment, extending to additional spectrum types such as EPR and XAS, integrating more deeply with lab workflows, and possibly combining the language-model framework with explicit geometric models for higher 3D fidelity (Shen et al., 26 Sep 2025). This suggests that MolSpectLLM is best understood not as a terminal solution, but as a concrete multimodal formulation in which spectra, molecular strings, and geometry are learned jointly within a single autoregressive chemical model.
