ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding

Abstract: While Vision-LLMs (VLMs) have demonstrated significant potential in chemical visual understanding, current models are predominantly optimized for direct visual question-answering tasks. This paradigm often results in "black-box" systems that fail to utilize the inherent capability of LLMs to infer underlying reaction mechanisms. In this work, we introduce ChemVLR, a chemical VLM designed to prioritize reasoning within the perception process. Unlike conventional chemical VLMs, ChemVLR analyzes visual inputs in a fine-grained manner by explicitly identifying granular chemical descriptors, such as functional groups, prior to generating answers. This approach ensures the production of explicit and interpretable reasoning paths for complex visual chemical problems. To facilitate this methodology, we implement a cross-modality reverse-engineering strategy, combined with a rigorous filtering pipeline, to curate a large-scale reasoning-and-captioning dataset comprising 760k high-quality samples across molecular and reaction tasks. Furthermore, we adopt a three-stage training framework that systemically builds model perception and reasoning capacity. Experiments demonstrate that ChemVLR achieves state-of-the-art (SOTA) performance, surpassing both leading proprietary models and domain-specific open-source baselines. We also provide comprehensive ablation studies to validate our training strategy and data generation designs. Code and model weights will be available at https://github.com/xxlllz/ChemVLR.

• The paper introduces a reasoning-centric paradigm that leverages visual traversal and semantic anchors to generate interpretable mechanistic rationales.
• The paper employs a three-stage training strategy—continual pre-training, supervised fine-tuning, and reinforcement learning—to achieve a ~9% boost in complex task performance.
• The paper demonstrates state-of-the-art performance in molecular recognition and reaction prediction, aligning its precision with specialist chemical models.

ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding

Motivation and Conceptual Framework

ChemVLR introduces a reasoning-centric paradigm for chemical Vision-LLMs (VLMs), diverging from conventional direct-answering protocols toward explicit, interpretable visual reasoning. The rationale is anchored in the limitations of prior VLMs: most models operate in a "black-box" fashion—producing answers without granular analysis of chemical descriptors or mechanistic inference—thus neglecting LLMs' capacity for domain-specific reasoning. ChemVLR addresses this by integrating a fine-grained traversal of chemical substructures before answer generation. This methodological shift is operationalized by equipping ChemVLR with a visual traversal mechanism that identifies functional groups and reaction centers, ensuring the generation of detailed reasoning paths.

Cross-Modality Reverse-Engineering for Dataset Construction

Data scarcity, particularly in reasoning-annotated vision datasets, has stymied progress in the field. ChemVLR tackles this bottleneck via a cross-modality reverse-engineering strategy, synthesizing reasoning traces from textual queries and answers using advanced LLMs. The pipeline incorporates semantic anchors—retrieved IUPAC names, RDKit-functional groups, and curated expert demonstrations—to bias the reasoning process toward interpretability and chemical accuracy. Rigorous filtering is applied using structural checks, consistency validation, and external LLM assessments, resulting in a dataset of 760k high-quality samples distributed across captioning, molecular recognition, and reaction prediction tasks.

Figure 1: ChemVLR's cross-modality reverse-engineering pipeline for high-quality reasoning and caption data, integrating semantic anchors and multistage filtering.

Empirically, the integration of IUPAC and RDKit-derived anchors substantially increases retention rates of high-fidelity reasoning traces, surpassing benchmarks constructed from solely SMILES input. This augmentation catalyzes effective visual-grounded reasoning synthesis and aligns the data modality with the domain structure required for robust model training.

Progressive Three-Stage Training Strategy

ChemVLR employs a three-stage training protocol—Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Reinforcement Learning (RL)—to systematically instill chemical perception and step-wise reasoning.

• CPT Stage: Visual encoders and projectors are aligned to the chemical domain while freezing the LLM backbone, leveraging high-quality caption datasets to bridge the modality gap.
• SFT Stage: The model is trained on reasoning and instruction data with structured formatting (> , <answer>, <SMILES>, <IUPAC> tags), enhancing instruction-following and chain-of-thought output generation.

  • RL Stage: Non-trivial, solvable instances are selected for further policy optimization using Decoupled Clip and Dynamic Sampling Policy Optimization (DAPO). Composite rewards are enforced, combining format compliance and structural accuracy (Tanimoto similarity, exact string matching).

    Figure 2: ChemVLR's progressive training framework: CPT for domain alignment, SFT for reasoning capacity, RL for policy refinement on curated tasks.

    

    Figure 3: RL reward curve during DAPO optimization—marked performance surge evidences emergent reasoning phase.

  Ablation studies reveal CPT and RL stages are critical for domain perception and robust reasoning: SFT-only baselines are outperformed by CPT+SFT, with RL providing a further ~9% boost in complex tasks.

  Quantitative and Qualitative Evaluation

  ChemVLR achieves SOTA performance across molecular and reaction benchmarks (MMChemOCR, img2smiles, ChemRxn-V Recognition and Prediction), outstripping both proprietary generalist models (Gemini-3-Flash, GPT-5-mini, GPT-4o) and specialized chemical VLM baselines (ChemVLM-8B, ChemDFM-X, TinyChemVL). Notably, ChemVLR-8B matches the precision of specialist SMILES OCR models while simultaneously generating explicit functional group analysis and mechanistic rationales.

  Figure 4: Case showcase: ChemVLR-8B outperforms baselines in both molecular recognition and reaction prediction, providing granular mechanistic reasoning.

  A detailed error analysis demonstrates ChemVLR-8B's superiority in mechanistic deduction: it avoids over-interpretation and irrelevant pathway construction (typical of insufficiently specialized SFT or generalist models), correctly identifies thermodynamic termination points in reactions, and exhibits chemoselectivity in recognizing inert functional groups.

  Data Composition and Reward Design: Empirical Insights

  Ablations on SFT data confirm the necessity of diverse modality coverage: inclusion of molecule-to-IUPAC conversion and reasoning data is essential for surmounting performance plateaus inherent in SMILES-only instruction. The structural identity reward (Tanimoto == 1.0) for RL yields maximal improvement, outperforming continuous similarity and naive string matching, as it enforces chemical equivalence.

  Implications and Future Directions

  ChemVLR's explicit step-wise reasoning advances the interpretability and accuracy of chemical VLMs, addressing the demand for scientific assistant models capable of complex multimodal problem-solving. Practically, this enables deployment in molecular recognition, reaction prediction, and potentially synthetic planning, contingent on continued improvements in reasoning data generation and error refinement.

  Future research should focus on:

  • Refining filtering mechanisms to eliminate residual reasoning artifacts,
  • Extending reasoning annotation to tasks like property prediction using code-centric or formal logical representations,
  • Broadening the dataset's coverage to encompass real-world scenarios (e.g., K-12 educational contexts, experimental laboratory reports),
  • Exploring transfer and adaptation to other scientific modalities via analogous cross-modality synthesis and progressive training.

  Conclusion

  ChemVLR establishes a robust reasoning baseline for multimodal chemical vision-language understanding, combining cross-modality reverse-engineering, semantic anchoring, and progressive staged training. These methodological advancements yield significant improvements in performance, interpretability, and utility, positioning ChemVLR as a foundational model for future multimodal chemical AI research and application (2604.06685).