MolBench: Drug Discovery Benchmark

• MolBench is a benchmark suite for drug discovery that evaluates multi-tool workflows and hierarchical planning across screening, optimization, and end-to-end tasks.
• It measures complex aspects such as error recovery, conditional branching, and iterative optimization using real assay data and deterministic metrics.
• Empirical findings reveal that hierarchical skills significantly boost performance in workflow-heavy tasks compared to ad hoc scripting approaches.

Searching arXiv for MolBench and closely related benchmark papers to ground the article in recent literature. MolBench is a benchmark suite for drug discovery agents introduced in "MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization" (Zhang et al., 2 Apr 2026). It is defined as “a multi-dimensional evaluation suite comprising three complementary tiers. MolBench-MS (Molecular Screening)… MolBench-MO (Molecular Optimization)… MolBench-E2E (End-to-End Discovery)… spanning 8–50+ sequential tool calls.” Its central purpose is to evaluate drug discovery agents rather than standalone models or isolated algorithms, with particular emphasis on whether an AI system can orchestrate real tools in realistic computational drug-discovery workflows. In the MolClaw formulation, MolBench is explicitly designed to stress-test “workflow orchestration competence,” which is treated as the primary capability bottleneck for AI-driven drug discovery (Zhang et al., 2 Apr 2026).

1. Definition and rationale

MolBench was introduced to address a gap in the evaluation of LLM-plus-tool systems for chemistry and drug discovery. Existing work such as ChemCrow, Biomni, DrugAgent, and TxAgent is described as focusing on single-step tool calls or short scripts, narrow tasks such as text QA, retrieval, ML programming, or simple property prediction, and lacking systematic, multi-dimensional benchmarks for agents that must plan over long horizons, call many heterogeneous tools, handle error recovery, file management, and quality control, and balance multiple objectives over multiple rounds (Zhang et al., 2 Apr 2026). MolBench is therefore not a property-prediction benchmark in the conventional sense. Its intended object of measurement is agentic execution quality under realistic workflow constraints.

This design choice makes MolBench diagnostic rather than merely score-aggregating. The benchmark is constructed so that some tasks are intentionally simple controls, while others require extended planning, conditional branching, fallback logic, and iterative optimization. This suggests that MolBench is meant to separate basic tool accessibility from higher-order orchestration ability. In the MolClaw study, that distinction is made explicit: gains from hierarchical skills concentrate on tasks that demand structured workflows and disappear on tasks solvable with ad hoc scripting (Zhang et al., 2 Apr 2026).

2. Task hierarchy and problem classes

MolBench covers three major stages of early-stage drug discovery: molecular screening, molecular optimization, and end-to-end discovery. Each tier contains multiple task types with different input modalities, objective structures, and workflow lengths (Zhang et al., 2 Apr 2026).

Tier	Focus	Representative tasks
MolBench-MS	Ranking and filtering molecules	Property filtering, binding affinity comparison, docking screening
MolBench-MO	Improving molecules for properties	Molecule editing, physicochemical property optimization
MolBench-E2E	Long-horizon multi-tool workflows	E2E-Q1, E2E-Q2, E2E-Q3

MolBench-MS contains three subtasks. Molecular Property Filtering asks for the subset of SMILES satisfying explicit numeric constraints such as MW, LogP, HBD/HBA, TPSA, rotatable bonds, ring count, aromatic ring count, fraction sp$^3$, heavy atom count, and heteroatom count. It is described as conceptually simple, typically solvable via 1–2 RDKit calls per query, and functions as a control task. Binding Affinity Comparison presents a target protein name and two SMILES in a matched pair and asks which molecule has higher or lower affinity against the same target, based on experiment-derived $K_i$, with “higher affinity” and “lower affinity” prompts sampled with equal probability. Molecular Docking Screening asks for a ranked list of 60 candidate SMILES for a given target, using end-to-end docking and evaluating whether true actives appear near the top via Hits@3 (Zhang et al., 2 Apr 2026).

MolBench-MO contains two subtasks derived from ChemCoTBench, with ill-defined tasks removed and target-specific optimization excluded because LLMs tended to recall known strategies from training. Molecule Editing is a constrained medicinal-chemistry editing task: given a starting SMILES and textual instructions for functional group addition, deletion, or substitution, the system must return a single SMILES satisfying all edit constraints. Physicochemical Property Optimization is a one-pass optimization task over QED, LogP, and LogS, in which the model must propose optimized molecules that improve the specified properties and satisfy pre-set criteria (Zhang et al., 2 Apr 2026).

MolBench-E2E is the long-horizon tier and is the most distinctive part of the suite. E2E-Q1 requires coarse-grained conformational sampling plus all-atom reconstruction for the EGFR kinase domain (PDB 1M17, chain A), using two different coarse-grained force fields, GoCa and OpenAWSEM, followed by extraction of representative frames and rebuilding of 20 all-atom structures via PULCHRA. E2E-Q2 is a five-round QED-driven optimization loop on a fixed triazolo-benzodiazepine scaffold with targets $\text{QED} \ge 0.70$ and Tanimoto similarity $\ge 0.40$ to the starting molecule. E2E-Q3 is a structure-guided iterative lead-optimization task for EGFR/Erlotinib, with a maximum of 15 rounds and a success target of improving the QuickVina docking score by at least $2$ kcal/mol relative to baseline for at least two molecules, i.e.

$\Delta \text{Score} \le -2.0 \text{ kcal/mol}.$

These tasks require 8–50+ tool calls and encode explicit workflow phases, convergence rules, and strategy pivots (Zhang et al., 2 Apr 2026).

3. Construction principles and workflow complexity

MolBench-MS is built entirely from real assay data. Property filtering and similarity come from the CARA lead optimization subset; binding affinity comes from ACNet $K_i$ pairs; docking screening comes from the CARA virtual screening subset. Sampling is tightly controlled. For filtering and similarity, each assay contributes 10 molecules, 11 descriptors are computed, constraints are set using quantile-based thresholds, and samples are resampled until 1–5 valid molecules remain. Binding Affinity Comparison uses 37 unique targets, uniformly sampled across target IDs, with one pair per target. Docking Screening uses 25 assays, each with at least 60 molecules, at least 6 actives with pChEMBL $\ge 6$, and at least 50 inactives with pChEMBL $< 6$ (Zhang et al., 2 Apr 2026).

MolBench-MO is derived from ChemCoTBench with ambiguous or ill-defined items removed. The authors explicitly exclude the target-specific optimization subset because it encouraged recall rather than novel reasoning. Editing tasks are phrased in natural language but scored by exact satisfaction of all edit constraints. Optimization tasks specify property changes and are evaluated by property improvement and threshold success rather than by textual plausibility (Zhang et al., 2 Apr 2026).

MolBench-E2E is organized around phase-structured workflows rather than free-form prompts. E2E-Q1 is a fixed linear sequence from retrieval and cleaning through coarse-grained setup, simulation, extraction, and reconstruction. E2E-Q2 instantiates an explicit Assess→Diagnose→Design→Verify loop over up to five rounds, with early stopping if $\text{QED} \ge 0.70$ and convergence declaration if there is no significant improvement for consecutive rounds. E2E-Q3 contains baseline establishment, docking-box locking, modification-map definition, iterative generation and docking, SAR reasoning, and a strategy pivot when best score stagnates for three rounds. Approximate tool-call counts are 8–12 distinct tool invocations for E2E-Q1, 12–15 for E2E-Q2, and 20–50+ for E2E-Q3. Figures described in the paper emphasize tool failures, recovery actions, decision points annotated with skill layer, and verification checkpoints, underscoring that MolBench measures branching and recovery behavior rather than one-shot execution (Zhang et al., 2 Apr 2026).

This construction makes MolBench unusual among molecular benchmarks. Its task instances are not only chemically grounded but operationally specified: success criteria, fallback logic, stopping rules, and output organization are benchmarked alongside domain correctness. A plausible implication is that MolBench treats workflow form as part of the scientific task, not merely as an implementation detail.

4. Tool substrate and execution environment

MolBench is built on the same tool substrate as MolClaw, which integrates more than 30 specialized resources. The benchmark therefore presupposes access not just to chemistry knowledge but to a heterogeneous computational environment. Protein-structure and ensemble tools include ESMFold, Chai-1, GoCa, OpenAWSEM, PULCHRA, AttnPacker, and BioEmu. Pocket-detection tools include fpocket and P2Rank. Docking and binding-evaluation tools include Vina-GPU/QuickVina, DiffDock, KarmaDock, HDOCK, EquiScore, and Boltz-2. Molecular generation and editing use REINVENT4, RDKit tools, and a SMILES FG editor skill. MD and free-energy tools include GROMACS, OpenMM, and gmx_MMPBSA. ADMET and property calculation rely on ADMET-AI, DLEPS, and RDKit. Cheminformatics and similarity utilities include RDKit, Open Babel, and Morgan fingerprints, while databases include UniProt, RCSB PDB, AlphaFold DB, PubChem, and ChEMBL (Zhang et al., 2 Apr 2026).

Tool usage differs by tier. MolBench-MS mainly exercises RDKit and docking tools. MolBench-MO relies primarily on RDKit, and may additionally invoke generation or editing tools. MolBench-E2E more heavily uses specific workflow stacks: GoCa, OpenAWSEM, and PULCHRA for E2E-Q1; REINVENT4, RDKit, and ADMET-AI for E2E-Q2; and PDBFixer, QuickVina, REINVENT4, and related utilities for E2E-Q3 (Zhang et al., 2 Apr 2026).

All tools are exposed via the Science Context Protocol (SCP), described as MCP-compatible. Benchmark execution therefore assumes an SCP server with required tools deployed, GPU resources for heavy tools, and standardized JSON schemas for inputs and outputs. The benchmark is not merely a collection of static questions; it is coupled to a runtime environment. This dependency is also one of its explicit limitations, because tool failures and deployment quality can influence measured performance (Zhang et al., 2 Apr 2026).

5. Metrics, protocol, and empirical findings

MolBench-MS uses deterministic task metrics. Molecular Property Filtering is evaluated by Accuracy (%) and F1 score (%), with true and false positives and negatives defined by exact SMILES matching against ground truth. Binding Affinity Comparison is scored by Accuracy (%) over 37 pairs. Molecular Docking Screening uses average Hits@3, where for each assay

$K_i$0

averaged across 25 assays. MolBench-MO uses Accuracy (%) for Molecule Editing, and for Physicochemical Property Optimization it uses Delta, the numerical improvement in the objective, together with Success Rate (SR), the fraction of tasks meeting pre-defined thresholds. MolBench-E2E uses rubric-based normalized scores in $K_i$1, based on weighted criteria such as tool selection and sequence, scientific validity of reasoning, data integrity and parameter consistency, termination behavior, and completeness of outputs. Each criterion is scored 0–2 by three independent raters, weighted, summed, and normalized, and inter-rater reliability is checked with Krippendorff’s $K_i$2 (Zhang et al., 2 Apr 2026).

The evaluation benchmark is purely held-out assessment: no train/validation/test split is described. Task counts are 50 queries for property filtering, 37 pairs for binding affinity comparison, 25 targets for docking screening, 39 tasks for molecule editing, and exactly 3 rubric-scored E2E tasks. The evaluated methods include eight standalone LLMs—GPT 5.2, Claude Sonnet 4.6, Gemini 3, DeepSeek v3.2, Qwen 3.5, Kimi 2.5, GLM 5, and Minimax 2.5—and several agentic systems: Biomni, Claude Code, OpenClaw, MolClaw-CC, and MolClaw-OC (Zhang et al., 2 Apr 2026).

The main empirical finding is a sharp difficulty gradient. On simple, scriptable tasks such as property filtering and molecule editing, vanilla agents and MolClaw are both near ceiling, and hierarchical skills add little. On complex workflow-heavy tasks, the effect is large. In Binding Affinity Comparison, vanilla agents score 51.4% on both platforms, while MolClaw-CC reaches 81.1% and MolClaw-OC 73.0%; the gain over Claude Code vanilla is +29.7 percentage points with $K_i$3 and Cohen’s $K_i$4. In Docking Screening, Hits@3 rises from 0.56 to 0.80 on Claude Code and from 0.20 to 0.64 on OpenClaw. In Physicochemical Property Optimization, the optimization delta on Claude Code increases from 0.866 to 1.724, while SR reaches 100% for MolClaw on both runtimes. E2E results are also reported as strong: MolClaw reconstructs 20 all-atom structures in E2E-Q1, reaches $K_i$5 by round 4 in E2E-Q2, and in E2E-Q3 improves Erlotinib’s baseline docking score from $K_i$6 to $K_i$7 kcal/mol twice via two distinct binding modes, generating and docking 54 molecules with 100% scaffold retention; MolClaw receives 100/100 on the Q3 rubric (Zhang et al., 2 Apr 2026).

These results are used to support a specific interpretation: MolBench reveals that workflow complexity is the axis along which hierarchical skills matter most. The benchmark is therefore both evaluative and explanatory. It does not merely rank systems; it identifies workflow orchestration competence as the operative differentiator.

6. Benchmark position, external usage, and limitations

MolBench occupies a different niche from several similarly named or neighboring molecular benchmarks. MolLangBench evaluates the molecule–language interface through deterministic tasks in molecular structure recognition, editing, and generation, including SMILES and image modalities, rather than multi-tool drug-discovery workflows (Cai et al., 21 May 2025). MolViBench evaluates “Molecular Vibe Coding,” namely RDKit-based executable program synthesis across 358 curated tasks and five cognitive levels, again focusing on code generation rather than agentic orchestration across deployed scientific tools (Li et al., 4 May 2026). TOMG-Bench targets open-domain text-guided molecule editing, optimization, and customized generation with RDKit-based automatic evaluation, emphasizing open-ended structure production rather than long-horizon workflow execution (Li et al., 2024). These comparisons clarify that MolBench is specifically a benchmark for drug discovery agents operating over heterogeneous tools and multi-stage pipelines, not a general chemistry-language or chemistry-coding benchmark.

MolBench has already appeared in later agent evaluations. "Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent" reports a MolBench-Bind result, defined there as the score of Binding Affinity Comparison following the official evaluation setting and averaging over three repeated runs; Agents-A1 obtains 56.8 on MolBench-Bind (Bai et al., 29 Jun 2026). This indicates that the binding-affinity sub-benchmark has begun to function as a reusable scientific-agent evaluation target beyond the original MolClaw study.

The benchmark’s limitations are explicit. Coverage is narrow at the E2E level: only three end-to-end tasks, all in small-molecule structure-based contexts, with no GPCRs, ion channels, proteases, or biologics-only tasks. E2E workloads are computationally expensive and depend on a particular SCP deployment. Tool failures can influence outcomes. Residual bias and memorization remain possible, even though MolBench-MO excludes target-specific tasks to reduce recall-based behavior. The benchmark is also static and does not capture learning over repeated runs (Zhang et al., 2 Apr 2026).

Future directions named by the authors include extending MolBench-E2E to more targets and modalities such as selectivity profiling, cryptic pocket discovery, MM-PBSA-guided optimization, and protein–protein modulators; adding tasks that more explicitly test multi-objective optimization with Pareto-front reasoning and formal convergence criteria from Bayesian optimization and surrogate modeling; and developing automated blind-spot alarms, for example on ADMET endpoints, to incorporate into evaluation (Zhang et al., 2 Apr 2026). In that sense, MolBench is best understood as a fully specified benchmark suite for workflow-centric drug discovery agents whose main contribution is to redefine molecular AI evaluation around orchestrated scientific execution rather than isolated prediction or isolated tool use.