ChEMBL Downloader: Reproducible Data Workflows

• chembl-downloader is a Python package for reproducible, version-pinned bulk acquisition and processing of ChEMBL data, addressing transparency in cheminformatics workflows.
• It features a layered architecture that integrates low-level FTP retrieval, mid-level parsing, and high-level utilities to manage diverse ChEMBL artifacts like SQLite dumps, SDF files, and binary fingerprints.
• The package enhances methodological transparency by enabling explicit version selection, automated caching, and scriptable dataset reconstruction for reliable data derivation.

Searching arXiv for papers on chembl-downloader and closely related ChEMBL workflow reproducibility/use cases. chembl-downloader is a Python package for the reproducible acquisition, access, and manipulation of ChEMBL data through the ChEMBL FTP server. It was introduced to address a recurrent problem in cheminformatics: many derived datasets built from ChEMBL are published without the code used to create them, leaving selection, filtering, aggregation, and related processing decisions opaque, difficult to reproduce, and rapidly outdated as new ChEMBL releases appear. In that framing, the package is infrastructure for version-pinned, scriptable, bulk ChEMBL workflows rather than a new modeling method or a formal retrieval theory contribution (Hoyt, 23 Jul 2025).

1. Motivation in reproducible cheminformatics

The package is explicitly motivated by a reproducibility failure mode in ChEMBL-derived research. When only a final derived dataset is published, without the scripts or notebooks that generated it, a reader cannot reliably reconstruct how records were selected, what assay or target criteria were applied, how duplicates were treated, how measurements were aggregated, or which identifiers were retained or discarded. The paper treats this as a methodological problem as much as a software problem, because downstream biological interpretation and model benchmarking depend heavily on such preprocessing choices (Hoyt, 23 Jul 2025).

Version staleness is a second central motivation. The paper notes that the current version of ChEMBL was v35 as of December 2024, while several widely used resources were tied to substantially older releases: ExCAPE-DB uses v20, Deep Confidence uses v23, the consensus dataset from Isigkeit et al. uses v28, and Papyrus uses v30. This matters because later ChEMBL releases may contain many more compounds, additional activities, corrected annotations, changed mappings, and altered aggregation outcomes. chembl-downloader is therefore designed to let a workflow target either the latest release or a specified historical release, so that a published dataset can be reconstructed and then refreshed against newer ChEMBL versions without changing the access mechanism (Hoyt, 23 Jul 2025).

A plausible implication is that the package’s primary contribution is methodological transparency. It does not remove the need for explicit dataset-generation logic, but it makes that logic easier to publish, re-run, and audit.

2. Layered architecture and supported ChEMBL artifacts

The software is described as lightweight and layered, with a low level for acquisition and preprocessing, a mid level for parsing and access, and a high level for a small number of task-specific helper utilities. At the low level, it uses pystow plus custom logic to retrieve ChEMBL FTP artifacts for either the latest version or a specified historical version and to perform preprocessing such as uncompressing downloaded files. At the mid level, it exposes common Python-native access routes for the resulting artifacts. At the high level, it provides reusable extractions built on top of those lower layers (Hoyt, 23 Jul 2025).

Resource category	Example formats	Access/parsing route
Relational database dumps	SQLite	sqlite3, pandas
Molecule lists	SDF, TSV	RDKit, pandas
Pre-computed molecular fingerprints	binary	chemfp
Monomer library	XML	Python xml
UniProt target mappings	TSV	pandas

The breadth of supported resource types is important because ChEMBL-based workflows often require more than one artifact class. A target-focused activity extraction may start from a SQLite dump, but structure-driven work may also need SDF files, mapping workflows may need TSV resources, and fingerprint-based screening may depend on pre-computed binary fingerprints. The package is presented as supporting reproducible acquisition of most resources on the ChEMBL FTP server, not only a single canonical database export (Hoyt, 23 Jul 2025).

The implementation philosophy is intentionally minimalist. The low- and mid-level utilities are described as small and simple so that they can be arbitrarily extended by users. This suggests an emphasis on composability over a rigid abstraction layer.

3. Access model, version pinning, and provenance

The package’s main reproducibility mechanism is explicit version selection. A workflow can fetch the latest ChEMBL release or pin itself to a given historical release, making the source snapshot a visible parameter of the analysis rather than an undocumented environmental accident. The paper treats this as the most important reproducibility feature because it allows the same extraction logic to be run on ChEMBL v23, v30, v35, or another archived release with minimal procedural change (Hoyt, 23 Jul 2025).

A second mechanism is cached acquisition through pystow. Within the paper’s framing, pystow provides organized local storage and repeatable retrieval of downloaded resources, which implies deterministic placement of files, reuse of previously fetched artifacts, reduced manual file management, and easier re-execution of scripts and notebooks. Provenance is then strengthened by making it easier to encode, in executable code, which ChEMBL release was used, which FTP resource was downloaded, which parsing route was applied, and which extraction script produced the final table (Hoyt, 23 Jul 2025).

The scope of these guarantees is explicitly limited. The package improves reproducibility of data acquisition, local bulk access, and dataset reconstruction workflows, but it does not claim formal end-to-end provenance graphs, immutable environment capture, cryptographic validation or checksum provenance, or exact byte-for-byte reproducibility across all software stacks. The paper is also explicit that users still need to publish scripts or notebooks, document filtering and aggregation choices, and pin software environments if exact computational replication is required (Hoyt, 23 Jul 2025).

Distribution and availability are described in conventional Python terms. The package is available on PyPI, the source repository is hosted at https://github.com/cthoyt/chembl-downloader, documentation is at https://chembl-downloader.readthedocs.io, and the project is archived on Zenodo with doi:10.5281/zenodo.5140463. The abstract states that it can be downloaded under the MIT license and installed from PyPI with pip install chembl-downloader (Hoyt, 23 Jul 2025).

4. Task-specific helpers and reconstruction-oriented use cases

Above the acquisition and parsing layers, the package includes a small number of task-specific helper utilities. The paper specifically mentions pre-formatted SQL queries for retrieving bioactivities associated with a given assay, retrieving bioactivities associated with a given target, retrieving compounds mentioned in a publication or patent, and retrieving the names of all compounds. These helpers reduce boilerplate for recurring bulk extraction tasks while keeping the workflow scriptable and inspectable (Hoyt, 23 Jul 2025).

The most detailed demonstration is a reconstruction of the dataset-generation workflow from Cortés-Ciriano and Bender’s Deep Confidence. The corresponding notebook, cortes-ciriano-refresh.ipynb, is described as showing that the high-level utilities can reproduce the dataset generation, highlight methodological controversies, show the immense variability introduced by new datapoints in later ChEMBL releases, and highlight the number of additional compounds added to each of the 24 target-specific datasets. The paper identifies one concrete controversy: using arithmetic mean instead of geometric mean for pIC$_{50}$ values. It further notes that Landrum and Riniker (2024) investigated the impact of such aggregation choices, reinforcing the claim that apparently minor implementation details can materially affect model inputs and conclusions (Hoyt, 23 Jul 2025).

A second case study, documented in chebi-mappings.ipynb, uses reproducible scripts plus lexical mappings from Gilda to identify missing ChEMBL–ChEBI mappings. The paper reports 4,266 potential mappings for curation in a workflow such as Biomappings. This expands the package’s relevance beyond bioactivity benchmark reconstruction to identifier harmonization, ontology alignment, and curation support (Hoyt, 23 Jul 2025).

A third demonstration consists of contributions to existing community repositories to make their code more reproducible and ChEMBL-version-agnostic. The paper lists pull requests to greglandrum/rdkit_blog (#5) for generating a substructure library, PatWalters/sfi (#11) for calculating compounds’ solubility forecast index, PatWalters/jcamd_model_comparison (#1) for comparing classification models, iwatobipen/playground (#4) for parsing and using ChEMBL’s monomer library, and iwatobipen/playground (#5) for analyzing chemical space of molecules from patents. In encyclopedic terms, these examples show the package functioning as a reproducibility substrate rather than as a closed application (Hoyt, 23 Jul 2025).

5. Position within the broader ChEMBL access ecosystem

The paper situates chembl-downloader against several existing ChEMBL access paths. ChEMBL web/API access and the Python client are presented as useful for online programmatic access but not efficient for querying and processing data in bulk, especially when workflows require repeated large-scale extraction, ad hoc local querying, and integration into reproducible data-engineering pipelines. The RDF platform is acknowledged but not framed as the easiest route for mainstream bulk dataset generation. Raw FTP access already exists, but manual download, decompression, local setup, connection, and custom parsing are described as cumbersome for most users. KNIME and Pipeline Pilot are described as relatively inflexible, inextensible, or proprietary (Hoyt, 23 Jul 2025).

A useful contrast is provided by Atreya, a Telegram-based chatbot for exploring ChEMBL. That system uses the official ChEMBL webresource client through a rule-based conversational interface, supports lookups by identifiers such as InChIKey, SMILES, ChEMBL ID, synonym, gene name, disease name, and ontology-linked tissue identifiers, and returns small, human-readable result sets, images, or CSV exports. Its access model is interactive API retrieval rather than full local-database replication, and the paper characterizes it as a proof of concept optimized for simple guided queries rather than bulk ETL or large reproducible reconstructions (Sharma et al., 2021). In that ecosystem, chembl-downloader occupies the complementary role: bulk local access, version pinning, inspectable extraction code, and refreshable dataset generation (Hoyt, 23 Jul 2025).

This distinction matters conceptually. An interactive system such as Atreya reduces the cognitive burden of fielded lookup over structured ChEMBL facts, whereas chembl-downloader reduces the operational burden of turning ChEMBL releases into reproducible, version-controlled local inputs for downstream cheminformatics and machine learning workflows.

6. Limitations, non-goals, and common misconceptions

The package is presented as a strong reproducibility enabler, but not as a complete workflow management system by itself. The paper is explicit that it does not replace the need to publish code: researchers still need to provide their scripts, notebooks, query definitions, and processing logic. It also does not eliminate the need for technical literacy, because bulk local access still assumes facility with SQLite, pandas, RDKit, SQL, and local files (Hoyt, 23 Jul 2025).

Several other non-goals are stated or implied. The paper does not describe a formal reproducibility metadata standard. It does not provide formal runtime benchmarks, throughput measurements, or complexity analyses. Its value proposition is reproducibility, convenience, version control, and practical bulk access rather than quantified performance claims. The paper excerpt also does not document a concrete CLI syntax or specific command names, so it should not be treated as a command-line tooling paper in the narrow sense (Hoyt, 23 Jul 2025).

These limits help resolve a common misconception. chembl-downloader does not guarantee that a published ChEMBL-derived dataset is reproducible merely because the package was used. Reproducibility still depends on whether the authors exposed their exact selection, filtering, aggregation, and transformation logic. The package makes those steps easier to script and share; it does not infer them automatically.

7. Relation to downstream ChEMBL-derived modeling and data integration

Several adjacent papers illustrate why a tool of this kind matters. In VECT-GAN, the authors state that they downloaded a database of approximately 2.5 million drug-like molecules from ChEMBL42, extracted molecular descriptors using RDKit, and used those descriptors to pre-train a tabular generative model. However, the paper does not specify a ChEMBL release number, a URL, the subset definition behind the “approximately 2.5 million drug-like molecules,” accession identifiers, filtering rules, or whether the source was the web interface, bulk database dump, API, or a third-party package. The same summary explicitly notes that the paper does not mention chembl-downloader at all (Abdalla et al., 15 Jan 2025). This is precisely the sort of acquisition-and-provenance gap that chembl-downloader was designed to narrow (Hoyt, 23 Jul 2025).

A second example is Multi-task Bioassay Pre-training for Protein-ligand Binding Affinity Prediction, which introduces ChEMBL-Dock. That work reports detailed ChEMBL-side filters such as STANDARD_TYPE = 'IC50' / 'Ki' / 'Kd', STANDARD_RELATION = '=', STANDARD_UNITS = 'nM', ASSAY_TYPE = 'B', TARGET_TYPE = '[SINGLE](https://www.emergentmind.com/topics/smooth-incremental-graphical-lasso-estimator-single) PROTEIN', COMPONENT_TYPE = 'PROTEIN', MOLECULE_TYPE = 'Small molecule', and BAO_FORMAT = 'BAO_0000357', and it describes subsequent UniProt mapping, PDBbind matching, RDKit conformer generation, and SMINA docking. Yet the paper does not specify the ChEMBL version or fully report duplicate handling, join logic, or several reconstruction details, and its own summary concludes that the dataset is conceptually reproducible from public data but not exactly reproducible from the paper text alone (Yan et al., 2023). A plausible implication is that version-pinned bulk access is necessary but not sufficient: downstream assay logic and integration rules must still be made explicit.

A third example comes from large-scale data integration. The byte-offset indexing study integrates PubChem, ChEMBL version 33, and eMolecules into a validated multi-source dataset and shows that, at hundred-million scale, the hard problem is no longer merely downloading ChEMBL rows but constructing an extraction and validation architecture that preserves chemical identity. That paper reports 163 unique InChIKey values with collisions affecting 326 molecular records in a PubChem-scale audit and therefore reconstructs the pipeline around full InChI rather than InChIKey (Malikussaid et al., 26 Jan 2026). In that setting, chembl-downloader addresses the upstream acquisition and versioning layer, while structure-level validation, identifier policy, and large-scale join architecture remain separate scientific and engineering responsibilities.

Taken together, these adjacent works place chembl-downloader in a specific methodological niche. It is not an assay curation schema, not an interactive front end, not a structural validation framework, and not a large-scale indexing architecture. Its role is the reproducible, scriptable, bulk acquisition and local access layer that makes those other activities easier to document, refresh, and audit.