---
title: PMC-15M Biomedical Image–Text Dataset
url: https://www.emergentmind.com/topics/pmc-15m-dataset
type: topic
---

# PMC-15M Biomedical Image–Text Dataset

PMC-15M is a large-scale biomedical image–text dataset comprising more than 15 million figure–caption pairs extracted from the full-text open-access biomedical literature. Collected by parsing 4.4 million articles from the PubMed Central Open-Access Subset (PMC-OA), PMC-15M offers more than two orders of magnitude greater data volume than previous biomedical multimodal corpora and encompasses every major modality in contemporary biomedical imaging and illustration. The dataset constitutes the foundation for BiomedCLIP, a multimodal biomedical foundation model designed for vision–language processing across diverse biomedical domains [2303.00915].

## 1. Data Source and Extraction Pipeline

PMC-15M is derived from the PMC-OA corpus, which, as of June 15, 2022, consisted of 4.4 million open-access, full-text biomedical articles. Articles were downloaded from the NCBI PMC FTP service and parsed into per-article directories containing XML, PDF, media, and supplementary files.

For figure–caption extraction, the pipeline utilized the PubMed Parser tool (Achakulvisut et al., JOSS 2020). The parser targeted all `<fig>` XML blocks, extracting figure file references, the associated `<caption>` text, and provenance metadata (PMID/PMCID). Articles lacking figure references or failing XML parsing (malformed or missing fields) were excluded. No additional manual or subject-specific filtering was performed. All large-scale parsing and preprocessing operations were executed on Azure Databricks with Apache Spark.

## 2. Dataset Composition and Structure

Following normalization and filtering, PMC-15M contains 15,282,336 (image, caption) pairs drawn from over 3 million distinct source articles. The dataset reflects the heterogeneous imaging landscape of biomedical science:

- Nearly 50% of figures are multipanel or “compound” images.
- For modality frequency estimation, compound figures were further split into individual panels, creating the “PMC–Fine-Grained–46M” set with approximately 46 million panels, classified by nearest-keyword embedding.
- Image modalities span more than 30 subfields, including:
  - Statistical and scientific illustrations (graphs, charts, plots, tables, chemical diagrams)
  - Radiology (X-ray, CT, MRI, PET, ultrasound)
  - Microscopy and pathology (light microscopy, electron microscopy, fluorescence/molecular microscopy)
  - Additional domains (flow cytometry, electrophoretic gels, anatomical diagrams)

A summary of the core statistics is given below:

| Statistic                                    | Value                   |
|----------------------------------------------|-------------------------|
| Total source articles (PMC-OA)               | 4.4 million             |
| Articles with at least one figure            | ≈3.1 million            |
| Image–caption pairs in PMC-15M               | 15,282,336              |
| Panels in PMC–Fine-Grained–46M               | 46,000,000 (approx.)    |
| Predefined splits (train/dev/test)           | 13.9M / 13.6K / 726K    |
| Maximum caption length (90th percentile)     | ≈256 tokens             |
| Image size (90th percentile)                 | >336 pixels             |
| Major image types                            | 30+ subfields           |

## 3. Text–Image Pairing and Preprocessing

Caption text was captured verbatim from the XML `<caption>` blocks, with no further language-model–based rewriting or additional cleaning. Pairing follows the intrinsic structure of the XML, in which each `<figure>` element references specific image files plus one caption block. Consequently, the mapping of image to caption is strictly 1:1, defined by XML identifiers, requiring no probabilistic or similarity-based alignment heuristics.

For downstream model training, preprocessing included:

- **Image preprocessing:** Images resized to 224×224 pixels using RandomResizedCrop (as in CLIP); normalized with ImageNet-style mean=(0.48145, 0.45783, 0.40821) and std=(0.26863, 0.26130, 0.27578); gradient checkpointing and bfloat16 mixed-precision enabled scalable large-batch, multi-GPU training.
- **Text preprocessing:** Tokenization with WordPiece (SentencePiece) and a 30K-token vocabulary; maximum context window set to 256 tokens, covering >90% of all figure captions (default CLIP uses 77).
- **Quality control:** Only articles with missing or invalid XML were excluded; no duplicate image filtering or manual review was applied.

## 4. Data Splits, Licensing, and Access

PMC-15M includes predefined splits designed for cross-modal retrieval and benchmark evaluation:

- **Training:** 13,892,001 pairs
- **Development:** 13,600 pairs
- **Test:** 725,739 pairs

The source (PMC-OA) is available under the National Library of Medicine’s open-access policy. The dataset’s creators provide scripts to reproduce PMC-15M from the public FTP repository and release pretrained models and code at https://aka.ms/biomedclip. Usage restrictions are limited to compliance with standard PMC-OA citation guidelines.

## 5. Dataset Analysis and Coverage

Analysis of figure and caption properties reveals several characteristics:

- **Image resolution:** More than 75% of biomedical figures exceed 336 pixels in at least one dimension, in contrast to the 224×224 size used for model input.
- **Caption length:** The median caption length is substantially greater than default CLIP’s 77-token context window, which motivated the expansion to 256 tokens in BiomedCLIP.
- **Modality diversity:** The 30 most frequent image types range from generic statistical charts to radiology modalities (X-ray, MRI, ultrasound) and advanced microscopy, confirming comprehensive cross-field representation.

## 6. Limitations and Biases

Explicitly discussed limitations include:

- The dataset does not capture in-line textual references (“citances”), only figure captions.
- Approximately 50% of images are compound multipanel figures; unless split, such figures provide reduced panel-level granularity.
- XML parsing errors and missing files resulted in approximately 1 million lost figures, potentially introducing bias toward higher-quality or simpler figure formats.
- PMC-OA is known to overrepresent basic-science and review articles relative to closed-access clinical publications, leading to underrepresentation of niche or closed clinical imaging modalities.

A plausible implication is that although PMC-15M spans the major image types of biomedical research, certain clinical specialties reliant on closed-access literature may be less well-covered.

## 7. Research Applications and Reproducibility

PMC-15M is designed to enable large-scale pretraining of domain-adapted vision–language models, evidenced by its use to pretrain BiomedCLIP, which reports state-of-the-art performance on retrieval, classification, and VQA across standard biomedical imaging benchmarks, including outperformance of radiology-specific models such as BioViL on RSNA pneumonia detection. All code and pretrained model weights are publicly released, and the dataset construction process is fully reproducible from the PMC-OA XML with provided scripts, facilitating further research in multimodal biomedical AI [2303.00915].

Source: https://www.emergentmind.com/topics/pmc-15m-dataset