MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification (2110.14795v2)

Abstract: We introduce MedMNIST v2, a large-scale MNIST-like dataset collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D. All images are pre-processed into a small size of 28x28 (2D) or 28x28x28 (3D) with the corresponding classification labels so that no background knowledge is required for users. Covering primary data modalities in biomedical images, MedMNIST v2 is designed to perform classification on lightweight 2D and 3D images with various dataset scales (from 100 to 100,000) and diverse tasks (binary/multi-class, ordinal regression, and multi-label). The resulting dataset, consisting of 708,069 2D images and 10,214 3D images in total, could support numerous research / educational purposes in biomedical image analysis, computer vision, and machine learning. We benchmark several baseline methods on MedMNIST v2, including 2D / 3D neural networks and open-source / commercial AutoML tools. The data and code are publicly available at https://medmnist.com/.

• The paper introduces MedMNIST v2 as a benchmark with 18 diverse 2D/3D datasets to standardize evaluation in biomedical image classification.
• It employs pre-processed, uniformly sized images, facilitating rapid prototyping with models like 2D/3D ResNets and AutoML tools.
• The benchmark’s diversity and standardized splits promote fair comparisons and accelerate advances in machine learning for healthcare.

MedMNIST v2: A Benchmark for Biomedical Image Classification

The paper introduces MedMNIST v2, an extension of the MedMNIST v1 benchmark, designed for the classification of 2D and 3D biomedical images. This comprehensive dataset aims to provide a standardized, lightweight, and easily accessible resource for evaluating machine learning models in the context of biomedical image analysis.

Dataset Overview

MedMNIST v2 comprises 18 datasets, with 12 dedicated to 2D image classification and 6 to 3D image classification. These datasets are derived from primary biomedical imaging modalities, such as X-ray, CT scans, and microscopy. The images are pre-processed to small, uniform dimensions of $28 \times 28$ for 2D and $28 \times 28 \times 28$ for 3D, to facilitate rapid evaluation without the need for extensive computational resources.

Key Features

• Diversity: The dataset encapsulates a wide variety of tasks including binary/multi-class classification, ordinal regression, and multi-label classification. This diversity reflects real-world biomedical imaging challenges and is conducive to testing model generalizability.
• Standardization: Each dataset within MedMNIST v2 is pre-processed into a consistent format, eliminating the need for extensive preparatory work. Standardized train-validation-test splits are provided, encouraging fair comparisons.
• Lightweight Nature: With small image sizes, MedMNIST v2 allows for the quick evaluation of models, making it suitable for exploratory research and educational purposes.
• Educational Value: The dataset is accessible under a Creative Commons license, aimed at fostering interdisciplinary education and research in biomedical imaging, computer vision, and machine learning domains.

Experimental Validation

The paper benchmarks several baseline models against MedMNIST v2, employing methods such as ResNet architectures, both 2D and 3D, alongside AutoML tools like auto-sklearn and AutoKeras. These experiments underscore the efficacy of MedMNIST v2 in evaluating the generalization capabilities of various machine learning frameworks in handling biomedical data.

• 2D Results: The ResNet models, along with AutoML tools, yield competitive results across the 2D datasets. Notably, Google AutoML Vision demonstrates high average performance, although ResNet models remain strong contenders in terms of accuracy.
• 3D Results: ResNet architectures adapted for 3D convolutions outperform other models, indicating their robustness in dealing with volumetric data.

Implications and Future Directions

MedMNIST v2 offers a valuable resource for researchers aiming to benchmark machine learning algorithms in the biomedical domain. Its lightweight nature and standardized formats make it particularly useful for educational purposes and rapid prototyping of models. The inclusion of diverse tasks and modalities ensures that the benchmark remains relevant across various application scenarios in biomedical imaging.

Future expansions may involve the inclusion of additional modalities and tasks, enhancing the dataset’s utility in emerging research areas such as federated learning, domain adaptation, and interpretable AI. Moreover, MedMNIST v2 paves the way for the development of novel AutoML techniques, specifically tailored to the unique challenges posed by medical imaging.

Overall, MedMNIST v2 facilitates a broad spectrum of research and educational activities, driving forward the integration of AI in healthcare.