MedBank-100k Medical Segmentation Dataset

• MedBank-100k is a large-scale, multimodal medical segmentation dataset comprising over 122k image–mask pairs across 7 imaging modalities and 21 tasks.
• The dataset uses automated preprocessing filters and standardized annotation protocols to ensure high-quality inputs and mitigate noise during model training.
• Benchmark evaluations demonstrate significant improvements with techniques like temporal adapters and confidence-driven memory mechanisms in heterogeneous clinical settings.

MedBank-100k is a large-scale, multimodal medical image segmentation corpus designed to advance the training and benchmarking of foundational “segment anything” architectures within the medical imaging domain. Developed to support generalization across diverse modalities, anatomical structures, and segmentation tasks, MedBank-100k enables rigorous pre-training, multi-task learning, and robust evaluation of models under realistic, heterogeneous clinical conditions (Yan et al., 4 Jul 2025).

1. Compilation and Composition

MedBank-100k was constructed by aggregating publicly available medical segmentation datasets over a six-month period. It encompasses 21 distinct segmentation tasks sourced from 7 imaging modalities: fundus photography, dermoscopy, X-Ray, computed tomography (CT), magnetic resonance (MR), colonoscopy, and echocardiography. The total corpus comprises 122,594 image–mask pairs, encompassing video, volumetric, and 2D data. The CT subset is dominant, accounting for 10 segmentation tasks centered on abdominal and facial organs as well as tumors, reflecting typical class imbalances observed in the field. Each singleton modality—fundus, dermoscopy, X-Ray, colonoscopy, and echocardiography—targets unique anatomical or pathological structures, enhancing the dataset’s breadth.

The distribution of frames and tasks is summarized below:

Modality	Number of Tasks	Number of Frames
Fundus	1	559
Dermoscopy	1	2,621
X-Ray	1	23,822
CT	10	34,521
MR	6	19,522
Colonoscopy	1	3,838
Echocardiography	1	1,800
Total	21	122,594

This composition ensures a broad spectrum of anatomical and technical imaging challenges, suited for large-scale model pre-training and stress-testing multimodal architectures (Yan et al., 4 Jul 2025).

2. Data Acquisition, Annotation, and Quality Control

All source data derive from recognized public repositories such as the Medical Segmentation Decathlon, ISIC skin-lesion challenges, Kaggle’s thyroid ultrasound, DRISHTI fundus, and the SPIE polyp set. Original annotations, performed by challenge experts or semi-experts, are preserved without manual re-annotation.

To standardize quality and combat annotation noise, a series of automated preprocessing filters were applied:

• Empty Masks: Discarding frames where the ground-truth mask contains only background.
• Aspect Ratios: Removal of images where the short edge is less than half the long edge, addressing anatomical distortion.
• Label Decomposition: For multi-label data (e.g., multi-organ CT), decomposing masks into single-class masks to align with SAM-style prompting.
• Temporal Structure: Randomizing 2D image ordering while preserving sequence for volumetric and video scans to allow downstream temporal model exploitation.

No multi-annotator labels are available in MedBank-100k, but rule-based normalization and model-based confidence filters—specifically the confidence-driven memory mechanism—limit the influence of noisy ground truth during model training (Yan et al., 4 Jul 2025).

3. Dataset Structure and Preprocessing Pipeline

Images are stored in 8-bit PNG or JPEG formats, retaining native spatial resolutions (commonly 256×256 to 512×512 pixels for CT/MR and higher for dermoscopy/fundus). Masks are provided as single-channel PNG files, one per class. Dataset indexing is managed with a JSON file per frame recording image/mask file paths, modality and task identifiers, and—where applicable—slice/frame indices.

Preprocessing for model input involves on-the-fly intensity normalization: pixel intensities are min–max scaled to [0, 1] for X-Ray, CT, and MR volumes by volume; dermoscopy and fundus images are RGB-scaled in the same range. No geometric resizing is performed aside from aspect ratio cropping (Yan et al., 4 Jul 2025).

4. Statistical Properties and Partitioning

The corpus exhibits a mean short-edge size of 350 ± 80 pixels and a mean long edge of 420 ± 95 pixels, reflecting realistic clinical imaging resolutions. Internal reliability is estimated by a model-based confidence module reporting predicted IoUs on held-out frames with a mean of 0.78 ± 0.12, indicating high self-assessed annotation quality.

Partitioning follows a strict frame-level 90/10 split for training and validation/testing, ensuring no patient overlap for volumetric or sequential data. Benchmarking includes ten external datasets from distinct modalities, vendors, and patient populations—such as prostate MR, brain tumor MR, and public dermoscopy/fundus benchmarks—supporting robust zero-shot generalization assessment (Yan et al., 4 Jul 2025).

5. Evaluation Protocols and Experimental Settings

Performance on MedBank-100k is measured primarily using the Dice Similarity Coefficient (DSC):

$$\mathrm{DSC}(P, G) = \frac{2\,|P \cap G|}{|P| + |G|}$$

Results are reported as averages first per task, then per modality. The loss function corresponds to the SAM-2 dual-headed objective:

$$\mathcal{L} = \mathcal{L}_{\text{mask}}(M, \hat M) + \lambda\,\mathcal{L}_{\text{IoU}}(y, \hat y)$$

where $\mathcal{L}_{\text{mask}}$ denotes binary cross-entropy, and $\mathcal{L}_{\text{IoU}}$ is squared error on predicted IoU, with $\lambda = 1.0$.

Training leverages AdamW (initial learning rate $1 \times 10^{-4}$, weight decay $1 \times 10^{-2}$), batch size 16, for 100 epochs, using random flips and ±5% intensity jitter augmentation, on a single NVIDIA H100 GPU (Yan et al., 4 Jul 2025).

6. Benchmark Findings and Analysis

On internal validation across all 21 tasks, models trained on MedBank-100k exhibit:

• SAMed-2: DSC 0.7118 (average across tasks)
• MedSAM: DSC 0.6247
• SAM-2: DSC 0.3954
• Per-task U-Net: DSC 0.6516

External zero-shot evaluation across ten held-out benchmarks yields a SAMed-2 DSC of 0.6938, a 10.5% gain over MedSAM and marginally ahead of per-task U-Net (0.6879).

Representative per-task performance is summarized as follows:

Task	SAMed-2 DSC	MedSAM-2 DSC	SAM-2 DSC	U-Net (per task)
Spleen (CT)	0.8566	0.8260	0.4441	0.5239
Inferior Vena Cava (CT)	0.7800	0.6821	0.1738	0.5471
Optic Cup (Fundus)	0.8971	0.4040	0.6209	0.8073
Melanoma (Derm)	0.9119	0.6877	0.5293	0.8169

Ablation studies confirm that temporal adapters alone boost challenging organ segmentation (e.g., spleen DSC by +8.92%); the confidence-driven memory mechanism further increases mean performance by +1.1%. Optimal external generalization occurs with a memory base size of 640 entries, and confidence + cosine-similarity-based retrieval outperforms random schemes by ∼1.5% on external benchmarks. Few-shot transfer is effective: with just four annotated examples, DSC on a novel prostate MR task improves from 0.65 (zero-shot) to above 0.75. In a user study, SAMed-2 reduced expert cardiac MR annotation time from 609 ± 137s to 75.4 ± 18.2s per frame (−87.6%), highlighting practical gains (Yan et al., 4 Jul 2025).

7. Limitations and Prospects

MedBank-100k offers extensive modality and task diversity, enabling systematic validation of foundational and memory-augmented segmentation models. However, its reliance on single-annotator masks precludes direct inter-rater agreement computation, and several tasks remain under-represented (<2,000 frames), introducing potential modality bias.

Recommendations for future directions include:

• Enriched annotation protocols: Multi-rater adjudication or algorithmic consistency checking to quantify and control label uncertainty.
• Balanced dataset expansion: Increasing sample sizes for sparse modalities (e.g., echocardiography, thyroid ultrasound) to mitigate class imbalance and enhance data efficiency.

These properties position MedBank-100k as a benchmark resource for development, pre-training, and reproducible evaluation of medical image segmentation foundation models with open-source infrastructure and standardized protocols (Yan et al., 4 Jul 2025).