MedRegion-500k: Region-Annotated Medical Dataset

• MedRegion-500k is a large-scale, region-annotated multimodal dataset comprising 500k image-text pairs to advance region-aware pretraining.
• It offers detailed annotations including global views, ROI crops, and multiple structured textual descriptions for precise clinical insights.
• Rigorous quality control and diverse modality distribution make it a robust resource for developing state-of-the-art medical AI systems.

MedRegion-500k is a large-scale, region-annotated multimodal medical image-text dataset developed to advance region-aware pretraining and evaluation in automated medical image understanding. It is designed to address limitations in existing datasets, namely the scarcity of high-quality, region-level medical annotations and the predominance of global image-text pairs which may overlook subtle but clinically relevant regional findings. MedRegion-500k serves as the foundational data resource for RegionMed-CLIP, a region-aware multimodal contrastive learning framework developed by Tianchen Fang and Guiru Liu (Fang et al., 7 Aug 2025).

1. Dataset Structure and Content

MedRegion-500k comprises 500,000 medical image–text pairs. Each sample consists of (i) a global imaging view (e.g., chest X-ray, CT/MRI slice, endoscopic photograph, or pathology slide), (ii) one or more region-of-interest (ROI) crops, and (iii) four structured textual descriptions per image:

• Summary caption: ≤15 words; concise clinical summary.
• Detailed report caption: Extended clinical report format.
• Region-specific caption: Text localizing findings to individual ROI(s).
• Hard-negative captions: For each ROI, five distractor captions derived from systematic perturbations of the region description.

ROI annotations total approximately 1.5 million, yielding an average of $\lambda \approx 3.0$ ROIs per image. Each ROI is assigned a bounding box (x1, y1, x2, y2), a region caption (using the template “adjective | lesion_term | position”), and five hard negative captions. All terminology adheres to domain standards (RadLex, BI-RADS).

Modality and Anatomical Distribution

The dataset spans 12 imaging modalities and 30 clinical specialties as summarized below:

Modality	Approx. %	Example Use Cases
Chest (X-ray/CT)	20	Pulmonology, thoracic pathology
Pathology Slide Imaging	12	Histopathology
MRI (Cranial, Cardiac)	10	Neurology, cardiology
Endoscopy	9	Gastroenterology
Dermatological Photography	8	Dermatology
Bone & Joint (X-ray/MRI)	8	Orthopedics
Breast (Mammography/Ultrasound)	7	Oncology, women’s health
Fundus Photography	7	Ophthalmology
Dental (Panorex)	6	Dentistry
Gynecological Ultrasound	5	Obstetrics/gynecology
Abdominal CT/Ultrasound	5	Abdominal imaging
Misc. (cardiac US, etc.)	3	Various

2. Annotation Process and Quality Assurance

ROI Collection Pipeline

MedRegion-500k annotation follows a two-phase protocol:

1. Seed manual annotation: Two board-certified radiologists annotate a core set of ~2,000 images, providing bounding boxes and clinical labels.
2. Automatic expansion:
   • Detection and segmentation are performed iteratively via Med-SAM (segment anything model adapted for medical images) and Grounding DINO (open-vocabulary detector).
   • Proposals are scored using both detection confidence and a report-consistency score. Low-confidence candidates are filtered out.
   • For each accepted ROI, four textual fields are generated via Qwen-2.5VL-72B prompted with structured JSON templates.

Textual Annotation and Standardization

Region captions employ a fixed template: “adjective | lesion_term | position,” with positions chosen from {left, right, center} × {upper, middle, lower}. All classification terms are mapped to established medical lexica.

Hard-Negative Caption Generation

For each ROI, five hard negative texts are synthesized by systematically altering at least one of the clinical dimensions (shape, lesion, position), ensuring minimal semantic similarity while remaining clinically plausible.

Quality Control

• Manual: 5% of ROI annotations (~25,000) undergo expert review.
• Agreement: The inter-annotator agreement (Cohen’s κ) for ROI presence and category is κ ≈ 0.82.
• Automated: Each hard-negative undergoes a sanity check for distinctness at the clinical term level.

3. Statistical Properties and Stratification

Distributional Characteristics

• ROI Count Per Image: Empirical distribution $P(n) = e^{-λ} λ^n / n!$, with λ = 3.0.
• Fraction of images by ROI count:

n	0	1	2	3	4	≥5
P(n)	0.05	0.20	0.30	0.25	0.12	0.08

• Pathology Distribution: Let $C_k$ denote counts of ROIs labeled with pathology $k$, and $F(k) = {C_k} / {\sum_{j=1}^{30} C_j}$.
  • Leading categories: lung nodule/opacity (F ≈ 0.18), breast mass (F ≈ 0.12), intracranial hemorrhage (F ≈ 0.10), gastrointestinal lesion (F ≈ 0.08), dermatologic lesion (F ≈ 0.07).

Data Splits

• Training: 400,000 pairs
• Validation: 50,000 pairs
• Test: 50,000 pairs

Splits are stratified by (modality × major pathology) to ensure proportionality across the top pathologies and imaging modalities.

4. Sample Data and Usage Recommendations

A representative annotation is provided for image_id MR500012345:

• Global image: Chest X-ray, PA view
• ROI1 bbox: (x1=120, y1=80, x2=300, y2=260)
  • Region caption: “round | opacity | left_lower_lobe”
  • Negative captions (perturb shape, lesion, or position): e.g., “irregular | opacity | left_lower_lobe,” “round | nodule | right_lower_lobe,” etc.
• Summary caption: “Left lower lobe opacity suggests possible pneumonia.”
• Detailed report caption: “There is a well-defined, round opacity in the left lower lobe measuring approximately 2.3 cm. No pleural effusion or pneumothorax. Cardiac silhouette is within normal limits.”

Preprocessing requirements for research use include resizing all images and ROIs to 224×224 pixels and lowercasing, punctuation-stripping all text fields. Feature embeddings are L2-normalized, and contrastive training employs a softmax temperature of $\tau=0.07$.

5. Licensing, Access, and Compliance

MedRegion-500k is distributed under the Creative Commons CC BY-NC-SA 4.0 license, restricting use to non-commercial research with share-alike obligations. Access is available upon request to the authors, conditional on a data-use agreement that prohibits patient re-identification, enforces HIPAA/GDPR compliance, and forbids commercial deployment.

6. Applications and Research Context

MedRegion-500k was developed as the core training and evaluation resource for RegionMed-CLIP, which demonstrates state-of-the-art performance in image-text retrieval, zero-shot classification, and visual question answering benchmarks. The explicit provision of region-level annotation enables hierarchical multimodal alignment that exceeds global-only methods by a substantial margin. The inclusion of hard-negative captions is critical for robust contrastive learning and discrimination at the fine-grained clinical level (Fang et al., 7 Aug 2025).

The dataset structure and annotation pipeline facilitate scalable exploration of localized pathological content and support rigorous benchmarks in both supervised and zero-shot paradigms for multimodal medical AI. A plausible implication is that region-aware datasets such as MedRegion-500k may inform the creation of future medical foundation models capable of nuanced clinical reasoning based on localized imaging findings.