RadImageNet: Radiology Pretraining Dataset

• RadImageNet is a large-scale, multi-modal radiology dataset with approximately 1.35 million annotated images from CT, MRI, ultrasound, and chest X-ray.
• It supports a variety of applications including pretraining of CNN architectures, transfer learning, and evaluation of generative model features through perceptual metrics.
• Empirical findings indicate that while ImageNet may outperform RadImageNet on conventional accuracy, RadImageNet shows enhanced robustness against out-of-distribution confounders.

Searching arXiv for the original RadImageNet paper and closely related analyses to support the article. RadImageNet is a large-scale radiology image dataset used as a medical-domain source for pretraining, transfer learning, feature extraction, and methodological benchmarking in medical imaging. Across the studies summarized here, it is consistently described as comprising approximately 1.35 million annotated radiology images, with reported coverage of CT, MRI, and ultrasound, and one replication summary additionally describing chest radiography (X-ray) as part of the public release (Juodelyte et al., 2024, Woodland et al., 2023, Juodelyte et al., 2023). It was introduced as an ImageNet-style in-domain alternative for radiology, with the stated goal of learning transferable low- and mid-level medical-image features rather than relying exclusively on natural-image pretraining (Juodelyte et al., 2024).

1. Dataset identity and reported composition

Secondary descriptions of RadImageNet agree on its scale but differ slightly in how they summarize its composition. Multiple papers describe it as approximately 1.35 million annotated radiology images spanning CT, MRI, and ultrasound, with pathologies covering musculoskeletal, neurologic, oncologic, abdominal, endocrine, and pulmonary domains (Choudhury et al., 22 Oct 2025). A separate replication study reports that the publicly released form comprises approximately 1.35 million images collected from 131,872 patients and lists chest radiography, CT, MRI, and ultrasound among the represented 2D modalities (Juodelyte et al., 2023).

Several papers also report label granularity in different ways. One methodological study describes RadImageNet as stratified across 165 pathologic labels (Yavuz et al., 2024), while a robustness study refers to “165+ different radiologic findings or pathologies” (Juodelyte et al., 2024). By contrast, the Online Label Smoothing study denotes the total number of classes abstractly as $K$, equal to the number of distinct anatomical or pathological categories used in its experiments, without specifying a fixed count for that paper’s split (Choudhury et al., 22 Oct 2025). A plausible implication is that later studies sometimes emphasize the global label space, whereas others describe the effective class structure only insofar as it enters a specific training protocol.

One paper provides the most detailed modality-by-organ breakdown. In that account, the stratified splits comprise 1,048,575 training images, 50,182 validation images, and 100,364 test images. The aggregated modality totals are CT: 292,353 images, ultrasound: 389,885 images, and MRI: 672,675 images; the MRI subset is further broken down into knee, shoulder, spine, ankle/foot, abdomen/pelvis, brain, and hip categories (Yavuz et al., 2024). This description situates RadImageNet not merely as a monolithic corpus, but as a heterogeneous multi-organ, multi-modal source dataset.

2. Pretraining resource and released weight ecosystem

RadImageNet is used in the literature both as a dataset for training models from scratch and as a source of publicly released pretrained weights. In Fréchet Distance evaluation for generative medical imaging, four standard CNN backbones pretrained on RadImageNet were used out-of-the-box: InceptionV3, ResNet50, InceptionResNetV2, and DenseNet121. In that setting, no further fine-tuning or domain-specific adaptation was applied; penultimate-layer activations from the released RadImageNet weights were used directly to fit multivariate Gaussians (Woodland et al., 2023).

The weight ecosystem is not fully uniform across frameworks. One transfer-learning study states that original RadImageNet pretraining used TensorFlow for InceptionV3 and ResNet-50 and published PyTorch weights for DenseNet-121, and it explicitly raises the possibility that differences between TensorFlow and PyTorch weight dumps may degrade downstream performance in PyTorch (Frees et al., 25 Aug 2025). Another downstream brain MRI study is even more explicit about missing provenance: it loads the file RadImageNet-DenseNet121_notop.h5 into Keras but reports no details of the original optimizer, learning-rate schedule, batch size, number of epochs, or loss formulation used during RadImageNet pretraining (Abedini et al., 23 Nov 2025).

Where full training recipes are reported, they vary by study because the papers are not all reusing the same published backbone. A robustness analysis describes RadImageNet pretraining for ResNet-50 with multiclass cross-entropy, Adam, learning rate $1\times 10^{-3}$, weight decay $1\times 10^{-4}$, batch size 256, up to 100 epochs, and augmentation via random crop and resize, horizontal flip, and intensity jitter of $\pm 10\%$ (Juodelyte et al., 2024). A cross-dimensional transfer paper instead trains ResNet-18 variants from scratch on RadImageNet for 90 epochs with AdamW, initial learning rate 0.001, effective batch size 512 on 16 NVIDIA V100 GPUs, step decay by 0.1 every 30 epochs, and augmentations consisting of random horizontal flip, random crop to $224^2$, and intensity jitter of $\pm 10\%$ (Yavuz et al., 2024). These differences are significant because they show that “RadImageNet pretraining” in later literature may denote either reuse of public medical-domain weights or independent large-scale pretraining on the RadImageNet corpus under a study-specific recipe.

3. Transfer-learning outcomes: mixed empirical evidence

The central empirical controversy around RadImageNet is whether medical-domain pretraining is reliably superior to ImageNet initialization on downstream medical tasks. The literature summarized here does not support a universal answer.

A seven-task replication study using ResNet-50 reports that ImageNet initialization tended to yield the highest AUC on six of seven tasks, with RadImageNet slightly outperforming ImageNet only on knee MRI. The reported mean test AUCs include breast ultrasound: $94.3 \pm 1.7$ or $95.1 \pm 3.6$ for ImageNet versus $91.0 \pm 5.2$ or $89.4 \pm 3.8$ for RadImageNet, chest X-ray: $1\times 10^{-3}$0 or $1\times 10^{-3}$1 for ImageNet versus $1\times 10^{-3}$2 or $1\times 10^{-3}$3 for RadImageNet, and knee MRI: $1\times 10^{-3}$4 or $1\times 10^{-3}$5 for ImageNet versus $1\times 10^{-3}$6 or $1\times 10^{-3}$7 for RadImageNet, depending on whether the two-stage freezing schedule was used (Juodelyte et al., 2023).

A more targeted study on ACL tear detection and breast lesion malignancy also fails to confirm a general RadImageNet advantage. Its best ImageNet-pretrained models achieve test AUCs of 0.9969 for ACL tear detection and 0.9641 for breast lesion malignancy, whereas the corresponding RadImageNet-pretrained models achieve 0.9594 and 0.8889. That paper further reports statistically significant DeLong-test differences favoring ImageNet for both tasks: ACL $1\times 10^{-3}$8, $1\times 10^{-3}$9, 95% CI $1\times 10^{-4}$0; breast $1\times 10^{-4}$1, $1\times 10^{-4}$2, 95% CI $1\times 10^{-4}$3 (Frees et al., 25 Aug 2025).

A small-data brain MRI tumor classification study reaches a similar conclusion at the architecture level. Under identical fine-tuning conditions on a combined Kaggle four-class brain MRI dataset with 8,582 training-plus-validation images and 1,705 test images, ConvNeXt-Tiny attains 93% test accuracy and mean ROC-AUC 0.985, EfficientNetV2S attains 85% and 0.960, and RadImageNet-pretrained DenseNet121 attains 68% and mean ROC-AUC approximately 0.88. The DenseNet121 per-class ROC-AUCs are 0.84 for glioma, 0.80 for meningioma, 0.91 for no-tumor, and 0.96 for pituitary (Abedini et al., 23 Nov 2025).

At the same time, a robustness study reports a different pattern: ImageNet- and RadImageNet-pretrained ResNet-50 models have essentially identical IID AUCs on chest X-ray and CT classification, but RadImageNet pretraining is markedly more robust under out-of-distribution confounder shifts constructed via the Medical Imaging Contextualized Confounder Taxonomy (MICCAT). For example, on CXR mass detection with a “Tag” confounder, IID AUCs are 0.84 for ImageNet and 0.85 for RadImageNet, while OOD AUC at $1\times 10^{-4}$4 drops to 0.02 for ImageNet and 0.22 for RadImageNet. For CT mass detection with a denoising confounder, IID AUCs are 0.70 for ImageNet and 0.83 for RadImageNet, with OOD AUCs 0.01 and 0.60, respectively (Juodelyte et al., 2024).

Study	Task setting	Reported outcome
(Juodelyte et al., 2023)	Seven medical classification tasks	ImageNet higher on six of seven tasks; RadImageNet slightly higher on knee MRI
(Frees et al., 25 Aug 2025)	ACL tear and breast lesion classification	ImageNet significantly higher AUC on both tasks
(Abedini et al., 23 Nov 2025)	Small-data brain MRI tumor classification	ConvNeXt-Tiny and EfficientNetV2S outperform RadImageNet DenseNet121
(Juodelyte et al., 2024)	CXR and CT with controlled confounders	IID AUC similar, but RadImageNet more robust OOD

Taken together, these results undermine the common simplification that in-domain radiology pretraining is automatically superior. The evidence instead indicates criterion dependence: RadImageNet may underperform on conventional downstream accuracy under some architectures and small-data regimes, yet show stronger resistance to shortcut learning under controlled confounder stress tests.

4. RadImageNet as a feature space for evaluation metrics

RadImageNet is not only used to initialize classifiers; it is also used as a feature extractor for perceptual metrics in medical image synthesis. In that setting, the most explicit challenge to RadImageNet appears in work on Fréchet Distance evaluation. The study computes Fréchet Distance between real and generated image feature distributions as

$1\times 10^{-4}$5

and normalizes across extractors using a relative FD obtained by dividing the real-vs-generated FD by the FD between two random splits of the real data (Woodland et al., 2023).

The experiment spans four datasets—SLIVER07 abdominal CT, ChestX-ray14 frontal chest radiographs, MSD Brain Tumor axial brain MRI, and ACDC cardiac cine-MRI slices—and sixteen StyleGAN2 networks trained with no augmentation, ADA, APA, or DiffAugment. Eleven feature extractors are compared: seven ImageNet-based and four RadImageNet-based (Woodland et al., 2023). The paper’s conclusion is unambiguous: across all four modalities and augmentation schemes, ImageNet-trained extractors produce tightly clustered FD rankings aligned with human rankings, whereas RadImageNet extractors yield widely varying and often contradictory rankings.

Several examples are reported. On SLIVER07, human experts unanimously rank DiffAugment highest in realism, while RadImageNet-InceptionV3 relative FD places DiffAugment among the worst. On ChestX-ray14, RadImageNet-InceptionV3 ranks the unaugmented model best despite radiologists noting blatant anatomic distortions. Across modalities, RadImageNet extractors show no consistent or significant correlation with false-positive rate, false-negative rate, Likert difference, or Kolmogorov–Smirnov $1\times 10^{-4}$6-values from the visual Turing test, whereas an ImageNet-trained SwAV feature extractor achieves Pearson $1\times 10^{-4}$7 against the difference in mean Likert scores with $1\times 10^{-4}$8 (Woodland et al., 2023).

The paper attributes this instability to disease-focused training bias: RadImageNet networks were optimized for image-level disease classification and may attend strongly to small patches or subtle textural cues rather than global anatomical realism. This suggests that a feature space useful for radiologic classification need not be reliable for generative evaluation.

5. RadImageNet as a methodological benchmark

Because of its scale and multi-modal heterogeneity, RadImageNet has become a benchmark for studying optimization, regularization, calibration, and cross-dimensional transfer.

In policy-gradient-driven noise masking, RadImageNet is used both for in-domain classification and for constructing transfer features later evaluated on an enhanced MedMNIST decathlon. The method trains a randomly initialized ResNet-50 classifier jointly with a lightweight ResNet-10t policy network that predicts image-specific Beta-distributed noise masks during a “heated” pretraining phase, after which the policy is discarded and the classifier is fine-tuned without masking. On RadImageNet, the baseline ResNet-50 obtains balanced accuracy 0.5014 and AUROC 0.9884; the heated-then-fine-tuned model obtains balanced accuracy 0.5211, AUROC 0.9900, and predictive entropy 0.3177, while the noisy heated model without final fine-tuning obtains balanced accuracy 0.5136 and AUROC 0.9898 (Yavuz et al., 2024). For unseen-concept transfer across 13 MedMNIST tasks, the paper reports that GradPolicy RadImageNet features win 11 of 13 average F1 comparisons, including Breast US $1\times 10^{-4}$9 and Brain MRI $\pm 10\%$0 for IN1K, RadImageNet, and RadImageNet plus GradPolicy, respectively (Yavuz et al., 2024).

In Online Label Smoothing, RadImageNet serves as the large-scale supervised training set for studying accuracy and confidence calibration. The method maintains epoch-indexed soft-label matrices $\pm 10\%$1 and optimizes a hybrid objective

$\pm 10\%$2

where the soft targets are updated from the model’s own correct predictions at the previous epoch (Choudhury et al., 22 Oct 2025). On RadImageNet, the reported Top-1/Top-5 classification errors for ResNet-50 are 24.27/5.20 with hard labels, 23.84/6.60 with uniform label smoothing, 25.17/5.90 with teacher-free knowledge distillation, and 23.80/4.61 with OLS. For MobileNetV2, the corresponding numbers are 25.01/5.04, 21.79/6.71, 26.49/6.49, and 24.43/4.34; for VGG-19 they are 30.38/5.84, 29.26/5.62, 30.24/5.73, and 27.19/5.37 (Choudhury et al., 22 Oct 2025). On ResNet-50, expected calibration error drops from 0.1537 for the baseline to 0.0611 with label smoothing and 0.0151 with OLS, while mean predicted confidence decreases from 0.8783 to 0.8303 and 0.7906 (Choudhury et al., 22 Oct 2025). The same paper reports that OLS yields more compact and better separated penultimate-layer t-SNE clusters.

In Cross-D Conv, RadImageNet is the 2D source domain for pretraining a ResNet-18 whose convolution kernels can be mapped between 2D and 3D by Fourier-domain phase shifting. The core operation applies a phase factor

$\pm 10\%$3

with learned angle $\pm 10\%$4 constrained by $\pm 10\%$5 (Yavuz et al., 2024). On 2D RadImageNet classification, standard ResNet-18 obtains macro precision 0.5830, macro recall 0.4989, macro F1 0.5252, balanced accuracy 0.4989, and overall accuracy 0.8305, whereas Cross-D Conv obtains 0.5937, 0.5173, 0.5425, 0.5173, and 0.8361 (Yavuz et al., 2024). When transferred to 3D through a weak-probe protocol, the paper reports average performance 0.588 for Cross-D Conv pretrained on RadImageNet versus 0.586 for ACS-Conv pretrained on RadImageNet, 0.576 for ACS-Conv pretrained on ImageNet, and 0.575 for random-initialized 3D convolution (Yavuz et al., 2024).

These studies collectively position RadImageNet as more than a static source of pretrained weights. It functions as a high-scale laboratory for testing whether representation-learning interventions improve not only in-domain classification but also calibration, transfer to unseen concepts, and 2D-to-3D knowledge migration.

6. Representational interpretation, misconceptions, and open questions

A persistent misconception in medical imaging is that domain matching alone determines transfer quality. The studies summarized here do not support that claim. Instead, they identify several interacting factors: dataset diversity, target modality overlap, source-task bias, optimization protocol, model depth, downstream sample size, and even implementation framework.

The most detailed representation-level analysis comes from a Canonical Correlation Analysis study comparing ImageNet- and RadImageNet-pretrained ResNet-50 models across seven target tasks. It reports that ImageNet- and RadImageNet-initialized networks converge to distinct intermediate representations, and that after fine-tuning on the same target these representations become highly dissimilar, in some cases more dissimilar than two randomly initialized networks. Yet the models’ predictions remain surprisingly similar, with mistake correlations significantly higher than expected under independence (Juodelyte et al., 2023). The paper also reports no positive correlation between layer-wise representational reuse and the AUC advantage of pretraining over random initialization, suggesting that transfer-learning gains are not explained solely by early-layer feature reuse (Juodelyte et al., 2023).

Other papers provide task-specific hypotheses for when RadImageNet underperforms. The brain MRI tumor study suggests that the medical-image pretraining may itself have been on a limited set of radiology images and may have overfitted to those modalities or patterns, that DenseNet121 may need more domain data to fully leverage specialized features, and that large general-purpose datasets like ImageNet may provide more varied features that transfer better to very small downstream medical datasets (Abedini et al., 23 Nov 2025). The ACL and breast lesion study raises additional possibilities: RadImageNet may lack breast-ultrasound examples, framework conversion between TensorFlow and PyTorch may impair weight quality, and architecture plus unfreezing policy may have larger effects than the source of pretraining (Frees et al., 25 Aug 2025).

Conversely, the robustness study argues that when the evaluation target is resistance to shortcut learning rather than only IID accuracy, RadImageNet may be preferable. Its recommendation is that researchers using ImageNet-pretrained models should reexamine robustness by defining domain-specific confounders via MICCAT, controlling confounder prevalence during training, and evaluating on flipped OOD test sets (Juodelyte et al., 2024). This is an important corrective to the narrower accuracy-only framing.

A final practical issue is documentation asymmetry. Some downstream studies employ RadImageNet weights while being unable to report the original pretraining optimizer, schedule, batch size, or loss, because those details are absent from the reused checkpoint’s immediate provenance (Abedini et al., 23 Nov 2025). This suggests that reproducibility around RadImageNet is not solely a question of dataset availability; it also depends on transparent weight release, framework parity, and complete pretraining metadata.

In aggregate, RadImageNet emerges as an in-domain radiology foundation dataset whose scientific importance lies not in a single settled advantage over ImageNet, but in the way it exposes the multidimensional nature of transfer in medical imaging. The literature reviewed here shows that its value can appear as stronger robustness, improved calibration, better cross-dimensional transfer, or improved unseen-concept generalization, while standard downstream accuracy can still favor large natural-image pretraining under some architectures and data regimes.