RAD-DINO: Unimodal Radiology Image Encoder

Updated 12 July 2026

RAD-DINO is a biomedical image encoder pre-trained solely on unimodal chest X-ray data, offering a new approach without relying on text supervision.
It continues pretraining a DINOv2 ViT-B/14 on a large, diverse Multi-CXR corpus using self-distillation, masked image modeling, and KoLeo regularization to capture both global and local features.
Evaluated on classification, segmentation, and report generation, RAD-DINO achieves comparable or superior performance to language-supervised models while preserving rich clinical variability.

RAD-DINO is a biomedical image encoder pre-trained solely on unimodal biomedical imaging data, introduced to test whether a strong general-purpose radiology representation can be learned without language supervision. It is obtained by continual pre-training a DINOv2 ViT-B/14 backbone on the chest X-ray corpus Multi-CXR, and it is evaluated on image classification, semantic segmentation, and image-to-text report generation. Its central claim is that image-only self-supervision can match or exceed state-of-the-art biomedical language-supervised models while preserving clinically useful visual variability that is often absent from radiology reports (Pérez-García et al., 2024).

1. Conceptual basis and problem setting

RAD-DINO is motivated by a critique of the prevailing medical vision-language paradigm. The underlying argument is that radiology reports are incomplete, noisy, and too narrow to serve as the sole source of supervisory signal for representation learning. Reports summarize salient findings, omit many visible anatomical and demographic attributes, and are difficult to scale because public image-report pairs are scarce and constrained by personal health information concerns. Within that framing, RAD-DINO is designed to learn the visual encoder from images alone and defer language alignment to downstream adaptation rather than pretraining (Pérez-García et al., 2024).

The model is explicitly contrasted with language-supervised encoders such as BioViL-T, BiomedCLIP, CLIP, and MRM. In those systems, image features are aligned to text or report-derived targets. RAD-DINO instead uses only imaging data during pretraining—no reports, no captions, and no text tokens. The intended consequence is that the learned representation is not constrained by report wording and therefore better preserves broader visual variability. The paper argues that this matters particularly for tasks requiring fine-grained localization, dense prediction, biomarker-related variation, or sensitivity to clinically meaningful properties that are usually not verbalized in reports.

A recurrent misconception addressed by the work is that semantically useful radiology features require paired text supervision. RAD-DINO is presented as a direct counterexample: it is evaluated as a frozen image encoder and still yields strong performance on both standard vision tasks and a downstream report-generation setup. A second misconception is that image-only pretraining is necessarily less scalable than multimodal pretraining. The paper argues the opposite, namely that unimodal image supervision scales more naturally because the supply of images exceeds the supply of curated image-report pairs.

2. Pretraining corpus, backbone, and optimization regime

RAD-DINO is built by continuing to train a pretrained DINOv2 ViT-B/14 on Multi-CXR, a combined chest X-ray dataset assembled from BRAX, CheXpert, MIMIC-CXR, NIH-CXR, PadChest, and a private outpatient dataset. The pretraining corpus contains 439,177 subjects and 838,336 images. This diversity is important in the paper’s interpretation because later ablations attribute part of RAD-DINO’s performance to both the quantity and the heterogeneity of the image-only corpus (Pérez-García et al., 2024).

The continual pretraining setup uses a batch size of 640, AdamW, a base learning rate of 0.001, cosine learning-rate decay with linear warmup, and image resolution fixed at $518\times 518$ . Training runs for 60k steps, also described as 100 epochs in the implementation details. The augmentations are adapted for chest X-rays: global crops are less aggressively blurred, and crop sizes are chosen to preserve the texture and context relevant for pathology recognition. These design choices reflect the fact that chest radiography contains diagnostically important low-contrast and localized structures that can be degraded by overly aggressive augmentation.

The underlying DINOv2-style training retains the teacher-student architecture, with exponential moving average updates for the teacher and backpropagation only through the student. This detail is significant because the paper does not propose a wholly new pretraining algorithm; rather, it tests whether a strong general-domain self-supervised recipe, when continued in-domain on radiographs, is sufficient to rival or exceed language-supervised medical encoders.

3. Training objective and representational properties

The paper describes RAD-DINO’s objective as a combination of global view alignment, masked image modeling, and KoLeo regularization. In compact form, the training recipe is summarized as

$\mathcal{L} = \mathcal{L}_{\text{global}} + \lambda \mathcal{L}_{\text{MIM}} + \gamma \mathcal{R}_{\text{KoLeo}}.$

The global component follows DINO-style self-distillation across multiple crops:

$\mathcal{L}_{\text{global}} = - \sum_{v \in \mathcal{V}_{\text{student}}} \sum_{u \in \mathcal{V}_{\text{teacher}}} p_t(u)\,\log p_s(v),$

while the masked image modeling term is written generically as

$\mathcal{L}_{\text{MIM}} = \sum_{i \in \mathcal{M}} \left\| z^{(t)}_i - z^{(s)}_i \right\|^2.$

Teacher parameters are updated by exponential moving average:

$\theta_{\text{teacher}} \leftarrow \tau \theta_{\text{teacher}} + (1-\tau)\theta_{\text{student}}.$

The paper emphasizes that the hybrid objective is what gives RAD-DINO both strong global semantic features and strong local texture or spatial features. Global self-distillation is associated with classification and report generation, whereas masked image modeling is explicitly credited with preserving the pixel-sensitive detail needed for semantic segmentation and localization. KoLeo regularization is described as encouraging a uniform feature distribution, which in turn supports retrieval-like behavior and reduces collapse (Pérez-García et al., 2024).

This representation-level argument is central to the RAD-DINO thesis. The model is not merely image-only; it is image-only with an objective designed to preserve both image-level invariances and patch-level correspondence. The paper therefore treats masked image modeling not as an auxiliary improvement but as a necessary component for dense radiological tasks.

4. Downstream evaluation and empirical profile

RAD-DINO is evaluated on three task families: image classification, semantic segmentation, and image-to-text report generation. The backbone is frozen during these downstream evaluations, and task-specific decoders or projectors are added on top.

Task family	Benchmarks	Setup
Classification	VinDr-CXR, CANDID-PTX, RSNA Pneumonia	Frozen backbone + linear probing
Segmentation	Lungs, lung zones, pneumothorax, chest tubes, VinDr-RibCXR ribs	Frozen backbone + linear head, ViTDet, or UPerNet
Report generation	MIMIC-CXR official split	Frozen encoder + 2-layer MLP projector + Vicuna-7B

On classification, RAD-DINO achieves a macro AUPRC of 66.63 on VinDr-CXR, compared with 33.03 for CLIP@224, 34.54 for CLIP@336, 34.19 for BioViL-T, 45.42 for BiomedCLIP, and 60.86 for MRM. On CANDID-PTX, it reaches 80.08 for pneumothorax, 90.80 for chest tube, and 13.43 for fracture, compared with 74.88, 58.16, and 12.25 for MRM, respectively. On RSNA Pneumonia, it obtains 70.99 AUPRC and 88.38 AUROC. The paper highlights these results as notable because RAD-DINO uses no text supervision during pretraining (Pérez-García et al., 2024).

On segmentation, the model reaches 95.9 Dice for lungs with a linear head, 85.7 for lung zones, 47.4 for pneumothorax, 90.4 for chest tubes, and 73.4 for ribs. ViTDet improves many results further, and UPerNet brings the frozen-encoder setup close to end-to-end U-Net performance. The interpretation given in the paper is that CLIP-style image-text contrastive objectives do not enforce pixel-level correspondence strongly enough for dense prediction, whereas RAD-DINO’s masked image modeling preserves the local detail required by these tasks.

On report generation, RAD-DINO is adapted using a frozen image encoder, a 2-layer MLP projector, and Vicuna-7B to generate the Findings section of MIMIC-CXR reports. It achieves 24.6 ROUGE-L, 9.3 BLEU-4, 22.8 RG $_{\text{ER}}$ , and 31.9 CheXbert Macro-F1-14. The paper presents these as the best reported among the compared encoders and interprets the especially large gain in Macro-F1-14 as evidence that the image representation preserves clinically relevant pathology information despite the absence of report supervision during pretraining.

5. Scaling behavior, metadata capture, and the critique of text supervision

A distinctive result in RAD-DINO is that its image features correlate more strongly with patient metadata than those of language-supervised encoders. Using 60.1k “no findings” MIMIC-CXR images linked to MIMIC-IV records, linear classifiers trained on frozen encodings predict sex, age, weight, and BMI more accurately from RAD-DINO than from BioViL-T or BiomedCLIP. The reported scores are 99.6 for sex, 72.3 for age, 62.4 for weight, and 71.3 for BMI, compared with 75.1, 60.8, 43.8, and 47.6 for BioViL-T and 86.0, 56.5, 52.8, and 54.2 for BiomedCLIP (Pérez-García et al., 2024).

The paper uses these results to support a specific criticism of report supervision: many clinically meaningful properties are visible in the image but are not generally mentioned in radiology reports. If representation learning is tightly constrained by text, those properties may be attenuated or discarded. RAD-DINO’s stronger correlation with non-report metadata is therefore presented as evidence that image-only pretraining preserves broader clinical information.

The ablation studies reinforce this interpretation. Higher resolution helps with subtle findings such as pneumothorax and chest tubes, but resolution alone does not explain the gains. General-domain DINOv2 initialization transfers well, yet continual in-domain pretraining gives a substantial further boost: on VinDr-CXR, the paper reports macro AUPRC values of 39.46 for DINOv2 ViT-B, 43.36 for DINOv2 ViT-G, 59.82 for RAD-DINO with random initialization, and 66.63 for RAD-DINO with continual pretraining. Performance also improves as more diverse chest X-ray data are added, and removing the masked image modeling component hurts segmentation substantially, especially on dense tasks. The appendix further reports that image-only pretraining handles lateral chest X-rays well, where report alignment is especially noisy.

Taken together, these findings motivate the paper’s broader claim that text supervision can become a bottleneck rather than an advantage in medical imaging. The claim is not that language is useless downstream; rather, it is that representation learning based purely on images may be better aligned with the full richness of medical image content.

RAD-DINO sits within a broader family of DINO-based representation-learning systems, but its scope is specific: it is a chest X-ray encoder trained only on images. A closely related radiology model is RayDINO, which follows the DINOv2 self-supervised recipe on four public chest X-ray datasets, uses a ViT-L with 307M parameters, keeps the backbone frozen for downstream tasks, and evaluates across nine radiology tasks and 21 benchmarks. RayDINO is positioned as broader and more thoroughly evaluated than prior self-supervised radiology encoders such as REMEDIS and RAD-DINO, while retaining the same general thesis that image-only self-supervision can support classification, dense segmentation, regression, and report generation without report supervision during pretraining (Moutakanni et al., 2024).

Beyond radiology, later DINO-informed systems reuse a related design principle: self-supervised transformer features can act as structural priors rather than merely auxiliary descriptors. DINO-SLAM introduces a Scene Structure Encoder that transforms raw DINO features into Enhanced DINO (EDINO) by fusing appearance, geometry, and DINO through attention and MLP refinement, then injects those features into NeRF-based and 3D Gaussian Splatting SLAM pipelines (Gong et al., 25 Jul 2025). DINO_4D similarly treats frozen DINOv3 features as structural priors for dynamic 4D reconstruction, using semantic injection and semantic consistency loss to suppress semantic drift while retaining $O(T)$ inference complexity (Yang et al., 10 Apr 2026). These systems are not radiology encoders, but they illustrate the portability of the broader DINO-based idea: frozen self-supervised visual descriptors can constrain downstream geometric or semantic inference in domains far removed from chest X-rays.

The name RAD-DINO also admits a purely notational ambiguity with unrelated algebraic terminology. In module theory, a Rad- $D_{11}$ -module is defined as a module in which every submodule has a Rad-supplement that is a direct summand, and the strengthened notion C-Rad- $D_{11}$ requires every direct summand to be Rad- $D_{11}$ (Abed et al., 2017). This algebraic usage shares only the string “Rad” and has no overlap in subject matter with the biomedical image encoder RAD-DINO.

RAD-DINO’s significance therefore lies less in a novel architectural primitive than in the position it stakes out within medical foundation modeling. It argues, with empirical support, that unimodal self-supervision on large-scale chest radiographs is not merely viable but competitive with, and in several settings superior to, language-supervised medical representation learning.