Protocol Genome: DICOM Self-Supervised Learning

• Protocol Genome is a self-supervised framework that treats DICOM headers as a 'genome' to capture acquisition parameters and enhance model generalization.
• It employs dual parallel encoders with a fusion module using FiLM and cross-attention to jointly learn image and protocol representations.
• The approach mitigates acquisition-induced confounders in CT, MRI, and CXR, significantly improving AUROC and model calibration in clinical settings.

Protocol Genome is a self-supervised learning framework for medical imaging that treats structured DICOM headers as a “genome” of acquisition parameters and uses them as a first-class training signal alongside image data. Rather than learning from pixels alone, it models scanner manufacturer, model, sequence descriptors, reconstruction settings, dose-related fields, and related metadata to improve cross-site generalization, calibration, and robustness under domain shift. In the reported formulation, Protocol Genome is designed for PACS/DICOM-centric clinical workflows and is evaluated across CT, MRI, and chest radiography, where hidden protocol confounders and scanner heterogeneity can materially affect image appearance and downstream model behavior (Joseph, 3 Sep 2025).

1. Concept and problem setting

Protocol Genome addresses three linked difficulties in medical-imaging ML: hidden confounders in acquisition protocols, domain shift induced by scanner and site heterogeneity, and label scarcity. The motivating premise is that procedure choices such as scanner make and model, sequence parameters, dose, reconstruction kernel, and slice thickness influence contrast, noise, and artifacts, and that downstream clinical labels may become spuriously associated with those choices. In that setting, image-only models may learn “protocol shortcuts” rather than pathology-relevant structure (Joseph, 3 Sep 2025).

The framework therefore treats the structured DICOM header as a “genome” of acquisition genes. The term refers to non-PHI metadata fields that encode how the image was formed. The central methodological claim is not merely that metadata can be appended as tabular covariates, but that self-supervised objectives can explicitly align image representations with protocol representations, reconstruct masked protocol content, and translate between related protocol descriptions. This suggests a dual objective: learning representations that are protocol-aware for auditing and calibration, while becoming protocol-agnostic in the clinically relevant subspace used for downstream prediction (Joseph, 3 Sep 2025).

A common misconception is that DICOM headers are purely administrative metadata. In this framework, the clinically relevant subset of headers is treated as a structured description of the acquisition process itself. A plausible implication is that header information functions as a latent causal context for image formation, which may help disentangle disease signal from acquisition-induced variation when incorporated into representation learning.

2. DICOM header modeling and de-identification

Protocol Genome selects non-PHI fields that govern image formation. The reported header representation includes ProtocolName, SeriesDescription, Manufacturer, ManufacturerModelName, kVp, TubeCurrent, RepetitionTime, EchoTime, AcquisitionMatrix, SliceThickness, ConvolutionKernel / ReconstructionAlgorithm, CTDIvol, MR coil/receiver information, and a contrast usage tag (Joseph, 3 Sep 2025).

De-identification removes direct identifiers such as PatientName and AccessionNumber, shifts dates by patient-specific random offsets, remaps UIDs under a secure key, and strips burn-in overlays on CR/DR, while leaving acquisition parameters intact. This separation is operationally important because the method depends on preserving protocol-relevant metadata after de-identification (Joseph, 3 Sep 2025).

The tokenization strategy is heterogeneous by field type. Categorical fields, including free-text protocol names, are lower-cased, hashed into a controlled vocabulary, and subword-tokenized; the paper gives the example “CHEST_PE_PROTOCOL_HIGH_RES” mapping to tokens such as “chest,” “pe,” “protocol,” “high,” and “res.” Continuous fields such as kVp, mA, TR/TE, and slice thickness are discretized into piecewise-linear bins, with each bin index assigned a learned embedding; the raw continuous value may also be sinusoidally projected and concatenated. Missing or trimmed fields are represented explicitly with [MASK] and [MISSING] tokens, while continuous missingness is signaled with a sentinel “zero” bin and a binary mask bit (Joseph, 3 Sep 2025).

Each token is embedded into a $d$-dimensional space with $d=512$, producing an input token sequence for a Transformer-based Protocol Encoder. This modeling choice makes the protocol stream structurally analogous to language-model inputs, but the semantics are acquisition-centric rather than linguistic. A plausible implication is that the framework benefits from both the compositionality of free-text protocol descriptions and the ordinal structure of discretized numeric parameters.

3. Architecture

Protocol Genome consists of two parallel encoders and a fusion module. The image encoder $f_I$ depends on modality. For CT and MR, the reported backbone is ViT-Base with patch size 16, 12 layers, and 768-dimensional tokens, processing 2.5D stacks of 7 CT slices or multi-contrast MR slices. For CR/DR, the framework uses a ResNet-50 or EfficientNet-B3 backbone, with the final feature map projected to 768-dimensional tokens via a small convolution and flattening step (Joseph, 3 Sep 2025).

The protocol encoder $f_P$ is a 4-layer Transformer with 8 attention heads and hidden size 512. It consumes the tokenized header sequence and outputs both a sequence of protocol embeddings $\{p_k\}$ and a pooled [CLS]-style vector $\bar p$ of size 512 (Joseph, 3 Sep 2025).

Fusion is implemented through a hybrid of protocol-aware FiLM and cross-attention. First, scale-and-shift parameters are generated from the pooled protocol vector:

$\tilde x = \gamma \odot x + \beta$

where $\gamma, \beta = \mathrm{MLP}(\bar p)$ modulate image feature maps. Second, protocol embeddings act as queries and the modulated image tokens act as keys and values in two layers of multi-head cross-attention, yielding fused tokens for downstream SSL heads (Joseph, 3 Sep 2025).

An adversarial confounder head is attached to the post-fusion pooled clinical embedding. Using gradient reversal, this head predicts protocol identity at the level of site, vendor, or model during fine-tuning. The optimization jointly minimizes the clinical prediction loss while maximizing the adversarial loss, thereby removing protocol signals from the clinical representation used for downstream tasks (Joseph, 3 Sep 2025). This design encodes a deliberate tension: protocol information is exploited during representation learning and auditing, yet suppressed in the final clinical embedding to reduce shortcut reliance.

4. Self-supervised objectives

The pretraining setup uses paired image-header samples $\{x_i, h_i\}$ and combines three self-supervised tasks. The first is protocol-image contrastive learning, implemented with an InfoNCE objective:

$$\mathcal{L}_{\mathrm{contrast}} = -\,\frac{1}{B}\sum_{i=1}^{B} \log \frac{\exp\!\big(f_I(x_i)\cdot f_P(h_i)/\tau\big)} {\sum_{j=1}^{B}\exp\!\big(f_I(x_i)\cdot f_P(h_j)/\tau\big)}$$

This aligns each image with its corresponding header representation while separating mismatched image-header pairs in a shared embedding space (Joseph, 3 Sep 2025).

The second task is masked protocol prediction. A subset of header tokens is masked, with 30% of categorical tokens and 20% of numeric tokens masked in the reported setup. The model then reconstructs the original tokens given the image representation and the unmasked header tokens. For categorical fields, the loss is cross-entropy:

$d=512$0

and for binned numeric slots an $d=512$1 regression may be used (Joseph, 3 Sep 2025).

The third task is protocol-protocol translation. Two series headers from the same study, $d=512$2, are sampled, and a sequence-to-sequence Transformer translates the source protocol tokens into the target protocol tokens. The loss combines token-wise cross-entropy with an $d=512$3 term on numeric fields:

$d=512$4

The total pretraining loss is

$d=512$5

with $d=512$6, $d=512$7, and $d=512$8 (Joseph, 3 Sep 2025).

The ablation findings assign distinct roles to these objectives: the contrastive term is reported as most critical for AUROC, masked modeling for calibration, translation for cross-series consistency, and the adversarial head for subgroup gap reduction (Joseph, 3 Sep 2025). This suggests that the framework is not a monolithic metadata-augmented encoder, but a composite objective in which different losses target different failure modes.

5. Training regime, data scale, and evaluation design

The reported dataset comprises 1.26 million studies from 7 health systems, 31 scanners, and 3 vendors—GE, Siemens, and Philips—covering CT, MR, and CR/DR modalities. Pretraining uses AdamW with learning rate $d=512$9, weight decay 0.05, cosine decay over 200 epochs, and a 10-epoch warmup. The batch size is 512 images $f_I$0 2 views for contrastive learning, the temperature is $f_I$1, and the mask rates are $f_I$2 and $f_I$3 (Joseph, 3 Sep 2025).

Fine-tuning uses learning rates of $f_I$4 for the backbone and $f_I$5 for the heads, batch size 64, adversarial weight $f_I$6, calibration weight $f_I$7, and early stopping on external validation. Evaluation is performed on external-site, patient-held-out splits with 4-fold site splits (Joseph, 3 Sep 2025).

The downstream tasks are chest CT pulmonary embolism triage, brain MRI glioma grading, and chest radiograph cardiomegaly detection. Reported metrics include AUROC, AUPRC, sensitivity at 95% specificity, expected calibration error, and Brier score. Statistical testing uses DeLong’s test for AUROC differences and McNemar’s test for accuracy (Joseph, 3 Sep 2025).

The following table summarizes the principal external-site results reported for the three benchmark tasks.

Task	Protocol Genome	Baseline
CT-PE AUROC	0.912 (0.902–0.922)	0.866 (0.853–0.879)
CT-PE ECE	0.032	0.051
Brain MRI AUROC	0.931 (0.915–0.946)	0.873 (0.848–0.895)
CXR cardiomegaly AUROC	0.892 (0.883–0.901)	0.851 (0.838–0.863)

Across these tasks, the reported external-site AUROC gains are $f_I$8 for PE, $f_I$9 for glioma, and $f_P$0 for cardiomegaly, with DeLong tests reported as significant. The abstract separately reports fully held-out external validation performance of AUROC 0.901 versus 0.847 baseline, and ECE 0.036 versus 0.058 baseline (Joseph, 3 Sep 2025). Because both site-fold and fully held-out evaluations are reported, direct numerical comparison across these summaries should be made with care.

6. Empirical behavior, robustness, and fairness

Protocol Genome is reported to improve both discrimination and calibration under external validation. For CT-PE, ECE decreases from 0.051 to 0.032, corresponding to a 37% reduction, while the abstract reports 25–37% calibration improvements across tasks. For brain MRI, ECE is reported to decrease by 25%. For chest radiography, sensitivity at 95% specificity increases by 4–7 points (Joseph, 3 Sep 2025).

In low-label settings, the method retains 92–95% of full-data AUROC when only 10–20% of labels are available, versus 84–89% for the baselines, and gains are reported to persist even at 5% labeling (Joseph, 3 Sep 2025). This is consistent with the framework’s motivation in label-scarce domains: the representation learning signal is derived from naturally occurring DICOM metadata rather than expensive manual annotation.

The fairness analysis reports that the vendor gap in CT-PE shrinks from 0.062 to 0.024 AUROC, that no significant sex bias is observed post-finetuning, and that sensitivity on patients aged at least 80 years increases by 5.2 points (Joseph, 3 Sep 2025). These findings are tied to the adversarial confounder head and the auditing strategy rather than to representation learning alone.

A possible misunderstanding is that protocol-aware learning must inevitably encode and amplify site bias. The reported design takes the opposite position: protocol information is made explicit so it can be aligned, audited, calibrated, and then partially removed from the task representation through adversarial training. This suggests that explicit modeling of nuisance structure may be more controllable than leaving it latent in pixel-only embeddings.

7. Clinical integration, deployment, and limitations

Protocol Genome is presented as directly deployable within PACS/DICOM ecosystems. The deployment pathway includes DICOM C-FIND/C-MOVE and DICOMweb QIDO-RS/WADO-RS, with interoperability notes referencing IHE XDS-I for cross-enterprise image exchange and HL7 FHIR for result reporting (Joseph, 3 Sep 2025). The paper also describes a model card and deployment checklist covering PHI removal, date shifting, UID remapping, a bias audit plan based on mutual information between headers and labels plus stratified metrics with FDR correction, and monitoring via protocol distribution drift using PSI/KL together with ECE tracking. Alerts are specified when ECE exceeds 0.06 or vendor drift exceeds 5% (Joseph, 3 Sep 2025).

Security notes include DICOM header schema validation, digital signatures, and optional DP-SGD on the protocol encoder (Joseph, 3 Sep 2025). These details indicate that the framework is not limited to pretraining methodology; it is also framed as an operational system for real-world monitoring and governance.

Several limitations follow directly from the reported claims. The abstract states that gains may be task-dependent (Joseph, 3 Sep 2025). The method also presumes that the retained de-identified header fields are accurate, available, and consistently encoded enough to be tokenized meaningfully. A plausible implication is that deployment performance may depend on local DICOM normalization practices, ontology harmonization for free-text fields, and the stability of vendor-specific protocol naming conventions. Likewise, although the framework seeks protocol-agnostic clinical representations, it explicitly relies on acquisition metadata during learning, so failure modes may arise when headers are missing, truncated, or systematically remapped across institutions.

Within the broader medical-imaging SSL landscape, Protocol Genome is positioned against image-only baselines such as SimCLR, BYOL, MAE, and ImageNet transfer, with higher external AUROC and improved calibration reported across CT, MRI, and CXR benchmarks (Joseph, 3 Sep 2025). Its distinctive contribution is the elevation of DICOM protocol metadata from auxiliary context to a structured self-supervised signal. In that sense, “Protocol Genome” denotes both a representation of acquisition parameters and a training paradigm for leveraging that representation to improve robustness under multi-center domain shift.