Site-Conditioned Deep Harmonization Framework

• The paper presents a framework that disentangles subject anatomy from site-specific image appearance using explicit target conditioning for multi-site MRI harmonization.
• It leverages diverse architectures—including dual-branch autoencoders, diffusion models, and normalizing flows—to achieve modular, privacy-preserving synthesis.
• Empirical evaluations show high reconstruction fidelity and improved downstream metrics while significantly reducing non-biological scanner variance.

Searching arXiv for papers on site-conditioned MRI harmonization, disentangled harmonization, diffusion/flow-based MRI harmonization, and related frameworks. A site-conditioned deep harmonization framework is a class of retrospective harmonization methods in which anatomical or biological content is modeled separately from site-, scanner-, protocol-, or contrast-specific appearance, and reconstruction is then performed under an explicitly chosen target condition. In multi-site MRI, this formulation is used to reduce non-biological variation while preserving anatomy, biological variability, and downstream task utility; representative instances include PRISM, MURD, CALAMITI, DDAE, HCLD, DLEST, MMH, DIST-CLIP, and related methods operating in voxel space, latent space, diffusion space, or connectome space (Galada et al., 2024, Liu et al., 2021, Zuo et al., 2021, Ijishakin et al., 2024, Wu et al., 2024, Wu et al., 2024, Wu et al., 13 Jan 2026, Avci et al., 8 Dec 2025).

1. Conceptual formulation

The central premise is that scanner-induced covariate shift primarily affects image appearance rather than underlying anatomy. SA-CycleGAN-2.5D states this explicitly as the case in which the marginal image-intensity distribution $P(\mathbf{x})$ varies non-linearly across acquisition protocols while the conditional anatomy $P(\mathbf{y}|\mathbf{x})$ remains constant (Gowda et al., 17 Mar 2026). PRISM operationalizes the same idea by disentangling anatomical features from style and site-specific variations through a dual-branch autoencoder with contrastive learning and variational inference, enabling unpaired image translation without traveling subjects or multiple MRI modalities (Galada et al., 2024).

Across the literature, “site conditioning” is implemented in several distinct but related ways. Some models inject an explicit target-site descriptor, such as a site label or a learned site code. Others condition on a target exemplar image, a target latent representation, a target-domain density model, or DICOM metadata. The common objective is to preserve subject-specific structure while replacing acquisition-dependent appearance. This framing also extends beyond voxel images: in structural connectome harmonization, the target site is represented by a site-mapper latent vector fused with a site-invariant embedding before graph reconstruction (Patel et al., 18 Jul 2025).

Framework	Conditioned representation	Target condition
PRISM (Galada et al., 2024)	Anatomy map $A$ decoded with target style	Target style encoder output or mean target style
HCLD (Wu et al., 2024)	Source latent translated by conditional latent diffusion	Full latent of a target-domain volume
DIST-CLIP (Avci et al., 8 Dec 2025)	Anatomical code fused with contrast embedding	Target image or DICOM metadata
DDAE (Ijishakin et al., 2024)	Diffusion decoder conditioned on $(z_s,z_b)$	Site label mapped to site code
Harmonizing Flows (Beizaee et al., 2024)	Harmonizer adapted to a frozen source density prior	Source-domain normalizing flow
SC framework (Patel et al., 18 Jul 2025)	Site-invariant connectome embedding fused with site mapper	Target protocol/site index

This diversity of conditioning mechanisms indicates that “site-conditioned” is not tied to a single architecture. It denotes a family of deep generative systems that separate stable subject information from domain-specific acquisition effects and then steer synthesis toward a desired domain.

2. Architectural families

Early multi-site MRI harmonization systems were dominated by disentangled encoder-decoder and cycle-translation designs. MURD uses a shared content encoder $E^{C}$, site-specific style encoders $E^{S_i}$, a shared generator $G$, site-specific style generators $G^{S_j}$, and site-specific discriminators $D_j$ in a content–style disentangled cycle-translation scheme; site conditioning is realized by selecting which style encoder, style generator, and discriminator correspond to the target site (Liu et al., 2021). CALAMITI likewise separates anatomy and contrast through a $\beta$-encoder and a $P(\mathbf{y}|\mathbf{x})$0-encoder, then harmonizes by fixing $P(\mathbf{y}|\mathbf{x})$1 from the source and swapping in a target-site proxy $P(\mathbf{y}|\mathbf{x})$2 (Zuo et al., 2021).

PRISM refines this family through a modular dual-branch autoencoder. Its anatomy branch is a U-Net-style convolutional encoder with four down-sampling and four up-sampling steps, outputting $P(\mathbf{y}|\mathbf{x})$3 with $P(\mathbf{y}|\mathbf{x})$4, while its style branch is a shallow convolutional network whose global average pooled output parameterizes a Gaussian style code $P(\mathbf{y}|\mathbf{x})$5 with $P(\mathbf{y}|\mathbf{x})$6 (Galada et al., 2024). Harmonization from source $P(\mathbf{y}|\mathbf{x})$7 to target $P(\mathbf{y}|\mathbf{x})$8 is performed by combining $P(\mathbf{y}|\mathbf{x})$9 with a target style from $A$0 or with the mean style of $A$1, and decoding with $A$2.

Diffusion-based frameworks generalize this separation into stochastic latent or image-space generation. HCLD compresses 3D volumes into a “4D latent map” $A$3 and performs conditional latent diffusion on source content while concatenating the latent of a target-domain volume as a style key at each denoising step (Wu et al., 2024). DDAE separates a known-variance site code $A$4 from an unknown-variance content code $A$5 and conditions a diffusion decoder on both variables, with site labels mapped through $A$6 and injected by FiLM (Ijishakin et al., 2024). MMH uses a two-stage 3D conditional diffusion design: a sequence-specific global harmonizer maps images to a unified domain using style-agnostic gradient conditioning and sequence-specific EMA statistics, and a target-specific fine-tuner adapts globally aligned images to a desired target domain with a Tri-Planar Attention BiomedCLIP encoder (Wu et al., 13 Jan 2026).

Other lines of work emphasize alternative inductive biases. Harmonizing Flows replaces adversarial translation with exact density estimation via a coupling-based normalizing flow trained on the source domain and a shallow U-Net harmonizer adapted at test time so that its output matches the learned source density (Beizaee et al., 2024). DLEST separates a latent autoencoder from an energy-based model that performs Langevin style translation in latent space (Wu et al., 2024). DIST-CLIP conditions harmonization on either image exemplars or metadata prompts through pretrained MR-CLIP encoders and a novel Adaptive Style Transfer module inserted at the bottleneck and upsampling layers of a U-Net decoder (Avci et al., 8 Dec 2025). SA-CycleGAN-2.5D, by contrast, retains adversarial image translation but augments it with 2.5D tri-planar context, dense self-attention, and a spectrally-normalized discriminator to better model global scanner field biases (Gowda et al., 17 Mar 2026).

3. Objectives and disentanglement criteria

The loss structure of site-conditioned harmonization frameworks typically combines reconstruction, site-style regularization, anatomy preservation, and domain-confound suppression. In PRISM, the reconstruction loss contains an $A$7 image term, a perceptual term, and a cycle-style/anatomy consistency term,

$A$8

and the style branch is regularized by a variational KL loss together with a PatchNCE-style contrastive loss on the anatomy branch (Galada et al., 2024). The design explicitly ties anatomical invariance to contrastive patch matching between an image and its gamma-augmented version.

HCLD adopts a different decomposition. After AdaIN alignment of source and target latents, diffusion training minimizes a noise-prediction loss $A$9, a content preservation loss $(z_s,z_b)$0 computed after instance normalization, and a Gram-matrix style-alignment loss $(z_s,z_b)$1, with total loss $(z_s,z_b)$2 and $(z_s,z_b)$3 (Wu et al., 2024). DDAE is more austere: the full training objective is a single denoising loss $(z_s,z_b)$4, with disentanglement induced architecturally by forcing $(z_s,z_b)$5 to depend only on known covariates and $(z_s,z_b)$6 only on image appearance (Ijishakin et al., 2024).

Several frameworks strengthen the style signal beyond simple site labels. DIST-CLIP uses a patch-wise InfoNCE loss $(z_s,z_b)$7 for contrast-invariant anatomy, pixel and perceptual reconstruction losses, an adversarial image loss, and two CLIP-space style constraints: a global CLIP loss and a directional CLIP loss that aligns image-style displacement with metadata displacement (Avci et al., 8 Dec 2025). The scanner-agnostic 3D framework based on SSIM-guided disentanglement instead incorporates a differentiable SSIM decomposition into luminance, contrast, and structural terms and places $(z_s,z_b)$8 inside both reconstruction and cycle-consistency losses (Caldera et al., 24 Oct 2025).

A recurring point is that “disentanglement” is enforced by heterogeneous mechanisms rather than a single formal recipe. The field includes KL regularization of style codes, adversarial confusion of content embeddings, patch-level contrastive learning, covariance penalties, CLIP-space alignment, Gram-matrix style matching, gradient consistency, and metadata-conditioned modulation (Galada et al., 2024, Wu et al., 2024, Caldera et al., 24 Oct 2025, Wu et al., 13 Jan 2026). This suggests that site-conditioned harmonization is defined more by the operational separation of content and site factors than by any one loss family.

4. Training regimes, deployment models, and privacy

A major motivation for site-conditioned harmonization is the impracticality of traveling subjects. MURD was introduced precisely as multi-site harmonization without traveling human phantoms, using more than 6,000 multi-site T1- and T2-weighted images and training with 20 volumes per site from GE, Philips, and Siemens in the ABCD cohort (Liu et al., 2021). CALAMITI likewise avoids traveling subjects but requires paired intra-site contrasts, whereas PRISM explicitly enables unpaired image translation without traveling subjects or multiple MRI modalities (Zuo et al., 2021, Galada et al., 2024).

PRISM is distinctive in its deployment model. Each site trains its own $(z_s,z_b)$9 locally; no raw images or anatomy/style codes leave the site; and harmonization is performed when site $E^{C}$0 publishes only its style encoder weights $E^{C}$1 and decoder $E^{C}$2 (Galada et al., 2024). For a brand-new site $E^{C}$3, only local training of $E^{C}$4 is required, after which harmonization to any known target site is performed by downloading the target style encoder and decoder, with no global re-training. This indicates a model-sharing regime intended to preserve privacy while still allowing cross-site adaptation.

Other frameworks pursue different operational trade-offs. Harmonizing Flows is explicitly unsupervised, source-free, and task-agnostic: the normalizing flow is trained on the source domain, the harmonizer is pre-trained on augmented source images, and at inference the harmonizer alone is updated so that outputs conform to the frozen source-domain density, with stopping criteria such as matching $E^{C}$5 to the source $E^{C}$6 or minimum Shannon entropy of downstream-task predictions (Beizaee et al., 2024). DIST-CLIP trains with random alternation between image guidance and metadata guidance with $E^{C}$7, which enforces a shared embedding space and permits harmonization by metadata alone at inference time (Avci et al., 8 Dec 2025). MMH separates training into a global diffusion harmonizer and a target-specific fine-tuner, while CALAMITI adapts to unseen sites by freezing the decoder and $E^{C}$8-discriminator and fine-tuning only the late encoder blocks on the new site (Wu et al., 13 Jan 2026, Zuo et al., 2021).

Scalability is a further axis of variation. MURD emphasizes a single unified model for all $E^{C}$9 sites with complexity scaling as $E^{S_i}$0 rather than $E^{S_i}$1 (Liu et al., 2021). By contrast, pairwise translation systems without shared content-space abstractions can proliferate models as the number of sites grows. The more recent metadata- and repository-based approaches imply a shift from pairwise harmonization toward compositional, target-addressable systems (Avci et al., 8 Dec 2025, Galada et al., 2024).

5. Evaluation methodology and empirical behavior

Evaluation generally combines three questions: whether site signal is removed, whether anatomical or biological information is preserved, and whether downstream models improve. PRISM reports test-set reconstruction SSIM of 0.9768 for Guys, 0.9775 for HH, and 0.9524 for ADNI; after harmonization to Guys, structural SSIM with respect to the original image is 0.9786 for Guys→Guys, 0.9598 for HH→Guys, and 0.9266 for ADNI→Guys (Galada et al., 2024). Site-classification recall drops sharply after harmonization to the Guys target, with recall(HH) decreasing from 1.00 to 0.31 and recall(ADNI) from approximately 0.86 to 0.06. In downstream U-Net segmentation for ADNI→Guys, Gray Matter Dice improves from 0.8482 to 0.8614, White Matter Dice from 0.8108 to 0.9087, and CSF Dice from 0.8099 to 0.8828 (Galada et al., 2024).

HCLD evaluates site-confound removal and biological fidelity jointly. On OpenBHB, site balanced accuracy decreases from 0.552 to 0.289, while age MAE decreases from 6.624 years to 5.245 years; on SRPBS and IXI, HCLD reports inter-site SSIM of approximately 0.937 and 0.612, PSNR of approximately 29.5 dB and 19.3 dB, PCC of approximately 0.995 and 0.955, and WD of approximately 0.004 and 0.007 (Wu et al., 2024). DDAE broadens the metric set further: FID is 7.40, site-classifier accuracy is $E^{S_i}$2, age $E^{S_i}$3 is $E^{S_i}$4, sex accuracy is $E^{S_i}$5, and PCC is 0.982, outperforming ComBat, cVAE, and Style-GAN in the reported table (Ijishakin et al., 2024).

The strongest recent diffusion results are reported by MMH. Against ground-truth traveling scans at the UCI target site, MMH achieves T1 SSIM $E^{S_i}$6, PSNR $E^{S_i}$7 dB, PCC $E^{S_i}$8, and WD $E^{S_i}$9, and T2 SSIM $G$0, PSNR $G$1, PCC $G$2, and WD $G$3 (Wu et al., 13 Jan 2026). In feature space, inter-site distance collapses from 4.62 to 1.19 for T1 and from 7.87 to 1.37 for T2 after harmonization, while subject clusters remain. On OpenBHB, site balanced accuracy falls to 15.3% and age MAE reaches 5.22 years (Wu et al., 13 Jan 2026).

Adversarial image-translation systems produce a different empirical profile. SA-CycleGAN-2.5D reduces Maximum Mean Discrepancy from 1.729 to 0.015, a 99.1% drop, and a ResNet-18 domain-classifier falls from 98.4% accuracy on raw slices to 59.7% after harmonization (Gowda et al., 17 Mar 2026). Its self-attention ablation raises cycle SSIM from 0.9282 to 0.9392 and PSNR by 1.01 dB, with Cohen’s $G$4 and $G$5 in the harder translation direction. In the structural connectome setting, the graph autoencoder variant achieves fingerprinting accuracy 0.99 and identifiability difference 2.53, showing that site-conditioned harmonization can also be evaluated by topology preservation and subject individuality rather than only voxel-level image similarity (Patel et al., 18 Jul 2025).

6. Limitations, misconceptions, and future directions

A persistent misconception is that strong domain alignment automatically implies acceptable harmonization. Several results argue against this. Harmonizing Flows reports that Wasserstein distance between intensity histograms is correlated with DSC and MAE except for naïve histogram matching, which minimizes WD but destroys spatial consistency (Beizaee et al., 2024). SA-CycleGAN-2.5D notes that ComBat attains MMD $G$6 yet leaves domain-classifier accuracy at 75% and cannot produce harmonized images for spatial tasks (Gowda et al., 17 Mar 2026). DDAE shows an even sharper tension: cVAE yields site accuracy $G$7 but age $G$8 and PCC $G$9, indicating that aggressive confound removal can coincide with poor preservation of biological and individual variation (Ijishakin et al., 2024).

Another misconception is that all modern harmonization is volumetric. PRISM explicitly lists “2D slices only” as a limitation, noting that extension to 3D volumes increases memory (Galada et al., 2024). By contrast, HCLD, MMH, and the scanner-agnostic SSIM-guided framework operate on 3D data, but they also require more elaborate latent compression, diffusion schedules, or normalization machinery (Wu et al., 2024, Wu et al., 13 Jan 2026, Caldera et al., 24 Oct 2025). The field therefore still trades off volumetric coherence against computational cost and modularity.

Future directions are already explicit in the literature. PRISM proposes joint harmonization plus downstream task learning in a federated loop, multi-modal disentanglement such as T1 plus FLAIR style codes, and a learned site-code repository enabling zero-shot harmonization to unseen scanners (Galada et al., 2024). DIST-CLIP proposes full 3D AST modules and integration of downstream tasks into the training loop for end-to-end site-generalized pipelines (Avci et al., 8 Dec 2025). MMH extends site conditioning into sequence-aware, semantically enriched diffusion by combining normalized gradient conditioning, sequence-specific EMA style modeling, and 3D semantic style priors via TPA-CLIP (Wu et al., 13 Jan 2026). In graph harmonization, future work is suggested to incorporate edge-edge convolutions, GAN-style discriminators on graphs, and losses penalizing spurious long-range links (Patel et al., 18 Jul 2025).

Taken together, these developments position site-conditioned deep harmonization as a unifying design principle rather than a single algorithm. Its defining operation is the controllable fusion of subject-preserving content with a specified acquisition-domain condition, and its technical frontier lies in making that fusion more modular, more privacy-compatible, more anatomically conservative, and more effective for downstream scientific inference across heterogeneous multi-site data.