Collaborative Domain Adaptation

• Collaborative domain adaptation is a set of transfer-learning methods that jointly exploit labeled source data and unlabeled or weakly labeled target data using multiple interacting models.
• Key mechanisms include adversarial alignment, consistency regularization, peer network disagreement, and separation of domain-specific and invariant representations to handle distribution shifts.
• This approach is applied in semantic segmentation, medical imaging, and federated learning, offering improved performance, privacy guarantees, and efficient deployment.

Collaborative domain adaptation denotes a family of transfer-learning methods in which adaptation is not entrusted to a single discrepancy-minimization module, but to interacting models, branches, experts, or objectives that jointly exploit labeled source information and unlabeled or weakly labeled target data. In the cited literature, collaboration appears as adversarial alignment plus self-ensembling, peer networks that estimate transferability from disagreement, mixtures of global and private experts, expert–student bridges across multiple target domains, bidirectional model–graph refinement, and joint use of multiple foundation models (Liu et al., 2019, Peterson et al., 2019, Zhang et al., 2024, Lee et al., 24 Nov 2025).

1. Problem formulation and scope

In the standard unsupervised domain adaptation setting used repeatedly across these works, the source domain provides labeled samples $(X_s, Y_s)$, the target domain provides unlabeled samples $X_t$, and the core difficulty is distribution shift, typically expressed as $P(X_s) \neq P(X_t)$ (Liu et al., 2019). In semantic segmentation, this shift is induced by changes in color distribution, contrast, illumination, noise, texture, image resolution, and field-of-view; in medical imaging it also reflects scanner, protocol, and population heterogeneity (Liu et al., 2019, Gao et al., 30 Jul 2025).

The collaborative formulation broadens this template in several directions. In federated settings, each user $i$ owns a private labeled dataset $\mathcal{D}_i$ drawn from a user-specific domain distribution $P_i(X,Y)$, so the data are non-IID across clients (Peterson et al., 2019). In source-free settings, adaptation proceeds without access to source samples and relies only on a pretrained source model and unlabeled target data (Fang et al., 2023, Zhang et al., 2024). In universal semi-supervised model adaptation, the source and target label spaces may differ, with common and private label subsets, while only a few labeled target samples are available (Yan et al., 2023). Multi-source and multi-target variants further replace the single source–single target pair by several labeled sources, several unlabeled targets, or both (He et al., 2021, Isobe et al., 2021).

A useful synthesis is that collaboration in this literature is not a single algorithmic primitive. It is a design principle for exploiting complementary inductive biases under domain shift: some modules preserve domain-specific structure, some enforce domain invariance, some stabilize training with historical averages, and some convert disagreement into supervision or weighting signals.

2. Core collaborative mechanisms

One recurrent mechanism is the combination of adversarial alignment and consistency regularization. In collaborative feature ensembling for optic disc and cup segmentation, the architecture contains a source domain network, a target domain student network, a target domain teacher network, and two domain discriminators, one on encoder features and one on decoder outputs (Liu et al., 2019). The teacher parameters are updated by exponential moving average,

$$\Phi_t^{\prime}=\alpha\Phi_{t-1}^{\prime}+(1-\alpha)\Phi_t,$$

and consistency is enforced at both encoder and decoder levels by mean-squared error losses on target data (Liu et al., 2019). CADA extends the same pattern with multi-scale input training and multiple domain adaptors applied hierarchically in feature and output spaces, combining adversarial learning at multi-scale outputs with EMA-based weight ensembling to reduce the uncertainty of adapting multiple discriminator learning (Liu et al., 2021).

A second mechanism uses disagreement between peer models as a transferability signal. In collaborative unsupervised domain adaptation for medical diagnosis, two peer networks are trained jointly, their prediction inconsistency is measured by

$\lambda = 1 - \cos(\hat{y}_1, \hat{y}_2),$

and this sample-wise weight is used to emphasize hard-to-transfer samples in the domain adversarial loss (Zhang et al., 2020). The same framework shares a noise co-adaptation layer that models feature-dependent label corruption, and it maximizes classifier diversity through Jensen–Shannon divergence so that disagreement remains informative rather than collapsing trivially (Zhang et al., 2020).

A third mechanism explicitly separates domain-specific and domain-invariant representation learning across network depth. CAN formulates domain-collaborative learning and domain-adversarial learning uniformly as domain classifier learning with positive or negative weights on the losses, learns domain-specific representations from lower CNN blocks, and learns domain-invariant representations from higher blocks; SPCAN adds self-paced pseudo-labeled target selection in an easy-to-hard fashion (Zhang et al., 24 Jun 2025). This suggests that collaboration can be internal to a single backbone rather than only between separate networks.

3. Structural variants of collaboration

The literature supports several recurring collaboration topologies.

Setting	Collaborative mechanism	Representative papers
Multi-source UDA	Source-specific models, cross-source consistency, target ensemble pseudo labels	(He et al., 2021)
Multi-target UDA	Expert per source-target pair, bridge across targets, student imitation of experts	(Isobe et al., 2021)
Source-free UDA	Multi-branch consistency, model–graph co-adaptation, multi-model prototype guidance	(Fang et al., 2023, Zhang et al., 2024, Fan et al., 1 Jun 2026)
Foundation-model-guided SFDA	Bidirectional adaptation between target model and multiple foundation models	(Lee et al., 24 Nov 2025)

In multi-source semantic segmentation, one model is instantiated per labeled source domain, each model is supervised on its own source, and collaborative KL consistency forces it to match the predictions of other source-native models on their images; unlabeled target data are then handled through pseudo labels generated online by an ensembled model (He et al., 2021). In multi-target adaptation, an expert model is first trained for each source-target pair, experts are encouraged to collaborate through a bridge built between different target domains, and a student model is trained to imitate the output of each expert on the corresponding target domain while also pulling different experts close with regularization on their weights (Isobe et al., 2021).

Source-free collaboration takes several forms. SCDA for fMRI uses multiple collaborative branches, each with a data-feeding module, a spatiotemporal feature encoder, and a class predictor, and trains them with pair-wise consistency of latent features and logits for the same input under different temporal transformations (Fang et al., 2023). GraphCTA couples model adaptation and graph adaptation in a bi-directional loop: the model uses neighborhood predictions, prototypes, and memory banks, while the graph is refined by updating structure and node attributes via neighborhood contrastive learning (Zhang et al., 2024). UCDA for driving scene parsing aggregates multiple pretrained source models through a class-level prototype memory bank and a two-stage refine-and-distill strategy, using prototype similarity to estimate cross-model prediction reliability on unlabeled target images (Fan et al., 1 Jun 2026).

Foundation-model collaboration pushes the idea further. CoMA jointly leverages two different foundation models, such as CLIP and BLIP, aligns them with the target model for task adaptation while maintaining their semantic distinctiveness, and transfers complementary knowledge back to the target model. Its Decomposed Mutual Information enhances dependencies in confident class subsets and suppresses false dependencies from incomplete class coverage under mini-batch training (Lee et al., 24 Nov 2025).

4. Representative application domains

Semantic segmentation is the most technically mature application in the cited material. In retinal optic disc and cup segmentation, CFEA reports an optic cup Dice of $0.8627$, an optic disc Dice of $0.9416$, and a CDR MAE of $X_t$0, improving over both source-only training and AdaptSegNet on the REFUGE-based cross-camera setting (Liu et al., 2019). CADA further targets REFUGE, Drishti-GS, and Rim-One-r3 with multi-scale inputs, multiple adversarial losses in encoder and decoder layers, and EMA-based ensembling of historical weights, all without annotating any sample from the target domain (Liu et al., 2021).

In multi-source driving-scene segmentation, collaborative learning with labeled Synscapes and GTA5 plus unlabeled Cityscapes achieves $X_t$1 mIoU on the validation set of Cityscapes, and the reported ablations show that source collaboration, target pseudo-label collaboration, and LAB-based image translation are complementary (He et al., 2021). More recent source-free driving-scene parsing extends collaboration to multiple pretrained source models: UCDA reports $X_t$2 mIoU on Cityscapes with a ResNet-101 teacher ensemble distilled into a unified target model, $X_t$3 mIoU with MiT-B5 on Cityscapes, and $X_t$4 mIoU on a real NIO vehicle dataset, while maintaining identical inference cost because only one target model is deployed (Fan et al., 1 Jun 2026).

Medical imaging supplies a second major application family. In rs-fMRI analysis, SCDA uses window warping, receptive field manipulation, and window slicing as three perspectives, and reports cross-scanner AUC $X_t$5 and ACC $X_t$6 on ABIDE, together with strong cross-study results on REST-meta-MDD, ADHD-200, and a private T2DM cohort (Fang et al., 2023). In late-life depression assessment from structural MRI, CDA uses a ViT branch for global anatomical context and a CNN branch for local structural features, combines supervised source training, discrepancy-based target feature adaptation, and collaborative pseudo-label training, and reports AUC $X_t$7 and ACC $X_t$8 for CN-D versus CN-N on the target cohort (Gao et al., 30 Jul 2025).

Lightweight deployment is another recurring motivation. CLDA observes Domain Shift induced Non-salient parameters in overparameterized teachers, uses layer saliency to update only non-salient teacher layers from the student through EMA, and improves both teacher and student on GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes; for example, on GTA5-to-Cityscapes it reports $X_t$9 mIoU for the teacher and $P(X_s) \neq P(X_t)$0 mIoU for the student (Cho et al., 2024).

5. Privacy, personalization, and source-free collaboration

A frequent misconception is that collaboration requires source-data sharing. The literature shows the opposite. Private federated learning with domain adaptation trains a global model $P(X_s) \neq P(X_t)$1 through FL and per-user private models $P(X_s) \neq P(X_t)$2, then combines them by a gating function,

$P(X_s) \neq P(X_t)$3

so that each user receives a personalized predictor without sharing raw local data (Peterson et al., 2019). The global model is trained with DP-SGD, while private experts and gates are optimized locally without DP noise (Peterson et al., 2019). On the spam dataset, when the noise multiplier is $P(X_s) \neq P(X_t)$4, plain FL drops from $P(X_s) \neq P(X_t)$5 to $P(X_s) \neq P(X_t)$6, whereas FL+DE remains at $P(X_s) \neq P(X_t)$7, showing that the collaborative global–private decomposition is especially effective under privacy constraints (Peterson et al., 2019).

Source-free collaboration generalizes the same principle beyond federated learning. SCDA assumes only a pretrained source model and unlabeled target fMRIs (Fang et al., 2023). GraphCTA assumes only a source-pretrained GNN and an unlabeled target graph (Zhang et al., 2024). UCDA assumes multiple pretrained source segmentation models but no original source images (Fan et al., 1 Jun 2026). CoMA assumes a pretrained source model, unlabeled target data, and external foundation models, again without source samples (Lee et al., 24 Nov 2025). CMDA-OT combines per-source optimal-transport adaptation with centralized collaborative aggregation of models from multiple sources without accessing their data, and reports average accuracy $P(X_s) \neq P(X_t)$8 on VLSC and $P(X_s) \neq P(X_t)$9 on Office-Caltech10 (Ghannou et al., 2024).

These variants suggest that collaboration is increasingly being treated as a model-space operation rather than a data-centralization operation. The shared objects are predictions, prototypes, prompts, or parameters; the private objects remain raw source samples and annotations.

6. Empirical regularities, misconceptions, and open issues

Several empirical patterns recur. First, collaboration is often more beneficial when the target domain is weakly supervised, unlabeled, privacy-constrained, or noisy. CoUDA improves over MCD on noisy histopathology adaptation, reaching $i$0 accuracy and $i$1 macro F1 while explicitly modeling transferability and label noise (Zhang et al., 2020). CoMA improves over prior source-free foundation-model baselines on Office-Home from $i$2 to $i$3, on DomainNet-126 from $i$4 to $i$5, and on VisDA from $i$6 to $i$7 (Lee et al., 24 Nov 2025). UCDA reports that using more source models improves performance, and its cross-target generalization average mIoU of $i$8 exceeds IAPC, SND, and ATP when adapting on Cityscapes and testing on Cityscapes, BDD100K, and ACDC (Fan et al., 1 Jun 2026).

Second, collaborative domain adaptation is not synonymous with adversarial alignment. Some methods are explicitly adversarial, such as CFEA, CADA, CAN, and CoUDA (Liu et al., 2019, Liu et al., 2021, Zhang et al., 24 Jun 2025, Zhang et al., 2020). Others are centered on consistency training, such as SCDA and USMA (Fang et al., 2023, Yan et al., 2023). Others are built around mixtures of experts or local–global personalization (Peterson et al., 2019). GraphCTA makes the graph itself adaptive, and CoMA makes multiple foundation models collaborate through mutual-information objectives (Zhang et al., 2024, Lee et al., 24 Nov 2025).

Third, collaboration does not necessarily imply higher deployment cost. In CFEA, at test time, only TTN, a standard U-Net, is used (Liu et al., 2019). UCDA distills refined expertise into a single deployable target model (Fan et al., 1 Jun 2026). CoMA likewise uses only the target model $i$9 at inference, even though adaptation involves CLIP and BLIP during training (Lee et al., 24 Nov 2025).

The limitations identified across the cited work are consistent. Multi-network training raises computational cost, especially when several segmentation networks and discriminators are optimized jointly (Liu et al., 2019). Performance depends on confidence thresholds, loss weights, and update schedules (Liu et al., 2019, Gao et al., 30 Jul 2025). Source-free methods often assume white-box access to source model parameters or structural correspondence between teacher and student backbones (Fang et al., 2023, Cho et al., 2024). Several papers explicitly note the absence of formal generalization or convergence analyses for their collaborative procedures (Peterson et al., 2019, Cho et al., 2024).

A plausible implication is that future progress will depend less on introducing yet another single discrepancy term and more on learning when collaboration should be symmetric, asymmetric, or selective. The cited directions already point that way: black-box source-free adaptation and richer feature enrichment for fMRI (Fang et al., 2023); theoretical analysis of non-salient parameters in teacher–student collaboration (Cho et al., 2024); alternative collaborative mechanisms, meta-learning, and PATE-like privacy strategies in federated adaptation (Peterson et al., 2019); and broader multi-source or multi-target extensions in semantic segmentation and source-free perception (Isobe et al., 2021, Fan et al., 1 Jun 2026).