---
title: 'MultiVesSeg: Domain Adaptation for Brain Vessels'
url: https://www.emergentmind.com/topics/multivesseg
type: topic
---

# MultiVesSeg: Domain Adaptation for Brain Vessels

MultiVesSeg is an **out-of-the-box semi-supervised domain adaptation framework** for **brain vessel segmentation across domains**. It is designed for segmentation of the cerebrovascular tree across **different medical centers**, **different imaging modalities**, **different vessel types**, and **different acquisition characteristics**, while avoiding **domain-specific model design** and **data harmonization between the source and the target domain**. Its central mechanism is **feature disentanglement** implemented through a **StyleGAN2-based shared latent space** and **label-preserving image-to-image translation**, so that vessel appearance can be manipulated during adaptation while spatial information such as **shapes** and **locations** is preserved [2510.00665].

## 1. Problem setting and domain-shift model

MultiVesSeg addresses **brain vessel segmentation under domain shift**, where a model trained on one domain does not generalize well to another. The paper studies three concrete transfer settings: **multi-center MRA** from **TOF MRA from OASIS-3 \(\to\) TOF MRA from IXI**, **MRA-to-CTA** from **TOF MRA \(\to\) CTA**, and **MRA-to-MRV** from **TOF MRA arterial scans \(\to\) SWI venography**. These shifts are not treated as only low-level appearance changes. The framework is explicitly motivated by variation in **imaging modality**, **medical center / scanner / protocol**, **vessel type**, and **acquisition and appearance factors** such as **intensity range**, **contrast**, **noise**, **pixel spacing / spatial resolution**, **texture**, and **vessel brightness polarity** [2510.00665].

The problem is difficult because **cerebral vessels are small and sparse relative to the full 3D volume**, vascular morphology is **complex and highly branching**, and domain gaps differ in severity. The paper characterizes **MRA vs MRA from another center** as mild, **MRA vs CTA** as moderate, and **MRA vs MRV/SWI** as very large, because arteries and veins differ not only in appearance but also in **geometry** and **anatomical distribution**. A naive image translation is therefore inadequate: if translation changes **vessel position**, **thickness**, **density**, or **shape**, the source labels cease to be valid. MultiVesSeg is built around the constraint that adaptation should alter vessel appearance in a **label-preserving manner**, rather than maximize visual realism.

The paper also distinguishes **volume-level properties** and **vessel-level properties**. Volume-level properties include **spacing**, **scale**, **contrast**, and **resolution**; vessel-level properties include **intensities**, **texture**, **shapes**, **locations**, and **densities**. This distinction is important because the method does not simply translate all domain-specific factors. Instead, it seeks to translate only a subset of factors compatible with segmentation supervision.

## 2. Disentangled latent space and model architecture

The framework has four principal components: a **Generator \(G\)** based on **StyleGAN2**, a **Discriminator \(D\)** used during latent-space learning, an **Encoder \(E\)** that maps an image into the StyleGAN latent space, and a **label-synthesis branch \(G_{lsb}\)** adapted from **DatasetGAN**. The generator learns a shared latent space \(\mathcal{W}\), more precisely \(\mathcal{W}^+\) on the encoder side, spanning both source and target domains. The encoder receives a **binary domain flag \(d\)**, with \(d=0\) meaning encode toward the **source-domain latent representation** and \(d=1\) meaning encode toward the **target-domain latent representation**. The label-synthesis branch is implemented as **three fully connected layers** attached to generator features and predicts segmentation masks aligned with generated images [2510.00665].

The encoder uses a **ResNet backbone** inspired by **pixel2style2pixel / residual encoder designs**. It predicts both a **latent code** in \(\mathcal{W}^+\) and a **feature tensor**, and it uses a **dynamic skip connection module** to connect encoder features into the generator and preserve fine content correspondence. The paper’s architectural interpretation is that these residual and skip pathways are essential for preserving spatial information during domain translation.

Disentanglement is induced functionally rather than through explicitly separated content and style vectors. The intended separation is between **domain-invariant structural/spatial information**—including **vessel positions**, **vessel shapes**, and anatomy needed for segmentation—and **domain-specific appearance information** such as **intensities**, **vessel darkness/brightness**, **textures**, and **contrast**. The method is careful not to force translation of properties that would violate label preservation, especially in **artery-to-vein adaptation**, where vessel geometry and density may differ intrinsically.

The latent space is regularized by **path length regularization**. The Phase 1 objective is
\[
\mathcal{L}_{tot} = \mathcal{L}_{adv}(G,D) + \mathcal{L}_{R_1}(D) + \mathcal{L}_{pl}(G),
\]
where \(\mathcal{L}_{adv}\) is the **non-saturating GAN loss**, \(\mathcal{L}_{R_1}\) is **\(R_1\) regularization**, and
\[
\mathcal{L}_{pl} = \mathbb{E}\left[\left(\|\nabla_w G(w)\|_2 - a\right)^2\right]
\]
encourages small latent changes to produce smooth, semantically structured image changes. In the paper’s formulation, this regularization is what makes independent manipulation of image properties possible.

## 3. Label-preserving translation and optimization

Training proceeds in **two phases**. In **Phase 1**, \(G\) and \(D\) are trained adversarially on both source and target images for **250,000 iterations** in order to learn a **unified latent space \(\mathcal{W}\)** covering both domains. In **Phase 2**, \(G\) is frozen except for \(G_{lsb}\), and \(E\) is trained for **20,000 iterations** to perform **label-preserving image-to-image translation** and segmentation. The implementation details also mention a **preliminary phase of 15,000 iterations before Phase 2** using only source data [2510.00665].

The semi-supervised setting uses three sets:
\[
S = \{(x_i^s, y_i^s)\}_{i=1}^{N},
\qquad
T_U = \{x_i^t\}_{i=1}^{M},
\qquad
T_L = \{(x_i^{lt}, y_i^{lt})\}_{i=1}^{m},
\]
with \(m \ll M\). No paired source-target scans are required. For a source image \(x_i^s\), the encoder can produce both an intra-domain reconstruction and an inter-domain translation:
\[
\hat{x}_i^{s}=G(w_i^{s})=G(E(x_i^{s} \mid d=0)),
\]
\[
\hat{x}_i^{t}=G(w_i^{t})=G(E(x_i^{s} \mid d=1)).
\]
The same logic is used symmetrically for target images. The label-synthesis branch predicts segmentation masks for both reconstruction and translation outputs, and both are supervised by the **same reference annotation** whenever translation is intended to be label-preserving.

The segmentation loss is
\[
\mathcal{L}_{s} = \mathcal{L}_{dice} + \mathcal{L}_{ce},
\]
with
\[
\mathcal{L}_{dice} = -\frac{2\cdot \mathrm{TP} + \epsilon}{2\cdot \mathrm{TP} + \mathrm{FP} + \mathrm{FN} + \epsilon},
\qquad \epsilon = 10^{-5}.
\]
The reconstruction loss is
\[
\mathcal{L}_{r} = \mathcal{L}_{MSE} + \mathcal{L}_{LPIPS}.
\]
The paper does not provide an explicit single weighted Phase 2 formula, but it states that \(E\) is updated using \(\mathcal{L}_{r}\) and \(\mathcal{L}_{s}\), while \(G_{lsb}\) is updated using \(\mathcal{L}_{s}\).

At test time, a target image \(x^t\) is encoded twice: once with \(d=1\) to obtain a target-side prediction \(\hat{y}^t\), and once with \(d=0\) to obtain a source-side prediction \(\hat{y}^s\). The final segmentation is produced by averaging \(\hat{y}^t\) and \(\hat{y}^s\), followed by **argmax**. This dual-view inference is one of the concrete mechanisms through which MultiVesSeg combines source and target latent interpretations.

The framework operates in a **2.5D approach**, meaning volumetric data are processed as a series of **multi-channel 2D slices** rather than full 3D volumes. This is a methodological choice rather than an incidental implementation detail, and it directly conditions the model’s computational profile and its handling of cross-slice continuity.

## 4. Data, preprocessing, and evaluation protocol

The source domain is **OASIS-3**, comprising **49 TOF MRA volumes** with median size \(576 \times 768 \times 232\) and median voxel size \(0.30 \times 0.30 \times 0.60\) mm. The target domains are **IXI** with **50 TOF MRA volumes**, **TopCoW** with **40 CTA volumes**, and a **private SWI venography dataset** with **28 SWI volumes**. For OASIS-3, the split is **35 volumes train, 7 validation, 7 test**. For the target domains, the paper uses unlabeled training sets \(T_U\), validation sets, test sets, and a very small labeled subset \(T_L\): **3 midpoint slices** for IXI, **3 whole target volumes** for TopCoW because annotations only cover the **circle of Willis**, and **3 midpoint slices** for the SWI dataset [2510.00665].

Preprocessing is done **separately per dataset**, without inter-domain harmonization. The steps are: **skull stripping / brain masks via SynthStrip**; resampling each dataset to a fixed spacing equal to that dataset’s **median spacing**, with minor increases if needed to fit in \(512^3\); normalization using **mean/std**; intensity clipping to the **0.1 and 99.9 percentiles**; rescaling to \([-1, +1]\); and one-hot encoding segmentation masks into **three classes**: **background**, **brain**, and **vessels**. The paper stresses that it avoids explicit cross-domain harmonization.

The evaluation uses **Dice**, **clDice**, **ASSD**, **Precision**, and **Recall**. The baselines are **2D U-Net**, **CycleGAN**, **SIFA**, **SynthSeg**, **UniverSeg**, **AADG**, **DCDA**, and **CS-CADA**. The paper notes fairness caveats because some comparison methods were originally designed for larger structures or 2D settings, but they were adapted where possible.

The main transfer settings are summarized below.

| Scenario | Target domain | Target supervision |
|---|---|---|
| Multi-center MRA | IXI | \(T_L\): 3 midpoint slices |
| MRA-to-CTA | TopCoW | \(T_L\): 3 whole target volumes |
| MRA-to-MRV | SWI private dataset | \(T_L\): 3 midpoint slices |

This experimental design makes the method explicitly **semi-supervised domain adaptation** rather than domain generalization. The paper also studies **unsupervised adaptation** when \(m=0\) and a **few-shot target-only segmentation** condition when \(N=0\), but the main framework is formulated for the SSDA regime.

## 5. Quantitative results and ablation findings

On the source OASIS-3 domain, the reported intra-domain performance is **Dice 73.7 ± 2.8**, **Precision 66.9 ± 4.8**, **Recall 82.5 ± 3.7**, and **clDice 76.9 ± 5.2**. The authors state that this is comparable to state-of-the-art artery segmentation methods. In the transfer experiments, MultiVesSeg achieves **Dice 69.9 ± 2.3**, **clDice 76.8 ± 2.9**, and **ASSD 1.55 ± 0.57** for **OASIS-3 \(\to\) IXI**; **Dice 74.5 ± 4.2**, **clDice 78.0 ± 8.7**, and **ASSD 1.46 ± 0.49** for **OASIS-3 \(\to\) TopCoW**; and **Dice 67.5 ± 1.7**, **clDice 69.9 ± 2.7**, and **ASSD 1.42 ± 0.20** for **OASIS-3 \(\to\) SWI** [2510.00665].

| Transfer setting | MultiVesSeg result | Noted comparison |
|---|---|---|
| OASIS-3 \(\to\) IXI | Dice **69.9 ± 2.3**, clDice **76.8 ± 2.9**, ASSD **1.55 ± 0.57** | best or near-best among all methods |
| OASIS-3 \(\to\) TopCoW | Dice **74.5 ± 4.2**, clDice **78.0 ± 8.7**, ASSD **1.46 ± 0.49** | better than U-Net (**70.5 Dice**) and SIFA (**60.9**) |
| OASIS-3 \(\to\) SWI | Dice **67.5 ± 1.7**, clDice **69.9 ± 2.7**, ASSD **1.42 ± 0.20** | huge margin over all baselines |

The **MRA-to-MRV** setting is the most striking. The reported baselines there are **U-Net 29.1 Dice**, **SynthSeg 10.9**, **CycleGAN 5.1**, **SIFA 0.8**, **DCDA 0.0**, **CS-CADA 0.4**, and **UniverSeg 3.6**. The paper interprets this as evidence that disentanglement plus label-preserving translation is especially effective when the domain shift involves both appearance and vessel-type changes.

The ablation studies are concentrated on the hardest **MRA-to-MRV** setup. Increasing target annotations from **\(m=0\)** to **\(m=1\) slice** gives a large improvement, after which gains **saturate**; the authors therefore use **\(m=3\)** for the main experiments. On the source side, going from **\(N=0\)** to **\(N=10\)** source volumes gives a sharp improvement, while increasing to **\(N=35\)** yields only a modest further boost of **+3.4% Dice** in the hardest setting. This suggests that MultiVesSeg is annotation-efficient on the target domain but still strongly benefits from source supervision.

Architectural ablations further identify the decisive components. In **MRA-to-MRV**, the **full model** achieves **Dice 72.2 ± 2.5** and **clDice 75.4 ± 3.3**. Removing **residual connections** reduces Dice to **14.4 ± 3.4**. Removing **DSBN** gives **Dice 69.3**; removing **balanced data sampling** gives **Dice 71.2**; removing **intensity inversion** gives **Dice 71.8**. The paper interprets this as showing that **residual connections are crucial**, likely because they preserve spatial information and support fine correspondences. In the **unsupervised** MRA-to-MRV case, removing **intensity inversion** causes a severe collapse, indicating that some appearance guidance is needed when the shift is extremely wide.

Qualitative findings are aligned with the quantitative results. The paper emphasizes improved preservation of **vessel continuity**, **complex branching patterns**, **spatial alignment**, and **thin vessel recovery** relative to alternatives. It also states that the translated images are often **hybrid translations** rather than perfectly source-like or target-like images. This is not treated as a flaw; the paper is explicit that **segmentation utility** rather than visual realism is the objective.

## 6. Position within vessel-segmentation research, limitations, and future directions

MultiVesSeg occupies a specific position within current segmentation research. It is a **domain adaptation** method for **brain vessels**, not a universal one-model-no-retraining framework. This distinguishes it from **UVSM**, which targets **multi-modality retinal vessel segmentation** across **CF**, **MC**, **FA**, **FAF**, and **IR/NIR** using a canonical-domain translation pipeline and a topology-aware segmentation stage without target-modality finetuning [2502.06987]. It is also distinct from **MDSVM-UNet**, which addresses **3D coronary artery segmentation from CTA** through **multidirectional snake convolution**, **residual visual Mamba**, and a **two-stage coarse-to-fine** pipeline rather than cross-domain adaptation [2603.21829]. In another direction, **MultiFlow** addresses **multi-vessel classification and segmentation** from **phase-contrast MRI** in single-ventricle patients, combining **joint vessel-plane classification**, **2D+time segmentation**, and downstream **deep temporal clustering** [2502.11993]. The name should also not be conflated with **MultiverSeg**, an interactive biomedical segmentation system based on **in-context guidance** rather than vessel-domain adaptation [2412.15058].

The paper explicitly discusses several limitations. First, MultiVesSeg still requires **retraining for each new target domain**. It is therefore not a domain generalization method. Second, for very large shifts—especially **MRA-to-MRV**—the method still benefits substantially from **a small number of target labels**, and completely unsupervised adaptation may require weak appearance guidance such as **intensity inversion**. Third, training is computationally costly: approximately **194 hours (~8 days)** in total on **two NVIDIA GeForce RTX 2080 Ti GPUs, 12 GB each**, with **Phase 1** taking about **166 hours** and **Phase 2** about **28 hours**. Fourth, the method is **2.5D rather than full 3D**, which may affect long-range vessel continuity across slices. Fifth, several implementation details remain unspecified in the manuscript, including the **optimizer for MultiVesSeg itself**, **learning rates**, **weight decay**, **exact Phase 2 loss weights**, and the full augmentation protocol.

The proposed future directions are correspondingly clear. The paper points to moving beyond repeated domain adaptation toward **domain generalization** or **in-context learning**, leveraging **foundation models** pre-trained on large collections of tree-like structures, reducing dependence on target annotations, and improving cross-domain vessel linking without manual guidance. A plausible implication is that MultiVesSeg is best understood as a strong transitional framework: it shows that **label-preserving translation in a disentangled latent space** can bridge large shifts across **centers**, **modalities**, and **vessel types**, while also exposing the remaining gap between semi-supervised adaptation and truly domain-agnostic cerebrovascular segmentation.

Source: https://www.emergentmind.com/topics/multivesseg