Vascular Anatomy-aware Masked Image Modeling (VasoMIM)

• The paper introduces a self-supervised masked image modeling framework that integrates an anatomy-guided masking strategy with an anatomical consistency loss to target vascular structures.
• It employs a weak-to-strong schedule that gradually shifts from random masking to vessel-focused masking, enhancing reconstruction of thin, sparse vessels.
• Empirical results show state-of-the-art performance in vessel segmentation and detection across multiple X-ray angiogram datasets, underscoring its practical impact.

Vascular Anatomy-aware Masked Image Modeling (VasoMIM) is a self-supervised masked image modeling framework for X-ray angiograms that incorporates vascular anatomical knowledge into pretraining in order to counter the severe imbalance between sparse vessel pixels and dominant background pixels. In its 2025 abstract presentation, VasoMIM is defined by two components—an anatomy-guided masking strategy and an anatomical consistency loss—and is reported to achieve state-of-the-art performance across three datasets for vessel segmentation (Huang et al., 14 Aug 2025). A later, more expansive presentation describes VasoMIM as vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis, introduces the XA-170K pretraining corpus, and evaluates transfer on four downstream tasks across six datasets (Huang et al., 12 Feb 2026). This suggests an evolution from a vessel-segmentation-centered formulation toward a broader angiogram representation-learning framework.

1. Problem setting and conceptual rationale

VasoMIM is motivated by a specific failure mode of generic self-supervised learning in angiography. X-ray angiography is the clinical gold standard for diagnosing and guiding treatment of cardiovascular disease, yet automatic analysis is constrained by low contrast, motion artifacts, overlapping anatomical structures, and the need to recover fine vascular topology and subtle stenoses. Supervised deep models for vessel segmentation, vessel-segment labeling, stenosis segmentation, and stenosis detection depend on costly pixel-level annotations from experts, so unlabeled angiography data are substantially easier to scale than expert annotations (Huang et al., 12 Feb 2026).

Within this setting, masked image modeling is attractive because it learns local spatial representations that transfer well to dense prediction. VasoMIM is built on the observation that direct transfer of standard MIM to angiograms is suboptimal for two domain-specific reasons. First, anatomically agnostic masking is poorly matched to thin and sparse vessels, so much of the mask budget is spent on background patches. Second, pixel-space reconstruction favors low-frequency background and image texture, whereas angiogram analysis depends on vascular structure and topology rather than raw pixel fidelity alone. VasoMIM therefore injects vascular anatomical priors into both the masking policy and the reconstruction objective (Huang et al., 12 Feb 2026).

The framework is anatomy-aware in two explicit senses. The masking distribution is computed from a vascular anatomy map rather than from generic image heuristics, and the reconstruction loss is augmented with a differentiable vascular-semantic consistency term. In the original abstract formulation, the anatomy-guided masking strategy preferentially masks vessel-containing patches to focus the model on reconstructing vessel-relevant regions, while the anatomical consistency loss enforces consistency in vascular semantics between original and reconstructed images (Huang et al., 14 Aug 2025).

2. Framework and architectural formulation

At a high level, VasoMIM follows an MAE-style masked image modeling design customized for angiograms. Given an unlabeled X-ray angiogram, the method extracts a vascular prior map, divides the image into patches, applies masking so that vessel-rich patches are more likely to be masked than background patches, reconstructs the original image with a masked autoencoder, and optimizes a combination of pixel reconstruction and anatomical consistency (Huang et al., 12 Feb 2026).

The default pretraining backbone is a Vision Transformer with ViT-B/16 as encoder and a lightweight MAE-style reconstruction decoder. The input image $I$ is patchified into

$x \in \mathbb{R}^{N \times (P^2 C)},$

where $P$ is the patch size, $C$ the number of channels, and $N = HW/P^2$ the number of patches. The anatomy guidance map is patchified in parallel as

$g \in \mathbb{R}^{N \times P^2}.$

The reconstruction target is the original angiogram image in pixel space, and the paper specifies MSE as the standard MIM reconstruction loss (Huang et al., 12 Feb 2026).

The scope reported for VasoMIM differs across descriptions. The 2025 abstract presents it as a framework tailored for vessel segmentation in X-ray angiograms (Huang et al., 14 Aug 2025). The later paper presents it as a transferable pretraining system for angiogram analysis more broadly, including segmentation and detection tasks, and explicitly positions it as a domain-specific foundation model within X-ray angiography (Huang et al., 12 Feb 2026).

3. Vascular prior extraction and anatomy-guided masking

A central design decision in VasoMIM is that anatomical priors are derived automatically rather than from manual labels during pretraining. The method first uses a Hessian-based Frangi filter to obtain a binary vascular mask $B \in \{0,1\}^{1 \times H \times W}$. The extraction consists of multi-scale Hessian analysis at scales

$\sigma = \{1,2,3,4\},$

followed by percentile-based adaptive thresholding with default

$\alpha = 92,$

and region growing seeded at the global maximum of the vesselness response in order to enforce vessel connectivity and suppress isolated artifacts (Huang et al., 12 Feb 2026).

Because Frangi filtering alone can be noisy, miss faint branches, and respond to non-vascular high-gradient structures such as bone, VasoMIM trains a lightweight UNeXt-S segmentor with 0.26M parameters on Frangi pseudo-labels to predict a softer vessel probability map

$M \in [0,1]^{1 \times H \times W}.$

The final co-guidance map is

$x \in \mathbb{R}^{N \times (P^2 C)},$0

This co-guidance map is the anatomical prior used in masking (Huang et al., 12 Feb 2026).

After patchifying $x \in \mathbb{R}^{N \times (P^2 C)},$1, a patch with larger total prior mass is treated as more vessel-rich. The anatomy-guided masking probability of a patch is therefore proportional to the accumulated vessel prior within that patch. Implementation uses torch.multinomial, so anatomy-guided masked patches are sampled from a non-uniform distribution rather than uniformly at random. The paper’s comparison among Frangi-only guidance, probability-map-only guidance, and co-guidance shows that co-guidance performs best, indicating that the learned probability map complements the handcrafted Frangi prior (Huang et al., 12 Feb 2026).

VasoMIM further introduces a weak-to-strong anatomical guidance schedule rather than applying full vessel-focused masking from the beginning. If $x \in \mathbb{R}^{N \times (P^2 C)},$2 is the current epoch and $x \in \mathbb{R}^{N \times (P^2 C)},$3 the total number of pretraining epochs, the anatomy-guided fraction is

$x \in \mathbb{R}^{N \times (P^2 C)},$4

With total mask ratio $x \in \mathbb{R}^{N \times (P^2 C)},$5, the model masks $x \in \mathbb{R}^{N \times (P^2 C)},$6 patches via anatomy-guided sampling and $x \in \mathbb{R}^{N \times (P^2 C)},$7 patches randomly. The default schedule is $x \in \mathbb{R}^{N \times (P^2 C)},$8 and $x \in \mathbb{R}^{N \times (P^2 C)},$9, so training starts fully random and gradually shifts until half the masked patches are anatomy-prioritized (Huang et al., 12 Feb 2026).

This scheduling is empirically important. The default weak-to-strong setting $P$0 outperforms purely random masking, stronger end states such as $P$1 and $P$2, constant full guidance $P$3, and a strong-to-weak schedule $P$4. The interpretation reported in the paper is that excessive vessel-focused masking makes reconstruction ill-posed, and that early excessive difficulty hurts convergence. The best masking ratio is a moderate

$P$5

which differs from the common MAE choice of $P$6 for natural images; the explanation given is angiogram sparsity, since heavier masking can remove too much of the sparse vessel signal and leave mostly background context (Huang et al., 12 Feb 2026).

4. Anatomical consistency loss and training objective

The second defining component of VasoMIM is the anatomical consistency loss. Its motivation is that pure pixel reconstruction does not ensure preservation of vascular semantics or topology. Because angiogram analysis depends heavily on vessel structure, the reconstruction is constrained to preserve the same vascular semantic structure as the input (Huang et al., 12 Feb 2026).

The anatomical consistency loss is defined as

$P$7

where $P$8 is the original angiogram, $P$9 the reconstructed angiogram, $C$0 the semantic extractor, and $C$1 the similarity or discrepancy metric. The paper states that $C$2 is the lightweight UNeXt-S segmentor and that $C$3 is cross-entropy by default. Operationally, the model compares the segmentor outputs on original and reconstructed images and penalizes disagreement (Huang et al., 12 Feb 2026).

This is a semantic consistency loss rather than an explicit topology loss in a graph-theoretic sense. It is not formulated as a centerline loss, a gradient loss, or teacher-feature alignment to a natural-image model. Instead, it uses a vessel segmentor trained from pseudo-labels as a differentiable surrogate for vascular semantics. The use of the surrogate is necessary because the Frangi filter itself is non-differentiable and therefore cannot be inserted directly into end-to-end pretraining (Huang et al., 12 Feb 2026).

In addition to $C$4, VasoMIM uses a conventional pixel-level reconstruction loss $C$5, specified as MSE. The total pretraining objective is presented as the unweighted sum

$C$6

The paper reports that $C$7 improves the baseline by +0.56 DSC on ARCADE-V and +1.40 DSC on XCAV. In comparisons against alternative objectives, $C$8 gives the best downstream DSC, adds only +0.16 GFLOPs, and avoids the overhead of teacher-based distillation losses. The same study reports improved clustering of vessel-versus-background patch representations, with the best Silhouette Score, CHI, and DBI on XCAV, indicating better separation of vessel-containing and background representations (Huang et al., 12 Feb 2026).

5. Pretraining corpus, downstream tasks, and empirical performance

The later VasoMIM paper couples the method with XA-170K, described as the largest publicly available X-ray angiogram pretraining dataset to date. XA-170K contains 171,478 X-ray angiogram images collected from four public sources: CADICA (6,594 images), SYNTAX (2,943 images), XCAD (1,621 images used from 1,747 total), and CoronaryDominance (160,320 images). Pretraining is self-supervised and does not use manual labels; the priors are derived automatically from Frangi filtering and the pseudo-label-trained segmentor (Huang et al., 12 Feb 2026).

The reported pretraining recipe uses ViT-B/16, 800 epochs, a default mask ratio of 0.5, and approximately 10 hours on 8× NVIDIA H20 GPUs. For downstream segmentation, the framework uses UNet with a SimpleFPN adaptor at 224×224 resolution and AdamW with cosine annealing. For downstream detection, it uses Faster R-CNN + FPN at 512×512 with a standard ViTDet setup in Detectron2 (Huang et al., 12 Feb 2026).

The evaluated downstream tasks are as follows:

Task	Datasets	Metrics
Vessel segmentation	ARCADE-V, CAXF, XCAV	DSC, clDice
Vessel segment segmentation	ARCADE-VS	DSC
Stenosis segmentation	ARCADE-S	DSC
Stenosis detection	Stenosis	$C$9, $N = HW/P^2$0, mAP

The paper reports the following principal results. On ARCADE-V, VasoMIM reaches 80.25 DSC and 82.06 clDice. On CAXF, it reaches 89.68 DSC and 92.57 clDice. On XCAV, it reaches 86.09 DSC and 84.12 clDice. On ARCADE-S, it reaches 55.62 DSC. On ARCADE-VS, it reaches 58.87 DSC. On stenosis detection, it reports 94.91 mAP$N = HW/P^2$1, 25.72 mAP$N = HW/P^2$2, and 41.07 mAP. These results are reported as better than the listed from-scratch, general SSL/general MIM, medical SSL, and VasoMIM-v1 baselines across the corresponding tasks (Huang et al., 12 Feb 2026).

Several empirical findings are especially central to the interpretation of VasoMIM. Pretraining data scaling from 10K to 170K improves performance, although gains saturate after the early increase from 10K to 40K. Scaling the backbone from ViT-B (86M) to ViT-L (307M) and ViT-H (632M) yields only modest gains relative to cost, which the authors attribute to downstream architecture bottlenecks, XA-170K image-quality variability, or approaching a performance ceiling. The framework is also strongly label-efficient: on ARCADE-V, using only 10% labeled data, VasoMIM with a UNet backbone achieves 76.01% DSC, surpassing TransUNet trained with 100% labels by +0.50%. The paper further states that VasoMIM outperforms very large generic pretraining such as DINOv3 pretrained on 1.6B images on several angiography transfer tasks, supporting the value of domain-specific pretraining over scale alone (Huang et al., 12 Feb 2026).

The 2025 abstract presents a narrower empirical summary, stating that VasoMIM achieves state-of-the-art performance across three datasets for vessel segmentation in X-ray angiograms (Huang et al., 14 Aug 2025). The broader 2026 presentation therefore appears to extend both the data scale and the transfer setting (Huang et al., 12 Feb 2026).

6. Relation to adjacent vascular representation learning, scope, and limitations

Within vascular self-supervision, VasoMIM belongs to a broader shift from anatomy-agnostic pretraining toward structure-aware objectives. A closely related example is VAMAE, a vessel-aware masked autoencoding framework for OCT angiography that modifies both the masking distribution and reconstruction targets by emphasizing vessel-rich regions through vesselness and skeleton-based cues and reconstructing intensity, vesselness, and skeleton targets. VAMAE is reported on OCTA-500 and shows its largest gains in limited-label settings, which aligns with the same general principle as VasoMIM: anatomy-aware masking makes masked modeling focus on the clinically meaningful sparse structure rather than the background (Abolade et al., 8 Apr 2026).

Complementary work extends the same anatomy-aware logic beyond masked modeling. VasTSD is a 3D vascular tree-state space diffusion model for angiography synthesis that organizes volumetric tokens according to a dynamically constructed feature-induced tree rather than a fixed raster order. It emphasizes continuity, tree topology, cross-slice consistency, and multi-modality structural correspondence, offering a topology-aware generative perspective rather than a self-supervised masked reconstruction perspective (Wang et al., 17 Mar 2025). UniVG approaches vascular learning through a generative data-engine framework that separates structural mask generation from image appearance generation and evaluates few-shot segmentation across 11 tasks and 5 modalities, again reinforcing the idea that vascular morphology and appearance should be modeled separately rather than treated as generic image texture (Ge et al., 12 Apr 2026).

Several misconceptions can therefore be excluded. VasoMIM is not simply standard MAE applied to angiograms; its novelty lies in anatomy-guided masking, co-guidance via Frangi and a learned probability map, weak-to-strong scheduling, and anatomical consistency via a differentiable vessel semantic extractor (Huang et al., 12 Feb 2026). It is also not an explicit vessel-graph or centerline model. The anatomical consistency term matches segmentation semantics between original and reconstructed images; it does not implement an explicit graph-topology loss, branch decoder, or centerline supervision (Huang et al., 12 Feb 2026). Nor is it a universal vascular representation framework across all modalities in its reported form; the later paper is explicitly designed for X-ray angiography, whereas related methods such as VAMAE target OCTA and UniVG spans multiple 2D vascular modalities (Abolade et al., 8 Apr 2026).

Its main limitations are correspondingly domain-specific. The framework depends on Frangi-derived pseudo-anatomy and a segmentor trained on those pseudo-labels, even though downstream performance is reported to remain stable when the Frangi threshold parameter $N = HW/P^2$3 is varied. The method is tailored to X-ray angiography rather than modality-agnostic vascular learning. XA-170K contains variable image quality and artifacts, which the authors identify as a potential reason for limited returns from model scaling. Larger backbones improve performance only modestly relative to compute cost. In the 2025 checklist-style material, statistical significance testing is also reported as not having been provided (Huang et al., 14 Aug 2025, Huang et al., 12 Feb 2026).

Taken together, VasoMIM exemplifies a vascular-domain adaptation of masked image modeling in which anatomy is embedded into both corruption and reconstruction. Its defining claim is that, for angiograms, the effectiveness of self-supervised pretraining depends not only on reconstruction capacity but on whether the pretext task is explicitly organized around vessel-rich patches and vascular semantic consistency (Huang et al., 14 Aug 2025).