SimCroP: Similarity-Driven Cross-Granularity Pre-training

• The paper demonstrates that SimCroP integrates multi-modal masked modeling with sentence-to-patch similarity alignment to improve radiograph interpretation.
• The framework fuses instance-level global features with word-patch signals, enabling fine-grained alignment across spatial and semantic levels.
• Empirical results show enhanced AUC and Dice scores compared to baseline methods in both classification and segmentation of chest CT data.

Similarity-Driven Cross-Granularity Pre-training (SimCroP) denotes a pre-training paradigm in which representation learning is organized around two coupled ideas: similarity-driven alignment and cross-granularity supervision. In its explicit named form, SimCroP is a medical vision-language pre-training framework for 3D chest CT volumes paired with radiology reports that combines multi-modal masked modeling, sentence-to-subregion similarity alignment, and fusion across instance-level and word-patch-level signals to improve downstream classification and segmentation (Wang et al., 10 Sep 2025). More broadly, the term also describes a recurrent design pattern across recent pre-training research: learn from relative similarity structure rather than only pointwise targets, and impose consistency across multiple semantic or spatial levels such as token versus sentence, scene versus object versus pixel, or instance versus class (Kim et al., 2023, Chi et al., 2020, Xiang et al., 10 Mar 2026).

1. Definition and conceptual scope

In the explicit formulation introduced for chest CT, SimCroP is a medical vision-language pre-training framework designed for paired 3D radiographs and reports. Its stated objective is to improve radiograph interpretation by combining similarity-driven alignment with cross-granularity fusion, with particular emphasis on the spatial sparsity of lesions and the implicit mapping between report sentences and radiographic subregions (Wang et al., 10 Sep 2025). The framework first uses multi-modal masked modeling to optimize the encoder for low-level semantics, then applies similarity-driven alignment to adaptively select the correct patches for each sentence, and finally fuses multimodal information across instance level and word-patch level.

The phrase “similarity-driven” refers to training signals defined by relative affinity or matching rather than only direct feature regression or global label prediction. In SimCroP, this is realized by ranking visual patches for each report sentence and aligning the resulting top-$K$ visual aggregate with the sentence embedding through a symmetric InfoNCE-style objective (Wang et al., 10 Sep 2025). The phrase “cross-granularity” refers to coupling supervision across levels of description. In SimCroP, these levels are instance-level global CT representations and local word-patch correspondences; in related work, analogous pairings include token versus sentence in cross-lingual language modeling, class-prompt similarity versus instance-level feature alignment in multimodal transfer, and semantics versus instances versus pixels in hierarchical masked autoencoding (Chi et al., 2020, Kim et al., 2023, Xiang et al., 10 Mar 2026).

A broader research interpretation is suggested by multiple earlier methods. BeamCLIP transfers CLIP representations into smaller vision-only backbones by matching similarity distributions over text anchors, which the paper explicitly frames as a building block for a more general similarity-driven cross-granularity transfer principle (Kim et al., 2023). InfoXLM formulates cross-lingual pre-training as maximizing mutual information between multilingual, multi-granularity views and adds a similarity-based contrastive objective over bilingual sentence pairs (Chi et al., 2020). C2FMAE, although generative rather than contrastive, enforces scene-level, object-level, and pixel-level dependencies through a cascaded decoder and progressive masking curriculum (Xiang et al., 10 Mar 2026). These works do not all use the term SimCroP, but they instantiate closely related mechanisms.

2. Core architecture in chest CT vision-language pre-training

SimCroP is built on a multi-modal masked autoencoder architecture for 3D chest CT and report pairs (Wang et al., 10 Sep 2025). The visual backbone is a 3D Vision Transformer $\text{ViT-B}$, used as both encoder and decoder. The input CT volume

$I \in \mathbb{R}^{H \times W \times D}$

is divided into 3D patches with image size $224 \times 224 \times 112$ and patch size $16 \times 16 \times 8$. Patches are linearly projected to tokens, combined with 3D positional embeddings, and passed to an 8-layer 3D ViT-B encoder initialized from 2D MAE pre-training on ImageNet-1K after inflation to 3D. A 4-layer 3D ViT-B decoder reconstructs the masked volume.

The textual side uses a BERT-based stack. Reports are tokenized with a BERT tokenizer, and a CXR-BERT encoder produces report features for both masked language modeling and sentence-level alignment. For report reconstruction, SimCroP uses a separate 6-layer BERT decoder (Wang et al., 10 Sep 2025). The report is treated as $T=[T_F,T_I]$, where $T_F$ is the “Findings” section and $T_I$ the “Impression”. The framework focuses on $T_F$ for fine-grained alignment because it is described as encapsulating comprehensive visual observations.

The architecture is explicitly multimodal but not restricted to global image-report matching. Its design assumption is that sparse lesions in chest CT are poorly served by a single global contrastive embedding, because the discriminative signal may occupy only a small subset of the volume (Wang et al., 10 Sep 2025). This motivates separating sentence-level textual units and patch-level visual units, then linking them adaptively. A plausible implication is that SimCroP is less a conventional CLIP-style dual encoder than a structured pre-training system in which masked modeling, retrieval-style alignment, and reconstruction share the same visual backbone.

3. Similarity-driven alignment mechanism

The central alignment stage begins by splitting the “Findings” section into $N_{sent}$ linguistically independent clauses or sentences: $\text{ViT-B}$0 Each sentence is encoded separately: $\text{ViT-B}$1 Meanwhile, for unmasked CT patches $\text{ViT-B}$2, patch features are computed as

$\text{ViT-B}$3

The sentence-patch similarity is a dot product: $\text{ViT-B}$4 For each sentence, SimCroP selects the top-$\text{ViT-B}$5 patches with the highest similarity and aggregates their features with global average pooling: $\text{ViT-B}$6 This produces a sentence-specific aligned visual representation (Wang et al., 10 Sep 2025).

The alignment objective is symmetric. With

$\text{ViT-B}$7

and temperature $\text{ViT-B}$8, the loss is

$\text{ViT-B}$9

Other sentences in the same case serve as negatives. The paper reports that the best top-$I \in \mathbb{R}^{H \times W \times D}$0 configuration covers about 64 patches, or roughly $I \in \mathbb{R}^{H \times W \times D}$1 of the CT area, which is presented as matching the typical spatial extent described by a sentence (Wang et al., 10 Sep 2025).

This mechanism differs from global image-report contrastive learning in two specific ways. First, the alignment granularity is sentence to subregion rather than report to volume. Second, the visual region is selected adaptively rather than defined by a fixed anatomical partition (Wang et al., 10 Sep 2025). Related methods expose the same general principle under different formulations. BeamCLIP aligns a student image encoder to a teacher by matching distributions over similarities to semantic text anchors rather than only regressing features, arguing that multiple anchors better preserve relational geometry (Kim et al., 2023). Unsupervised retrieval-based vision-language pre-training also begins with similarity as a corpus-construction device, retrieving candidate texts for each image and then learning region-to-tag, region-to-phrase, and image-to-sentence alignment (Zhou et al., 2022). SimCroP differs in using similarity not to synthesize weak pairs from non-parallel corpora, but to localize report sentences within a paired 3D study.

4. Cross-granularity fusion and masked modeling

SimCroP jointly optimizes masked image modeling, masked language modeling, and the alignment loss (Wang et al., 10 Sep 2025). On the visual side, $I \in \mathbb{R}^{H \times W \times D}$2 of 3D patches are randomly masked. Given encoded unmasked features $I \in \mathbb{R}^{H \times W \times D}$3 and mask tokens $I \in \mathbb{R}^{H \times W \times D}$4, the decoder reconstructs the full volume: $I \in \mathbb{R}^{H \times W \times D}$5 with

$I \in \mathbb{R}^{H \times W \times D}$6

On the textual side, a fraction $I \in \mathbb{R}^{H \times W \times D}$7 of report tokens is masked, with best performance reported at a $I \in \mathbb{R}^{H \times W \times D}$8 masking ratio (Wang et al., 10 Sep 2025).

The cross-granularity module combines two kinds of visual evidence. The first is an instance-level global feature

$I \in \mathbb{R}^{H \times W \times D}$9

obtained by pooling over all visual patches. The second is a word-patch cross-modal feature

$224 \times 224 \times 112$0

where text tokens query visual patches. These are fused by element-wise addition after broadcasting the global feature: $224 \times 224 \times 112$1 and the text decoder reconstructs masked tokens from the fused representation: $224 \times 224 \times 112$2 The masked language modeling loss is

$224 \times 224 \times 112$3

The full objective is

$224 \times 224 \times 112$4

with $224 \times 224 \times 112$5 in experiments (Wang et al., 10 Sep 2025).

This division between global and local signals is the cross-granularity core of SimCroP. The instance-level term captures macro anatomical context, while the cross-attentive word-patch term captures local pathology and anatomical location (Wang et al., 10 Sep 2025). Comparable cross-granularity structuring appears in C2FMAE, where a shared encoder must support semantic masks, instance masks, and RGB reconstruction, and a cascaded decoder enforces the order scene $224 \times 224 \times 112$6 instances $224 \times 224 \times 112$7 pixels (Xiang et al., 10 Mar 2026). In language modeling, InfoXLM jointly optimizes token-level and sentence-level mutual-information objectives, with token prediction on the top layer and sequence-level similarity on an intermediate “universal” layer (Chi et al., 2020). These parallels suggest that SimCroP belongs to a wider class of pre-training methods in which different granularities are not merely auxiliary tasks but coordinated constraints on a shared representation.

5. Empirical behavior and comparison with related methods

SimCroP is pre-trained on CT-RATE, described as a large-scale paired chest CT-report dataset, and evaluated on image classification and segmentation across five public datasets (Wang et al., 10 Sep 2025). For linear probing with a frozen 3D ViT-B encoder, it is compared against 3D medical self-supervised learning and medical vision-language pre-training baselines including MAE, M3AE, CT-CLIP, MRM, M3D, and fVLM.

Reported classification results show consistent gains. On CT-RATE, SimCroP reaches an AUC of $224 \times 224 \times 112$8 with $224 \times 224 \times 112$9 labeled data, compared with $16 \times 16 \times 8$0 for fVLM, and $16 \times 16 \times 8$1 with $16 \times 16 \times 8$2 labeled data versus $16 \times 16 \times 8$3 for fVLM (Wang et al., 10 Sep 2025). On CC-CCII it reaches up to $16 \times 16 \times 8$4 AUC at $16 \times 16 \times 8$5 labeled data; on RadChestCT it reaches $16 \times 16 \times 8$6; on LUNA16 nodule classification it reaches $16 \times 16 \times 8$7. The paper highlights that masked-modeling-based methods, including M3AE, MRM, and SimCroP, outperform contrastive methods such as M3D and fVLM especially on fine-grained nodule classification.

On segmentation, SimCroP obtains Dice $16 \times 16 \times 8$8 on LUNA16 lung segmentation with $16 \times 16 \times 8$9 labeled data, compared with $T=[T_F,T_I]$0 for the best listed baseline, MRM, and $T=[T_F,T_I]$1 on BTCV abdominal segmentation with $T=[T_F,T_I]$2 labeled data, compared with $T=[T_F,T_I]$3 for M3AE (Wang et al., 10 Sep 2025). The BTCV result is presented as evidence of cross-domain transfer from chest to abdomen.

Ablation results isolate the contribution of its constituent mechanisms. On RadChestCT AUC with $T=[T_F,T_I]$4 labeled data and LUNA16 Dice with $T=[T_F,T_I]$5 labeled data, the full model with similarity-driven alignment plus both instance-level and word-patch features reaches $T=[T_F,T_I]$6 AUC and $T=[T_F,T_I]$7 Dice. Without similarity-driven alignment, using both feature types yields $T=[T_F,T_I]$8 AUC and $T=[T_F,T_I]$9 Dice; with alignment plus only instance-level features, the result is $T_F$0 and $T_F$1; with alignment plus only word-patch features, $T_F$2 and $T_F$3 (Wang et al., 10 Sep 2025). The paper summarizes this as more than $T_F$4 AUC and about $T_F$5 Dice improvement from the alignment stage.

These outcomes place SimCroP between two families of prior methods. Compared with global contrastive Med-VLP such as CT-CLIP and M3D, it replaces single global image-report alignment with sentence-subregion matching (Wang et al., 10 Sep 2025). Compared with multimodal masked autoencoding such as M3AE and MRM, it adds an explicit fine-grained alignment loss and a dual-granularity fusion pathway. More generally, its empirical pattern resembles related findings outside medical CT: BeamCLIP shows that similarity matching to semantic anchors can let a small ResNet-18 reach $T_F$6 ImageNet-1K top-1 linear probe accuracy, above SimCLR at $T_F$7 and SwAV at $T_F$8, while approaching supervised $T_F$9 (Kim et al., 2023). The common thread is that structured similarity information can substitute for or augment direct labels.

6. Broader lineage, limitations, and future directions

SimCroP was introduced explicitly for chest CT in 2025, but several earlier papers articulate closely related principles. InfoXLM treats cross-lingual pre-training as maximizing mutual information between multilingual, multi-granularity texts and adds a contrastive sequence-level objective over bilingual sentence pairs (Chi et al., 2020). Geometric Visual Similarity Learning in 3D medical imaging uses topology-aware geometric matching to learn consistent global and local anatomy-centered representations across scans, which the paper explicitly interprets as a similarity-driven, multi-granular pre-training paradigm (He et al., 2023). Retrieval-based unsupervised V+L pre-training constructs weakly aligned image-text pairs from non-parallel corpora and then learns region-to-tag, region-to-phrase, and image-to-sentence alignment (Zhou et al., 2022). C2FMAE organizes pre-training around semantic masks, instance masks, and pixels with a coarse-to-fine curriculum (Xiang et al., 10 Mar 2026). HERMES generalizes the cross-granularity idea from representation learning to data mixture design by assigning each document a reusable coarse-to-fine residual vector quantization code whose prefix length controls granularity up to approximately $T_I$0 observed cells (Qiao et al., 2 Jul 2026).

These works indicate that SimCroP is both a specific framework and a more general research program. In the specific CT formulation, the main limitations are stated directly. SimCroP depends on large-scale paired CT-report data such as CT-RATE; it lacks instance-level cross-modal alignment for zero-shot use; it incurs substantial computational cost from 3D ViT processing, top-$T_I$1 sentence-patch similarity computation, and cross-attention; and it remains domain-specific, with pre-training centered on chest CT despite some cross-domain transfer to abdomen (Wang et al., 10 Sep 2025). The authors propose adding instance-level cross-modal alignment in future work to improve zero-shot and retrieval performance.

A broader methodological limitation, suggested by adjacent work, is that cross-granularity design introduces new choices about the level at which similarity should be computed and how fine a partition remains useful. HERMES reports that a quality-aware sampling rule improves performance at one intermediate hierarchy prefix but loses its measurable advantage at the next finer level as candidate pools contract by approximately $T_I$2, a phenomenon the paper attributes to candidate competition (Qiao et al., 2 Jul 2026). This suggests that cross-granularity learning is not automatically strengthened by moving to finer units; efficacy depends on whether the chosen scale preserves both semantic coherence and sufficient support.

In that sense, SimCroP can be understood as a concrete chest-CT realization of a larger pre-training doctrine: use similarity to identify meaningful correspondences, and use multiple granularities to constrain what a shared representation must preserve. The named SimCroP framework realizes this with sentence-subregion contrast, masked CT and report reconstruction, and fusion between global and local visual-textual pathways (Wang et al., 10 Sep 2025). The surrounding literature shows that the same doctrine is applicable across multimodal transfer, cross-lingual language modeling, hierarchical masked autoencoding, geometric medical pre-training, unsupervised V+L learning, and corpus partitioning (Kim et al., 2023, Chi et al., 2020, Xiang et al., 10 Mar 2026, He et al., 2023, Zhou et al., 2022, Qiao et al., 2 Jul 2026).