Scale-cascaded Prior Fusion (SPF)

• Scale-cascaded Prior Fusion (SPF) is a scale-aware mechanism that fuses priors from three support scales to address size mismatches in one-shot polyp segmentation.
• It generates multi-scale priors, evaluates each using reverse transfer and confidence IoU, and adaptively weights them to reduce noise and bias.
• SPF significantly improves segmentation performance by supplying robust, scale-robust priors to the prompt evolution stage, yielding higher IoU and Dice metrics.

Scale-cascaded Prior Fusion (SPF) is the prior-fusion component of OP-SAM, a one-shot polyp segmentation framework based on SAM. In that formulation, SPF is introduced to adapt to polyp size variations and to filter out noisy prior transfers generated from a single annotated support image. Operationally, it generates priors from three scale versions of the support lesion, evaluates the quality of each prior by reverse transfer, and fuses them adaptively before the result is passed to Euclidean Prompt Evolution (EPE) for iterative prompt refinement with SAM2 (Mao et al., 22 Jul 2025).

1. Definition and problem setting

Within OP-SAM, SPF is motivated by the observation that a single support image is not enough to robustly guide segmentation when polyp size varies dramatically across query images. The paper identifies several failure modes of single-scale prior transfer: a support polyp may be much larger than the query polyp, much smaller, or differ in shape and extent due to partial visibility and morphology. Under those conditions, a transferred prior can become too sparse when the query polyp is larger than the support lesion, too broad or noisy when the query polyp is smaller, or otherwise biased toward the support lesion’s scale; the stated consequence is false positives and false negatives (Mao et al., 22 Jul 2025).

SPF addresses that scale-mismatch problem by constructing multiple priors at different lesion scales and then selectively fusing them based on a quality estimate computed by reverse transfer. The paper’s division of labor is explicit: CPG improves the semantic quality of a prior for one support scale, SPF makes that prior robust to scale mismatch, and EPE uses the fused prior to iteratively drive SAM prompts (Mao et al., 22 Jul 2025).

This suggests that SPF is not a generic multi-scale ensemble, but a scale-aware prior-selection mechanism designed for one-shot transfer under substantial support-query lesion-size variability.

2. Position in the OP-SAM pipeline

OP-SAM is organized into three stages. The first stage is Correlation-based Prior Generation (CPG), which transfers the support mask to the query image using DINOv2 feature correlation and then refines the coarse transfer with self-correlation. The second stage is SPF, which generates priors from three scale versions of the support lesion and fuses them adaptively. The third stage is EPE, which uses the final fused prior as a prompt source and iteratively improves segmentation through prompt-evaluate-refine cycles with SAM2 (Mao et al., 22 Jul 2025).

SPF therefore sits between semantic prior generation and prompt-based segmentation. The paper characterizes it as the bridge between semantic prior generation and interactive prompting: it ensures that the prior given to EPE is both complete and not overly noisy. In this architecture, SPF is not a standalone classifier or decoder; it is an internal prior-processing stage that conditions later prompt evolution (Mao et al., 22 Jul 2025).

The input to SPF is the set of scale-specific priors produced by CPG. CPG operates on a support image $\mathbf{x}^s$ with mask $\mathbf{m}^s$ and a query image $\mathbf{x}^q$, extracts frozen DINOv2 patch features $$\mathbf{f}^{s}, \mathbf{f}^{q} \in \mathbb{R}^{hw \times D}$$, computes a cross-correlation matrix $\mathbf{S}_{corr} \in \mathbb{R}^{hw \times hw}$, and produces a prior for a given support scale according to

$$\mathbf{p}_{ori} = [\mathbf{S}_{self}]^{\rho} \cdot \mathbf{S}_{corr} \cdot \mathbf{m}^{s}_{r}.$$

SPF then applies the same CPG mechanism to three support-scale variants rather than only the original support instance (Mao et al., 22 Jul 2025).

3. Multi-scale support construction and scale-cascaded priors

The phrase “scale-cascaded” is operationalized through explicit support-lesion resizing. The paper creates three support variants: $(\mathbf{x}^{s}_{ori}, \mathbf{m}^{s}_{ori})$ for the original size, $(\mathbf{x}^{s}_{xl}, \mathbf{m}^{s}_{xl})$ for an extra-large lesion, and $(\mathbf{x}^{s}_{xs}, \mathbf{m}^{s}_{xs})$ for an extra-small lesion. These are produced by lesion scaling: the polyp is cut out using the mask, resized, pasted back at the same location, and, for the zoomed-out case, the blank region is inpainted (Mao et al., 22 Jul 2025).

Each support variant is passed through CPG, producing three priors:

$$\mathbf{p}_{ori}, \mathbf{p}_{xl}, \mathbf{p}_{xs}.$$

The paper assigns distinct roles to these priors: $\mathbf{m}^s$0 tends to capture broader lesion extent, $\mathbf{m}^s$1 emphasizes salient or compact lesion parts, and $\mathbf{m}^s$2 provides the baseline reference (Mao et al., 22 Jul 2025).

This formulation is narrower than generic multi-scale processing. The paper states that “scale-cascaded” is not just multi-scale in the generic sense; it is a support-lesion scaling strategy that generates a cascade of priors from one annotated example to cover different possible query lesion sizes. A plausible implication is that the method treats support-scale perturbation as a structured prior-generation process rather than as ordinary test-time augmentation (Mao et al., 22 Jul 2025).

4. Adaptive fusion by reverse transfer and confidence IoU

SPF first binarizes each prior $\mathbf{m}^s$3, where $\mathbf{m}^s$4, with threshold $\mathbf{m}^s$5. This thresholded prior selects query patches believed to belong to the lesion. The key novelty is the subsequent reverse-transfer step: for each thresholded query prior, the method masks the query features and computes cosine similarity between the selected query features and support features, averaged over the first dimension, to produce a reverse-transferred map $\mathbf{m}^s$6 on the support image (Mao et al., 22 Jul 2025).

The reverse-transferred map is interpreted as a quality check. If the original query prior is noisy or incorrect, reverse transfer highlights irrelevant support regions outside the true polyp area. The paper then thresholds the reverse prior again and computes a confidence IoU ($\mathbf{m}^s$7) between the reverse prior and the original support mask $\mathbf{m}^s$8. An important stated detail is that, unlike standard IoU, the intersection part is weighted by the probability from $\mathbf{m}^s$9, so the measure evaluates both overlap and overlap with confidence (Mao et al., 22 Jul 2025).

The scale weights are normalized from those confidence IoU values, and the final fused prior is

$\mathbf{x}^q$0

The paper explicitly contrasts this with naive averaging: if $\mathbf{x}^q$1 is low, “the prior generated by the size is biased, and less credit $\mathbf{x}^q$2 is given.” For that reason, SPF is described as an automatic scale selector and denoiser, rather than a simple arithmetic fusion rule (Mao et al., 22 Jul 2025).

The underlying logic is fully stated in the paper: a scale-specific prior may be partially correct but still noisy; if the prior is good, then transferring it back to the support image should reconstruct the support polyp region well; if the prior is noisy, reverse transfer will activate irrelevant support regions, lowering confidence IoU; therefore, low-quality priors receive lower weights in the fusion. This suggests that SPF uses reverse consistency as an internal prior-validation criterion (Mao et al., 22 Jul 2025).

5. Interaction with EPE and empirical evidence

After SPF computes $\mathbf{x}^q$3, the fused prior is passed to EPE. EPE does not dump all points into SAM at once. Instead, it threshold-splits the prior into a tight prior using $\mathbf{x}^q$4 and a loose prior using $\mathbf{x}^q$5, runs Euclidean Distance Transform on the binary prior to locate the point farthest from background, uses that point as a prompt, obtains a mask and IoU prediction from SAM2, and then iteratively refines prompting according to coverage of the mask over the tight prior and predicted IoU. If coverage is insufficient, it prompts again in the unsegmented parts of the tight prior; if coverage is adequate but confidence remains low, it expands to the loose prior; if the mask spills too far outside the loose prior, it adds negative prompts to suppress noise (Mao et al., 22 Jul 2025).

In that sequence, SPF supplies EPE with a prior that already has scale robustness, denoising, and better localization. The paper states that EPE’s iterative prompting works best when the prior is reliable, and that a noisy prior would mislead prompt evolution. This positions SPF as a quality-control stage for prompt generation rather than merely a preprocessing heuristic (Mao et al., 22 Jul 2025).

The empirical evidence reported in the paper is consistent with that interpretation. On Kvasir, the ablation table gives CPG only at 67.12 IoU / 76.43 Dice, CPG + SPF at 69.05 IoU / 78.07 Dice, and CPG + SPF + EPE at 76.93 IoU / 84.53 Dice. The same paper reports that on Kvasir, OP-SAM achieves 76.93% IoU, surpassing the state-of-the-art by 11.44%. In a support-configuration study, 1 image + 2 augmentations + Avg gives 48.67 IoU / 61.09 Dice, whereas 1 image + 2 augmentations + SPF gives 61.47 IoU / 73.71 Dice, which the paper uses to argue that simply having multiple priors is not enough and that the fusion strategy is crucial. On Kvasir-H, described as containing extreme-size polyps, OP-SAM achieves 57.31 IoU / 67.56 Dice, and the paper explicitly attributes this strong performance to scale-cascaded prior fusion (Mao et al., 22 Jul 2025).

6. Terminological scope and acronym collisions

The exact phrase “Scale-cascaded Prior Fusion” appears in OP-SAM and refers to the adaptive multi-scale prior ensemble described above (Mao et al., 22 Jul 2025). Several other papers use the acronym “SPF,” but those uses are unrelated.

Paper	Expansion of “SPF”	Relation to Scale-cascaded Prior Fusion
“Cornell SPF: Cornell Semantic Parsing Framework” (Artzi, 2013)	Cornell Semantic Parsing Framework	Unrelated acronym collision
“BrainTAP: Brain Disorder Prediction with Adaptive Distill and Selective Prior Integration” (Lei et al., 10 Feb 2026)	Selective Prior Fusion	Different prior-integration mechanism
“Scale-Aware Crowd Counting Using a Joint Likelihood Density Map and Synthetic Fusion Pyramid Network” (Hsieh et al., 2022)	Synthetic Fusion Pyramid / SPF-Net	Not a prior-fusion method
“SSPFusion: A Semantic Structure-Preserving Approach for Infrared and Visible Image Fusion” (Yang et al., 2023)	Structure-Preserving Fusion	Different fusion module

The Cornell paper states that SPF means Cornell Semantic Parsing Framework, a Java framework for mapping natural language sentences to CCG-based lambda-calculus logical forms, and explicitly notes that it has nothing to do with “Scale-cascaded Prior Fusion” (Artzi, 2013). BrainTAP introduces Selective Prior Fusion as a learned, subject-adaptive mechanism for injecting expert anatomical priors into Transformer attention; it uses global masks, personalized low-rank masks, soft gating, and attention-bias injection, but it is a different method family and a different problem setting (Lei et al., 10 Feb 2026). The crowd-counting paper uses SFP-Net or SPF-Net to denote Synthetic Fusion Pyramid Network, with “fusion” referring to synthetic feature-scale fusion and intra-block fusion rather than prior-based cascade fusion (Hsieh et al., 2022). SSPFusion defines SPF as Structure-Preserving Fusion, a multi-scale infrared-visible image fusion module guided by structural maps, again unrelated to OP-SAM’s prior-transfer mechanism (Yang et al., 2023).

A common misconception is therefore to treat “SPF” as a stable method name across domains. The literature represented here shows the opposite: the acronym is overloaded, and “Scale-cascaded Prior Fusion” is specific to the OP-SAM formulation for one-shot polyp segmentation (Mao et al., 22 Jul 2025).