MCP-Encoder: Multi-Scale Prompting in MSM-Seg

• The MCP-Encoder transforms memory-enhanced features and multi-scale backbone information into a decoder-ready tumor guidance representation.
• It replaces labor-intensive, category-specific prompts with a coarse, category-agnostic whole-tumor bounding box to improve clinical workflow.
• Ablation studies show incremental Dice and HD95 gains, validating the effectiveness of multi-scale prompt encoding in multi-modal brain tumor segmentation.

Searching arXiv for the cited papers and closely related prompt-learning work to ground the article. The Multi-Scale Category-Agnostic Prompt Encoder (MCP-Encoder) is a prompt-conditioning module introduced within the MSM-Seg framework for multi-modal brain tumor segmentation. Its function is to convert either a coarse whole-tumor prompt or automatically inferred tumor guidance into a spatial guidance representation that can be consumed by the decoder. In the MSM-Seg pipeline, MCP-Encoder is narrower in scope than the full framework but central to the claim that prompt-based segmentation for multi-modal brain tumor MRI can be made more clinically practical by replacing subregion-specific prompting with whole-tumor, category-agnostic guidance (Luo et al., 12 Oct 2025).

1. Position within the MSM-Seg pipeline

MCP-Encoder is situated between the modality-and-slice memory attention (MSMA) module and the modality-adaptive fusion decoder (MF-Decoder). MSM-Seg processes MRI data sequentially by slice and modality. For each input slice-modality pair $(t,m)$, the shared image encoder first extracts image features; MSMA then injects slice memory and modality memory to produce a memory-enhanced embedding $\mathbf{Z}_{t,m}$; MCP-Encoder transforms $\mathbf{Z}_{t,m}$, together with multi-scale backbone features and optional prompt input, into a tumor region guidance representation $\mathbf{P}_{t,m}$; MF-Decoder then uses $\mathbf{Z}_{t,m}$ and $\mathbf{P}_{t,m}$ to predict a modality-specific mask, after which modality predictions are fused into the final slice mask (Luo et al., 12 Oct 2025).

This placement gives MCP-Encoder a bridging role. It does not generate the final segmentation mask directly, nor does it operate on raw images in isolation. Rather, it takes features that have already been enriched by cross-modal and inter-slice memory and converts coarse prompt information into decoder-ready tumor guidance. The paper formalizes this placement in Algorithm 1 through

$$\mathbf{P}_{t,m} = \text{MCP-Encoder}(\mathbf{Z}_{t,m},\mathcal{Q}_{\text{box}},\{\mathbf{F}_i\}_{i=1}^{l-1}),$$

followed by

$$\mathbf{H}_{t,m} = \mathbf{Z}_{t,m} \oplus \mathbf{P}_{t,m}.$$

Accordingly, MCP-Encoder is best understood as the prompt-to-guidance transformation stage in MSM-Seg rather than as a standalone segmentation network.

The module is also operationally 2D slice-wise with memory across slices and modalities, not a native 3D prompt encoder. The framework processes inputs as $\mathbf{X}_{t,m}$, feature maps are written as $\mathbb{R}^{C\times H\times W}$, images are resized to $\mathbf{Z}_{t,m}$0, and inter-slice context is supplied through slice memory rather than 3D convolutions. This suggests that MCP-Encoder itself is a 2D multi-scale encoder operating on per-slice features whose contextual enrichment is delegated upstream to MSMA.

2. Motivation: from category-specific prompts to whole-tumor guidance

The introduction of MCP-Encoder is motivated by limitations in prior prompt-based medical segmentation methods, especially SAM- and SAM2-derived variants that rely on category-specific prompts for individual tumor subregions. The paper explicitly identifies enhancing tumor, tumor core, and peritumoral edema as examples of subregions that would require separate prompts under earlier formulations (Luo et al., 12 Oct 2025).

The paper argues that this requirement is problematic in brain tumor MRI because these subregions have overlapping spatial extents, ambiguous boundaries, and demand manual delineation with anatomical expertise. Its stated criticism is that such prompting “requires labor-intensive manual annotation and prior anatomical knowledge,” and “hinder[s] clinical workflow efficiency.” MCP-Encoder is introduced to remove dependence on subregion-specific prompting and replace it with whole-tumor region guidance.

In this formulation, category-agnostic prompting means that the prompt need only indicate the tumor region as a whole rather than a specific subcategory. The concrete prompt form emphasized by the paper is a bounding box covering the entire tumor region. A coarse whole-tumor box is easier to provide than multiple prompts for ET, TC, or WT-like substructures, and the same design also supports an automatic mode in which no manual prompt is supplied. The paper does not specify an external detector or box generator in automatic mode; instead, it states that guidance is generated “autonomously” from image features and MSMA-enhanced representations.

The module is also explicitly multi-scale because brain tumors vary greatly in size, “from small satellite lesions to large infiltrative masses.” This motivates encoding prompts across multiple feature scales rather than at a single resolution. The paper’s rationale is that small lesions require fine-resolution local guidance, large diffuse tumors require broader contextual guidance, and prompt-image correspondence must be preserved across feature levels. Thus MCP-Encoder receives a set of backbone features $\mathbf{Z}_{t,m}$1, aligns prompts to each scale via resizing, and fuses them layer by layer.

3. Architectural design and mathematical formulation

MCP-Encoder takes three forms of input: the memory-enhanced embedding $\mathbf{Z}_{t,m}$2 produced by MSMA, multi-scale backbone features $\mathbf{Z}_{t,m}$3, and optional category-agnostic prompt input $\mathbf{Z}_{t,m}$4, represented after scale translation as $\mathbf{Z}_{t,m}$5 (Luo et al., 12 Oct 2025). Its output is a tumor guidance representation $\mathbf{Z}_{t,m}$6, consistently described as tumor region guidance rather than as a segmentation result.

The confirmed prompt representation is a coarse spatial box. The paper states that the method accepts “category-agnostic prompts (e.g., bounding boxes),” and bounding box prompts are used in prompt-based evaluation. Internally, the paper specifies that a multi-scale translator uses bilinear interpolation to resize the user prompt input to each feature scale, producing $\mathbf{Z}_{t,m}$7, and that these prompt tensors are concatenated with image features during layer-wise fusion. It does not describe prompt positional embeddings, prompt tokenization into sparse or dense embeddings, transformer prompt encoders, or MLP encoding of box coordinates. A plausible implication is that $\mathbf{Z}_{t,m}$8 functions as a spatial prompt map derived from the box, but the exact rasterization scheme is not explicitly defined.

The core fusion mechanism is layer-by-layer and concatenation-based. The paper gives

$\mathbf{Z}_{t,m}$9

This makes prompt integration multi-scale in a strict architectural sense: prompt guidance is injected at every scale, not appended only once at the end. The indexing around $\mathbf{Z}_{t,m}$0, $\mathbf{Z}_{t,m}$1, and $\mathbf{Z}_{t,m}$2 is noted in the source as slightly inconsistent, but the intended design is that all backbone scales plus the top-level memory-enhanced representation participate in a progressive fusion stack.

The final guidance is generated by

$\mathbf{Z}_{t,m}$3

where $\mathbf{Z}_{t,m}$4 consists of two $\mathbf{Z}_{t,m}$5 convolutions followed by one $\mathbf{Z}_{t,m}$6 convolution for downsampling, and $\mathbf{Z}_{t,m}$7 is sigmoid (Luo et al., 12 Oct 2025). The operator $\mathbf{Z}_{t,m}$8 appears in the formula but is not redefined within the MCP-Encoder subsection. The most cautious interpretation is that $\mathbf{Z}_{t,m}$9 is a feature transform before sigmoid, while the exact reuse of earlier notation remains uncertain. The use of sigmoid suggests that $\mathbf{P}_{t,m}$0 behaves as a soft region prior.

The internal operator set is comparatively restrained. Confirmed components are the multi-scale translator, bilinear interpolation, concatenation-based fusion, the convolutional head $\mathbf{P}_{t,m}$1, and sigmoid activation. By contrast, the paper does not confirm any transformer block, attention block, MLP, positional encoding, or feature pyramid network inside MCP-Encoder. Relative to MSMA and MF-Decoder, MCP-Encoder is therefore described as a simpler multi-scale concatenative fusion plus convolutional prediction head.

4. Decoder interaction, supervision, and empirical evidence

MCP-Encoder conditions decoding through a direct additive mechanism. For each modality $\mathbf{P}_{t,m}$2 at slice $\mathbf{P}_{t,m}$3, the prompt guidance is injected by

$\mathbf{P}_{t,m}$4

after which MF-Decoder predicts

$\mathbf{P}_{t,m}$5

This means that MCP-Encoder output is consumed through element-wise addition to the memory-enhanced image embedding rather than via decoder-side cross-attention, concatenative decoder input, or an explicit gating module (Luo et al., 12 Oct 2025). In the paper’s own framing, the prompt-conditioned tensor acts as a dense spatial prior over $\mathbf{P}_{t,m}$6.

The module is not merely auxiliary conditioning; it is explicitly supervised. The overall loss is

$\mathbf{P}_{t,m}$7

with $\mathbf{P}_{t,m}$8 using binary cross-entropy loss, and $\mathbf{P}_{t,m}$9 and $\mathbf{Z}_{t,m}$0 using dice plus focal loss. The training description further states that optimization consists of three parts: optimization of tumor region guidance $\mathbf{Z}_{t,m}$1, optimization of each modality-specific segmentation mask $\mathbf{Z}_{t,m}$2, and optimization of the final mask $\mathbf{Z}_{t,m}$3. This strongly indicates that $\mathbf{Z}_{t,m}$4 is supervised as a whole-tumor guidance target rather than learned only indirectly through end-to-end gradients.

The most direct quantitative evidence for MCP-Encoder appears in the MSM-Seg ablation study. Starting from the original SAM2 framework with Hiera-S, the baseline reports 75.18 Avg Dice on BraTS-METS and 78.55 on BraTS-AGPT. Adding MSMA alone raises these values to 75.83 and 79.36. Adding MCP-Encoder on top of MSMA raises them further to 76.70 and 80.43, corresponding to incremental gains of +0.87 Dice on BraTS-METS and +1.07 Dice on BraTS-AGPT over the MSMA-only variant. HD95 also improves from 15.62 to 15.14 on BraTS-METS and from 6.55 to 6.30 on BraTS-AGPT. The paper explicitly interprets this as validating “the effectiveness of category-agnostic multi-scale prompt encoding” (Luo et al., 12 Oct 2025).

The evidence for superiority over category-specific prompting is more indirect. The paper conceptually contrasts its whole-tumor category-agnostic approach with labor-intensive subregion-specific prompting and reports that MSM-Seg outperforms several prompt-based baselines, but it does not provide a controlled ablation in which a category-specific prompt encoder is substituted under otherwise identical conditions. Likewise, qualitative claims about more complete tumor delineation, fewer false positives, and better morphology preservation are presented at the framework level rather than as isolated MCP-Encoder visualizations.

5. Relation to adjacent prompt-learning research

Within the broader prompt-learning literature, MCP-Encoder combines three ideas that appear separately in neighboring work: multi-scale prompting, category-agnostic or shared prompt parameterization, and robustness-oriented prompt representation. The specific combination in MSM-Seg is distinctive because it operates on per-slice multi-modal medical features and converts coarse whole-tumor guidance into a decoder-conditioning tensor rather than into class logits (Luo et al., 12 Oct 2025).

The paper “From Points to Clouds: Learning Robust Semantic Distributions for Multi-modal Prompts” reframes prompt learning as distributional denoising rather than optimization of a single deterministic prompt vector. Its core claim is that robust prompt adaptation should learn a semantic cloud, operationalized through noisy prompt perturbation and a denoising objective for a visual-language mapper (Li et al., 28 Nov 2025). That work is not a medical segmentation method and does not present an MCP-Encoder, but it is relevant because it argues that pointwise prompts are brittle, overfit base classes, and generalize poorly to ambiguity. This suggests a possible conceptual extension of MCP-Encoder toward prompt representations that are not only multi-scale and category-agnostic, but also explicitly robust to perturbation or uncertainty.

The paper “Multi-Scale Visual Prompting for Lightweight Small-Image Classification” presents a different notion of multi-scale prompting: a shared bank of global, mid-scale, and local prompt maps fused directly into image space (Khazem, 3 Dec 2025). Those prompts are task-shared rather than class-specific, which makes them category-agnostic in the practical sense that they are applied uniformly to all inputs within a dataset. Although this work concerns small-image classification rather than medical segmentation, it provides a concrete example of prompt encoding as the construction of aligned spatial conditioning maps at multiple resolutions. Relative to MSM-Seg, the major difference is that MSVP stores fixed prompt tensors, whereas MCP-Encoder fuses prompt information with memory-enhanced image features and can also operate in an automatic mode without manual annotation.

The earlier paper “Multi-Prompt with Depth Partitioned Cross-Modal Learning” is relevant mainly as a precursor for hierarchical prompt specialization (Tian et al., 2023). PMPO replaces a single prompt with multiple learned prompts tied to different depth partitions of a frozen CLIP visual encoder. Its learned prompt parameters are shared across classes, but the resulting text representations remain class-specific because each prompt is concatenated with class-name tokens. It is therefore only partially category-agnostic. The transferable lesson for MCP-Encoder is architectural rather than semantic: prompt multiplicity benefits from structured differentiation, and hierarchical deployment across encoder depth can reduce prompt collapse. By contrast, MCP-Encoder’s category-agnosticity is grounded in whole-tumor region guidance rather than in class-shared soft contexts.

Taken together, these neighboring works delineate a design space in which MCP-Encoder occupies a specific niche: a multi-scale, slice-wise, spatial prompt encoder for multi-modal medical segmentation, conditioned by memory and oriented toward whole-region guidance rather than class prototype construction.

6. Limitations, ambiguities, and open directions

Several aspects of MCP-Encoder are explicitly under-specified in the source description. The most immediate is the internal representation of the prompt maps $\mathbf{Z}_{t,m}$5. The paper confirms that box prompts are resized by bilinear interpolation into multi-scale prompt maps and concatenated with features, but it does not state whether these maps are binary rasters, soft masks, coordinate channels, or some other encoding (Luo et al., 12 Oct 2025). Likewise, it does not provide exact channel dimensionalities, feature resolutions per scale, or the exact number of scales $\mathbf{Z}_{t,m}$6.

A second ambiguity concerns the precise alignment of feature scales during progressive fusion. The fusion equation requires $\mathbf{Z}_{t,m}$7, $\mathbf{Z}_{t,m}$8, and possibly $\mathbf{Z}_{t,m}$9 to be spatially compatible, yet the paper does not describe explicit upsampling or downsampling between adjacent fused levels before concatenation. This suggests that either the features are already aligned when fused or omitted implementation details handle scale reconciliation. Since the paper does not specify which, the alignment mechanism remains a missing architectural detail.

The automatic mode is also only partially elaborated. The paper states that no manual prompt is required and that guidance is generated “autonomously” from image encoder features and MSMA, but it does not provide a separate prompt proposal network or a more explicit algorithmic account of how autonomous guidance is obtained. Similarly, $\mathbf{P}_{t,m}$0 in Eq. (7) appears without a local definition in the MCP-Encoder subsection.

At the empirical level, the module is supported by ablation gains and framework-level visualization results, but not by a direct head-to-head comparison against an otherwise identical category-specific prompt encoder. Therefore, the practical advantage of whole-tumor prompting is strongly motivated and partially validated, but not isolated in a controlled prompt-type experiment. A plausible implication is that the paper’s strongest contribution is the demonstration that coarse whole-tumor guidance can be productively fused with multi-modal and inter-slice memory, rather than a definitive proof that all category-specific prompting strategies are inferior.

From the perspective of later prompt-learning research, two open directions emerge. One is whether category-agnostic prompt maps should remain deterministic spatial priors or be treated as distributional prompt representations, as suggested by semantic-cloud approaches (Li et al., 28 Nov 2025). Another is whether multi-scale prompt conditioning should remain a fixed fusion process or evolve toward more explicitly structured hierarchies across input resolution, latent depth, or cross-modal prompt spaces, as explored in differing forms by MSVP and PMPO (Khazem, 3 Dec 2025, Tian et al., 2023). These extensions are not part of MCP-Encoder as defined in MSM-Seg, but they clarify the conceptual frontier around the module.

In summary, MCP-Encoder is the component in MSM-Seg that operationalizes multi-scale category-agnostic prompting for multi-modal brain tumor MRI. It accepts memory-enriched per-slice features, optionally incorporates a coarse whole-tumor box, fuses prompt and feature information layer by layer across scales, produces a sigmoid-shaped tumor guidance prior $\mathbf{P}_{t,m}$1, and injects that prior into decoding by additive fusion. Its importance lies less in architectural complexity than in the specific reformulation of prompt-based segmentation it embodies: prompting the tumor region as a whole, across scales, in a way compatible with both clinician interaction and automatic feature-driven guidance (Luo et al., 12 Oct 2025).