Edge2Prompt: Modality-Agnostic Liver Segmentation

• Edge2Prompt is a modality-agnostic liver segmentation pipeline that converts edge maps into dense logit prompts to condition a frozen SAM-2 for refined liver masking.
• It employs a modified U-Net prompt generator and a composite loss function to achieve effective segmentation even in scarce-data and cross-modality out-of-distribution scenarios.
• Experimental results show competitive Dice scores and superior OOD performance compared to classical U-Net methods, supporting its clinical utility in 3D reconstruction.

Searching arXiv for the cited papers and related Edge2Prompt work. arxiv_search(query="Edge2Prompt modality-agnostic liver segmentation SAM-2", max_results=5) arxiv_search(query="APEX Automated Prompt Engineering eXpert Dynamic Data Selection", max_results=5) Edge2Prompt is a modality-agnostic liver segmentation pipeline that combines classical edge detection, a U-Net prompt generator, and frozen Segment Anything Model 2 (SAM-2) to support cross-modality and out-of-distribution (OOD) segmentation from 2D abdominal slices, followed by reconstruction into a 3D liver volume (Hollet et al., 6 Aug 2025). In the cited work, the term denotes a specific medical-image segmentation method rather than a general prompt-engineering framework. Its defining idea is to replace modality-specific intensity cues with modality-agnostic edge structure, convert that structure into a dense logit-based prompt, and use the prompt to condition SAM-2 for refined liver masking across CT and MR, including settings with scarce labeled data (Hollet et al., 6 Aug 2025).

1. Definition and conceptual scope

Edge2Prompt is presented as a modality-agnostic, data-efficient liver segmentation pipeline for CT and MR, with particular emphasis on OOD cross-modality generalization such as training on MR and testing on CT, or vice versa (Hollet et al., 6 Aug 2025). The target task is segmentation of liver structures from 2D abdominal slices with subsequent reconstruction of a 3D segmented organ volume. The motivating application is preoperative planning for interventions such as tumor resection and transplantation, where modality specificity, domain shift across scanners and protocols, and scarcity of labeled data limit clinical deployment (Hollet et al., 6 Aug 2025).

The method is built around a structural premise: edge maps are treated as more stable across modalities than raw intensity patterns. Rather than requiring a network to learn modality-dependent appearance statistics directly from CT and MR images, Edge2Prompt uses a modality-agnostic edge representation and asks a learned prompt generator to translate those edges into a dense prompt that SAM-2 can exploit (Hollet et al., 6 Aug 2025). This makes the method “self-prompting” in the sense that prompts are produced automatically from the image, not manually specified as points or boxes.

A recurrent source of ambiguity is the phrase “Edge2Prompt” itself. In the provided literature, it names the liver-segmentation pipeline of "Edge2Prompt: Modality-Agnostic Model for Out-of-Distribution Liver Segmentation" (Hollet et al., 6 Aug 2025). By contrast, adjacent prompt-optimization papers use “edge,” “frontier,” or evidence-focused prompting in different senses. APEX defines a data-centric prompt optimization framework that prioritizes mixed-performance frontier examples during prompt search (Wang et al., 9 Jun 2026), while "Automatic Prompt Generation via Adaptive Selection of Prompting Techniques" treats prompt generation as semantic routing over task clusters (Ikenoue et al., 20 Oct 2025). This suggests that “Edge2Prompt” can function informally as a broader metaphor for converting edge, frontier, or distilled signals into prompts, but the concrete technical meaning in the cited medical-imaging paper is the segmentation pipeline itself.

2. Pipeline architecture

The pipeline begins with an Edge Detection Module (EDM) operating on grayscale input slices $i \in \mathbb{R}^{240\times240}$ (Hollet et al., 6 Aug 2025). Before edge extraction, the image is processed with histogram equalization and bilinear filtering. A Canny filter $C$ is then applied to compute an edge response,

$C(i) \in \mathbb{R}^{240\times240},$

and the response is thresholded into a binary magnitude map (Hollet et al., 6 Aug 2025). The stated purpose of binarization is to reduce dependence on modality-specific intensity statistics while preserving structural information such as boundaries and topology.

The edge map is passed to a slightly modified U-Net that acts as a prompt generator rather than a standalone segmenter. The paper describes this network as a standard U-Net-like encoder-decoder with bilinear upsampling in the decoder followed by convolution layers. Its output is a dense logit map,

$$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$

where $u_\theta$ denotes the U-Net with learnable parameters $\theta$ (Hollet et al., 6 Aug 2025). This output is the “logit-based prompt.” Unlike sparse point- or box-based prompting, the prompt is a learned dense spatial field that acts as a soft prior over the likely liver region.

The prompt is then consumed by a differentiable SAM-2 wrapper $\mathcal{S}$. The training flow is explicitly described as follows: the U-Net predicts $z_u$; the logits enter SAM-2’s prompt encoder; the original grayscale image is converted to RGB and passed to SAM-2’s image encoder; SAM-2’s mask decoder combines prompt and image embeddings; and the refined output mask is

$$\hat{m} = \mathcal{S}(z_u) \in \mathbb{R}^{240\times240}.$$

A key design choice is that all SAM-2 weights are frozen, and only the U-Net is trained (Hollet et al., 6 Aug 2025). The intended effect is lightweight adaptation: the trainable component learns how to emit prompts that a fixed foundation model can interpret effectively.

The paper’s interpretation is that SAM-2 provides a strong segmentation prior, so the U-Net need not solve the entire segmentation problem from scratch. Instead, it learns to synthesize prompts that steer SAM-2 toward the correct liver region, which is especially relevant in low-data and cross-modality settings (Hollet et al., 6 Aug 2025).

3. Formalization, objective, and 3D reconstruction

The training dataset is written as

$D_T^{TRAIN} = \{(i_t, m_t)\mid t \in T\},$

where $C$0 is the input image and $C$1 is the corresponding ground-truth mask (Hollet et al., 6 Aug 2025). To simulate data scarcity, the paper also defines

$C$2

Training uses a composite loss,

$C$3

with empirically chosen weights

$C$4

(Hollet et al., 6 Aug 2025). The focal term addresses class imbalance for small liver regions. For each pixel $C$5, with raw logit $C$6, label $C$7, and probability

$C$8

the paper defines

$C$9

and

$C(i) \in \mathbb{R}^{240\times240},$0

The Dice term $C(i) \in \mathbb{R}^{240\times240},$1 measures overlap, and the third term is log-cosh Dice,

$C(i) \in \mathbb{R}^{240\times240},$2

introduced to smooth Dice loss and stabilize gradients (Hollet et al., 6 Aug 2025).

The segmentation is produced slice-wise in 2D, but the end goal is 3D organ volume reconstruction. After 2D masks are predicted, an affine transformation places slices into physical space using voxel spacing and world coordinates, and the segmented slices are stacked to reconstruct the 3D liver volume (Hollet et al., 6 Aug 2025). The paper notes that sparse slice coverage, especially in MR, may require interpolation or super-resolution before affine-based reconstruction. This indicates that Edge2Prompt is designed as a clinically oriented 2D-to-3D pipeline rather than a native volumetric model.

4. Experimental design and evaluation protocol

Edge2Prompt is evaluated on the CHAOS dataset, the Combined (CT-MR) Healthy Abdominal Organ Segmentation Challenge Data, which contains 40 volumetric scans: 20 CT and 20 MR, including T2-SPIR and T1-DUAL in-phase and out-of-phase sequences (Hollet et al., 6 Aug 2025). This multimodal composition is central to the paper’s claim of modality-agnostic evaluation.

Two principal train/validation/test settings are reported. In the standard mixed-modality setup, training uses 20 volumes consisting of 10 CT and 10 MR, validation uses 10 volumes consisting of 5 CT and 5 MR, and testing uses 10 volumes consisting of 5 CT and 5 MR (Hollet et al., 6 Aug 2025). In the data-scarce setup, training uses only 2 volumes total, one CT and one MR, while testing uses 14 CT and 14 MR volumes (Hollet et al., 6 Aug 2025). The latter is explicitly framed as simulating clinical scenarios with very limited annotation availability.

The paper also introduces cross-modality OOD testing as its strongest generalization setting: train on CT and test on MR, and train on MR and test on CT (Hollet et al., 6 Aug 2025). Evaluation uses Dice score, IoU, and $C(i) \in \mathbb{R}^{240\times240},$3, defined as the mean Dice score over both in-distribution and OOD tasks. Dice is presented as the principal metric.

Comparisons are made against four baselines: imU-Net, a U-Net on raw images; emU-Net, a U-Net on edge maps; sU-Net, a U-Net on raw images plus SAM-2; and spSAM, a self-prompting SAM-2 method using logistic regression and sampled point or box prompts (Hollet et al., 6 Aug 2025). These baselines separate the contribution of edge representations, SAM-2 integration, and the specific dense-prompt design used by Edge2Prompt.

5. Reported performance and ablation findings

In the standard mixed in-distribution setting on $C(i) \in \mathbb{R}^{240\times240},$4, the paper reports Dice scores of $C(i) \in \mathbb{R}^{240\times240},$5 for imU-Net, $C(i) \in \mathbb{R}^{240\times240},$6 for emU-Net, $C(i) \in \mathbb{R}^{240\times240},$7 for sU-Net, $C(i) \in \mathbb{R}^{240\times240},$8 for spSAM, and $C(i) \in \mathbb{R}^{240\times240},$9 for Edge2Prompt (Hollet et al., 6 Aug 2025). These values support the paper’s characterization of Edge2Prompt as competitive with classical U-Net baselines in-distribution, while substantially outperforming spSAM.

In the data-scarce setting on $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$0, the reported Dice scores are $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$1 for imU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$2 for emU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$3 for sU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$4 for spSAM, and $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$5 for Edge2Prompt (Hollet et al., 6 Aug 2025). The paper treats this as evidence that the SAM-2-based prompting design remains effective even when only two training volumes are available.

The strongest results occur in cross-modality OOD evaluation. For CT-trained models evaluated on MR, the reported Dice scores are $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$6 for imU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$7 for emU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$8 for sU-Net, $$z_u = u_\theta(\mathrm{EDM}(i)) \in \mathbb{R}^{240\times240},$$9 for spSAM, and $u_\theta$0 for Edge2Prompt (Hollet et al., 6 Aug 2025). For MR-trained models evaluated on CT, the corresponding values are $u_\theta$1, $u_\theta$2, $u_\theta$3, $u_\theta$4, and $u_\theta$5 (Hollet et al., 6 Aug 2025). The abstract summarizes the OOD result as a mean Dice score of $u_\theta$6, outperforming U-Net baselines by $u_\theta$7 and other self-prompting methods by $u_\theta$8 (Hollet et al., 6 Aug 2025).

Ablation studies further specify the method’s dependencies. For loss design, the paper reports $u_\theta$9 with $\theta$0, $\theta$1 with $\theta$2, $\theta$3 with $\theta$4, $\theta$5 with $\theta$6, $\theta$7 with $\theta$8, $\theta$9 with $\mathcal{S}$0, and $\mathcal{S}$1 with the full composite loss (Hollet et al., 6 Aug 2025). For the edge detector, Laplacian yields $\mathcal{S}$2, Sobel $\mathcal{S}$3, and Canny $\mathcal{S}$4, with Canny slightly best and Sobel nearly identical (Hollet et al., 6 Aug 2025). The paper also states that SAM-2 shows sensitivity to object size, contributing to metric variance.

These results support two bounded conclusions. First, the structural edge representation appears to improve transfer across CT and MR relative to raw-image U-Net baselines. Second, the prompting interface to frozen SAM-2 is most beneficial under OOD and sparse-data conditions, rather than by maximizing in-distribution Dice where standard supervised U-Nets are already strong.

6. Position within prompt-centric research and related interpretations

Although Edge2Prompt is a medical-imaging segmentation method, its naming and mechanism invite comparison with prompt-centric research outside medical vision. In the prompt-optimization literature, APEX introduces a data-centric prompt optimization framework in which examples are dynamically stratified into Easy, Hard, and Mixed tiers based on optimization lineage, and Mixed examples are used to define an addressable frontier for mutation and a rank-sensitive frontier for candidate selection (Wang et al., 9 Jun 2026). That framework uses “edge” in the sense of borderline or unstable examples whose behavior changes across prompt variants. This is conceptually distinct from Edge2Prompt’s use of edges as modality-agnostic image structure, but both methods elevate a reduced, high-leverage signal into the prompting loop.

A different line of work, "Automatic Prompt Generation via Adaptive Selection of Prompting Techniques," constructs a knowledge base linking semantic task clusters to prompting techniques such as Role Playing, Emotion Prompting, Stress Prompting, Chain-of-Thought, Least-to-Most, Skeleton-of-Thought, and Scratchpad Prompting, then routes an abstract task description to the nearest cluster by cosine similarity and generates a prompt dynamically (Ikenoue et al., 20 Oct 2025). That paper explicitly frames prompt creation as a retrieval-and-composition problem over task semantics. PromptSplit, in turn, analyzes prompt-dependent disagreement between generative models via tensor-product prompt-output embeddings and covariance-difference eigenspaces, thereby identifying prompt families responsible for behavioral divergence (Lotfian et al., 3 Feb 2026). "Chain of Evidences and Evidence to Generate" proposes CoE and E2G, in which evidence is extracted first and answer generation follows from the extracted evidence, emphasizing grounding over unconstrained reasoning traces (Parvez, 2024).

Taken together, these neighboring works show that “prompt” spans at least three technical meanings in current research: dense conditioning signals for foundation models in vision, strategy selection or optimization in LLM prompting, and evidence- or prompt-aware analysis of model behavior (Hollet et al., 6 Aug 2025). This suggests that Edge2Prompt is best understood within the vision-and-foundation-model branch of prompt research, not as a direct synonym for general automatic prompt engineering.

The limitations reported for Edge2Prompt are correspondingly specific. The evaluation is limited to 2D slices rather than a native 3D model, only CHAOS is used, and the paper leaves ultrasound and fluoroscopy for future work (Hollet et al., 6 Aug 2025). It also notes possible future improvements in edge detection, prompt generation, loss design, and perhaps fine-tuning SAM-2 without inducing domain dependence (Hollet et al., 6 Aug 2025). The clinical significance claimed by the paper is therefore bounded but concrete: automatic liver segmentation across different scanners and modalities, with limited annotated data and 3D output suitable for surgical planning (Hollet et al., 6 Aug 2025).