UltraUPConvNet: Unified Ultrasound AI

• The paper demonstrates that UltraUPConvNet unifies tissue segmentation and disease prediction using a shared ConvNeXt-Tiny encoder and UPerNet-style decoder.
• It introduces four automatic prompts—nature, position, task, and type—that are injected into both segmentation and classification branches to enhance cross-task consistency.
• The model achieves state-of-the-art performance with segmentation and classification averages of 90.28% and 89.77%, respectively, while using 30% fewer parameters than comparable systems.

UltraUPConvNet is a UPerNet- and ConvNeXt-based multi-task network for ultrasound tissue segmentation and disease prediction that unifies image classification and segmentation within a single, computationally efficient, purely convolutional framework (Chen, 14 Sep 2025). The model couples a shared ConvNeXt-Tiny encoder with a UPerNet-style segmentation decoder, dedicated classification heads for binary and 4-way tasks, and a promptable mechanism in which four automatic prompts—nature, position, task, and type—are projected into feature space and injected into both task branches. It is trained on a large-scale ultrasound collection containing more than 9,700 annotations across seven anatomical regions and is presented as an ultrasound-specific contribution to a broader general medical AI direction rather than as a Transformer-heavy universal model.

1. Problem formulation and intended scope

UltraUPConvNet is motivated by a recurrent separation in ultrasound AI: tissue segmentation and disease prediction are commonly treated as distinct problems, with separate architectures, supervision regimes, and deployment pipelines (Chen, 14 Sep 2025). In the formulation described for the model, segmentation corresponds to delineating organs, anatomical regions, or lesions such as breast tumors, fetal head structures, cardiac chambers, thyroid, kidney, and liver, whereas classification corresponds to deciding whether a structure is normal or pathological or assigning disease subtypes. The paper identifies three structural limitations in prior work: task-specific model design, substantial computational overhead in recent universal systems, and limited cross-task knowledge transfer.

The architecture is therefore designed around four explicit goals. It aims to unify ultrasound tissue segmentation and disease prediction within one multi-task framework, remain lightweight and convolutional by using a ConvNeXt backbone and UPerNet decoder rather than Transformer blocks, generalize across multiple anatomical regions and datasets, and introduce promptable learning in a purely CNN framework through four prompts. In this sense, “universal” does not mean task-agnostic in the absence of conditioning; instead, the model uses shared feature extraction with task-specific heads and prompt-conditioned feature modulation.

A common misconception would be to treat UltraUPConvNet as merely a segmentation network with an auxiliary classifier. The published design is more specific: it uses a shared encoder and separate decoders or heads, and training alternates between segmentation batches and classification batches. Conversely, it is also not a segmentation-free classifier with optional dense prediction. Its encoder–decoder organization and UPerNet-derived segmentation branch place dense prediction at the center of the architecture.

2. Architectural organization

The published architecture consists of four principal components: a shared ConvNeXt-Tiny encoder, a prompt projection and injection module, a UPerNet-style segmentation decoder, and a classification branch (Chen, 14 Sep 2025). At the encoder level, ConvNeXt-Tiny produces multi-scale feature maps, denoted $F_1, F_2, F_3, F_4$ from shallow to deep stages. The paper does not provide a full layer-by-layer configuration for ConvNeXt-Tiny, but it attributes to the backbone the standard four-stage organization, large-kernel depthwise separable convolutions, inverted bottlenecks with pointwise convolutions, and LayerNorm-based modern ConvNet design.

The segmentation branch adapts UPerNet to ConvNeXt-Tiny feature dimensions. The deepest feature $F_4$ is first processed by a Pyramid Pooling Module. The PPM pools at multiple scales, exemplified in the paper as $1\times1$, $2\times2$, $3\times3$, and $6\times6$, upsamples each pooled feature back to the spatial size of $F_4$, concatenates the results, and applies convolution to produce a context-enriched representation. This output is then propagated through a Feature Pyramid Network. In the FPN, coarser features are upsampled step by step and fused additively with lateral projections of finer-scale encoder features. The final segmentation head combines fused feature maps and produces segmentation logits $x_{\text{seg}}$.

The classification branch uses high-level encoder features and prompts. It contains two heads: a 2-way classifier for binary tasks and a 4-way classifier for multi-class tasks. A batch-level metadata flag determines which head and loss are active. If a batch contains both task types, the losses from both heads are computed and summed. This arrangement allows the model to accommodate datasets with heterogeneous label granularities without redefining the backbone.

Prompting is implemented without user interaction. Each of the four prompts—nature, position, task, and type—is encoded as a one-hot vector, projected through a fully connected layer into a feature embedding, and added to the relevant feature maps by prompt projection embedding. The prompts therefore operate as structured priors rather than as interactive clicks or boxes of the kind used in SAM-like systems.

3. Prompting mechanism and multi-task optimization

The prompt design is central to how UltraUPConvNet handles heterogeneity across organs and tasks (Chen, 14 Sep 2025). The four prompts encode prior information about anatomy and clinical context, and the same prompt family is injected into both the segmentation decoder and the classification branch. The reported ablation indicates that prompts produce a modest overall gain, with the effect more visible in classification than in segmentation.

For segmentation, the model uses a compound objective combining cross-entropy and soft Dice loss:

$$\mathcal{L}_{\text{seg}} = 0.4 \cdot \mathcal{L}_{\text{CE}}(x_{\text{seg}}, y) + 0.6 \cdot \mathcal{L}_{\text{Dice}}(x_{\text{seg}}, y).$$

Here, $x_{\text{seg}}$ denotes the model output for segmentation and $F_4$0 the ground-truth mask. The weighting is intended to balance pixel-wise classification with region-wise consistency, which is particularly relevant for ultrasound because boundaries can be ambiguous and lesions can be small.

For classification, the paper specifies separate softmax cross-entropy objectives for binary and 4-class tasks:

$F_4$1

The batch-level metadata flag controls which classifier head is used.

The overall training strategy is alternating rather than fully joint at the batch level. Two data loaders are used, one for segmentation datasets and one for classification datasets, and segmentation and classification batches are processed separately during each epoch. The final loss is described as

$F_4$2

The paper explicitly sets $F_4$3, stating that this weighting was chosen empirically based on validation performance. This design is intended to stabilize optimization and to prevent one task from dominating the shared encoder. It also clarifies that the model’s multi-task behavior does not come from simultaneous per-sample supervision of all labels; rather, it comes from shared representation learning across alternating task-specific updates.

4. Data regime, annotation coverage, and implementation

UltraUPConvNet is trained and evaluated on a combined ultrasound collection covering seven anatomical regions: breast, liver, kidney, thyroid, fetal head, cardiac, and appendix (Chen, 14 Sep 2025). The public component includes BUSI, BUSIS, BUS-BRA, Fatty-Liver, kidneyUS, DDTI, Fetal HC, CAMUS, and Appendix. The private component includes Appendix, Breast, Breast-luminal, Cardiac, Fetal Head, Kidney, Liver, and Thyroid datasets. The total annotation count exceeds 9,700.

The aggregated dataset is split 7:1:2 into training, validation, and testing, corresponding to 70%, 10%, and 20%. This split follows the challenge setup identified as UUSIC25 in the supplied details. Public and private data are both used to support the model’s cross-region and cross-task scope.

Implementation is in PyTorch. ConvNeXt-Tiny is initialized from official pretrained weights. Training runs for 200 epochs with AdamW and an initial learning rate of $F_4$4. Data augmentation consists of random horizontal flipping, random rotation in the range $F_4$5 to $F_4$6, and random cropping. The paper emphasizes efficiency by noting that training fits on an RTX 2060 with only 6 GB VRAM.

The reported parameter counts are as follows:

Model	Parameters	Total Average
SAMUS	130.10 M	—
UniUSNet	86.29 M	81.93%
UltraUPConvNet w/o prompt	60.44 M	89.90%
UltraUPConvNet	60.48 M	90.11%

This parameterization supports two related claims in the paper: UltraUPConvNet is approximately 30% smaller than UniUSNet and less than half the size of SAMUS, and its memory footprint is compatible with mid-range GPUs. FLOPs and inference speed are not explicitly reported.

5. Empirical performance and ablation behavior

The reported quantitative evaluation compares UltraUPConvNet with SAMUS, UniUSNet, and a no-prompt ablation of the same architecture (Chen, 14 Sep 2025). The metrics are presented as percentages and are described in the supplied details as likely Dice scores or a similar aggregated measure. On segmentation, the prompted UltraUPConvNet reports dataset-specific scores of 88.46% on BUS-BRA, 91.33% on BUSIS, 94.71% on CAMUS, 80.55% on DDTI, 97.11% on Fetal_HC, and 89.49% on KidneyUS. Its segmentation average is 90.28%, compared with 85.80% for UniUSNet and 80.01% for SAMUS.

On classification, the prompted model reports 77.30% on Appendix, 92.02% on the classification component of BUS-BRA, and 100% on Fatty-Liver. The corresponding classification average is 89.77%, compared with 74.20% for UniUSNet. The total average reaches 90.11%, versus 89.90% for the no-prompt variant and 81.93% for UniUSNet. The paper characterizes this as state-of-the-art average performance across the joint segmentation and classification evaluation while using fewer parameters than the main multi-task comparator.

The ablation on prompts is notable because the gain is small but structured. The no-prompt model slightly exceeds the prompted model on some segmentation subsets, including BUSIS and Fetal_HC, and its segmentation average is marginally higher at 90.37% versus 90.28%. However, prompting improves classification average from 88.95% to 89.77% and yields the best total average. The supplied analysis therefore treats prompting as a mechanism whose principal benefit lies in classification and overall cross-task consistency rather than in a uniform lift across all segmentation benchmarks.

Qualitative results are reported to show good boundary delineation and shape consistency across organs such as fetal head and cardiac chambers. The prompted version is said to align better with the ground truth in some ambiguous regions where organ or position priors help. The paper also states that the model is visually robust to common ultrasound artifacts such as speckle and shadowing, although no separate robustness benchmark is reported. It additionally notes that confidence intervals and formal statistical significance tests are not provided.

6. Limitations, interpretation, and related architectural questions

The model is presented as broadly generalizing across public and private ultrasound datasets and across seven organs, but the paper also leaves several constraints explicit or implicit (Chen, 14 Sep 2025). Although more than 9.7k images is large for ultrasound, it remains small relative to vision foundation-model scales. Some private subsets are small, including 67-image datasets. Annotation quality and label definitions may vary across public sources. The authors further note that real-time clinical applicability in diverse environments still requires assessment, and they state that they will further assess the model’s adaptability in future work.

Another important interpretive point concerns what the paper does not evaluate. UltraUPConvNet is an upsampling-heavy encoder–decoder for segmentation, but its published results focus on multi-task accuracy and parameter efficiency rather than on exact translation consistency. Related work on adaptive polyphase downsampling and adaptive polyphase upsampling shows that symmetric encoder–decoder CNNs with ordinary downsampling and upsampling inherit shift-equivariance failures, and that APS-D together with APS-U can enforce

$F_4$7

while retaining a U-Net-like architecture with standard convolutions (Chaman et al., 2021). That work demonstrates near-perfect shift equivariance in MRI and CT reconstruction and shows that the gains extend outside the training distribution.

This does not mean that UltraUPConvNet currently incorporates APS-D/U. It does not. A plausible implication, however, is that its UPerNet-style top-down path could inherit the generic shift-equivariance pathologies associated with downsampling and upsampling in dense prediction networks. A corresponding plausible extension would be to replace fixed downsampling and upsampling stages with adaptive polyphase operators if future work prioritizes exact translation equivariance in ultrasound segmentation maps. Such an extension would remain conceptually separate from the published claims about promptable multi-task learning, ConvNeXt-based efficiency, and the reported ultrasound benchmarks.

UltraUPConvNet is therefore best understood as a compact, prompt-conditioned, multi-task ultrasound ConvNet: shared in representation, specialized in output heads, trained over heterogeneous datasets, and empirically strong on the reported benchmarks. Its primary contribution lies in demonstrating that a single moderately sized convolutional architecture can jointly support tissue segmentation and disease prediction across multiple organs without relying on Transformer-heavy universal models.