Cellpose-SAM: Hybrid Segmentation Framework

• Cellpose-SAM is a hybrid framework bridging generalist biomedical segmentation and prompt-based workflows to enhance cell and nuclei analysis.
• It applies auto-prompting strategies, fine-tuning of frozen encoders/decoders, and weak supervision to improve segmentation precision.
• Effective data selection, replay techniques, and domain sequencing in Cellpose-SAM mitigate forgetting and boost cross-domain performance.

Cellpose-SAM denotes, in recent biomedical image-segmentation discourse, a SAM-centered cell segmentation regime situated between generalist biomedical segmenters such as Cellpose and the broader family of promptable foundation models. The available data suggest that the term is used less as the name of a single canonical architecture than as a comparative label for auto-prompting and cell-specific SAM workflows, especially in nuclei and cell segmentation. In that literature, Cellpose supplies the reference point of a generalist biomedical segmenter, whereas SAM contributes prompt-based zero-shot segmentation and cell-specific direct segmentation; later systems are frequently described as extending, replacing, or outperforming “Cellpose-SAM” strategies (Zhao et al., 6 Nov 2025, Li et al., 21 Aug 2025, Fang et al., 23 Jul 2025).

1. Position within biomedical image segmentation

Generalist biomedical image segmentation models such as Cellpose are increasingly applied across diverse imaging modalities and cell types. In parallel, the SAM literature in computational pathology distinguishes two primary methods for nuclei segmentation: prompt-based zero-shot segmentation and the use of cell-specific SAM models for direct segmentation. The same literature also notes that general vision foundation models often face challenges with fine-grained semantic segmentation, such as identifying specific nuclei subtypes or particular cells. This situates Cellpose-SAM at the junction of generalist instance segmentation and domain-adapted promptable segmentation (Zhao et al., 6 Nov 2025, Li et al., 21 Aug 2025).

This suggests that Cellpose-SAM is best understood as a design space rather than a universally standardized model family. Its practical role is to mediate between two desiderata that are often in tension in biomedical segmentation: broad visual generalization and task-specific instance fidelity. The former is associated with Cellpose-like generalist behavior; the latter is pursued through prompt engineering, auto-prompt generation, or decoder adaptation around SAM.

2. Architectural motifs and prompting regimes

Recent literature attributes a recognizable set of motifs to “Cellpose-SAM-like frameworks.” One source states that All-in-SAM goes beyond the “typical auto-prompting strategy of Cellpose-SAM.” Another describes Cellpose-SAM-like frameworks as typically using a frozen SAM encoder plus a task-specific or fine-tuned decoder, possibly with point or box prompts or additional prompt encoders. That same comparison argues that such systems still rely on single-representation mapping and explicit user prompts, and may struggle to adjust for small objects or morphological outliers such as highly variable granules and mitochondria (Li et al., 21 Aug 2025, Fang et al., 23 Jul 2025).

Adjacent SAM-based cell systems instantiate several of these motifs explicitly. Segment Any Cell (SAC) augments the SAM Vision Transformer with Low-Rank Adaptation in the Query and Value matrices of the self-attention layers, keeps the main encoder and prompt encoder frozen, fine-tunes the mask decoder, and adds an auto-prompt generator based on an auxiliary U-Net. CellSAM instead uses a shared ViT backbone together with CellFinder, a transformer-based object detector that predicts bounding boxes as prompts for SAM; in its prompting experiments, bounding box prompts were found to be much more reliable than point prompts for cellular images (Na et al., 2024, Israel et al., 2023).

In architectural terms, the common denominator is not a single module but a recurrent pipeline logic: prompt construction, prompt-conditioned mask decoding, and domain adaptation with limited biomedical supervision. Where the literature diverges is in how prompts are obtained—manually, detector-generated, self-generated, or replaced by learned class prototypes.

3. Weak supervision, pseudo-masks, and corrective learning

A major operational context for Cellpose-SAM is weak supervision. SAM-assisted molecular-empowered learning (SAM-L) uses weak bounding box annotations to produce pixel-level segmentation masks and then uses those SAM-generated masks to train a downstream segmentation model. The pipeline allows lay annotators to work with box annotations rather than manual delineations; manual contouring a single cell type on one whole slide image was reported as approximately 9 hours. In the same study, SAM-L with random boxes reached mean F1 scores of 0.8577 for podocytes and 0.8469 for mesangial cells, compared with 0.8496 and 0.8473 for manual contour annotation, and point prompts were found less effective than box prompts (Li et al., 2023).

A related histopathology study formalized two complementary workflows: D-SAM, in which a detector predicts bounding boxes that prompt SAM at test time, and SAM-S, in which SAM generates pseudo-masks over training data to train a standalone segmentation model. Their reconciliation method, SAM-ILP, combined the two mask sets through an integer linear program over ambiguous pixels. On ConSep, MoNuSeg, and TNBC, SAM-ILP obtained Dice scores of 81.39, 81.45, and 83.58, and the SAM-based workflows were reported to yield 9–15 point Dice gains over previous weakly supervised segmentation models (Tyagi et al., 2023).

A plausible implication is that Cellpose-SAM derives much of its utility from annotation transformation rather than merely from direct inference. In this reading, it functions as an intermediary that converts weak supervisory signals—boxes, sparse prompts, detector outputs, or approximate masks—into dense supervision suitable for specialist biomedical segmentation training.

4. Data efficiency, redundancy, and transfer robustness

The most detailed data-centric analysis comes from a Cellpose case study that is directly relevant to Cellpose-SAM-like training pipelines. That study introduced a dataset quantization (DQ) strategy, a bin-based coreset selection methodology that uses pretrained Masked Autoencoder features $f(\cdot)$, divides the dataset into $N$ bins $\{S_1,\ldots,S_N\}$, and scores candidates with the submodular gain

$$P(x_k)=\sum_{p \in S^{k-1}_n}\|f(p)-f(x_k)\|^2-\sum_{p \in D \setminus S^{k-1}_n}\|f(p)-f(x_k)\|^2,$$

with the final coreset defined as

$S^*=\bigcup_{n=1}^{N} g(S_n,\rho).$

On the Cyto dataset, image segmentation performance saturated when training used only 10–40% of the data, implying that over 60% of the training images were functionally redundant. With only 10% of the training data, Cellpose achieved a Dice score of $0.823 \pm 0.171$ and panoptic quality of $0.633 \pm 0.247$, whereas full data achieved $0.855 \pm 0.157$ and $0.687 \pm 0.212$. The same work reported that DQ-selected patches were visually and structurally more diverse, and that t-SNE projections of MAE embeddings showed DQ-selected samples spanning the entire latent space more effectively than random sampling (Zhao et al., 6 Nov 2025).

The same study also documented catastrophic forgetting under cross-domain finetuning. In the Cyto $\rightarrow$ Histo setting, source-domain Cyto Dice dropped from $N$0 to $N$1, indicating near-total forgetting. Selective replay of only 5–10% of DQ-selected Cyto data during Histo finetuning substantially restored Cyto performance, with Dice climbing from $N$2 to $N$3, while incorporating 100% Cyto replay during Histo finetuning hurt Histo performance, with Dice dropping from $N$4 to $N$5. In multi-stage transfer, starting with diverse domains such as MultiInst yielded more robust representations, and interleaving generalist Cyto mid-sequence could recover generalist performance even after specialization. Although these findings were reported for Cellpose rather than for a named Cellpose-SAM implementation, they strongly suggest that any Cellpose-SAM pipeline inheriting Cellpose-style generalist-to-specialist adaptation must treat data selection, replay fraction, and domain ordering as first-class design variables (Zhao et al., 6 Nov 2025).

5. Comparative developments beyond Cellpose-SAM

Several later systems position themselves explicitly against Cellpose-SAM or against closely related design assumptions. All-in-SAM is presented as a full-stack framework for fine-grained multi-class nuclei segmentation that combines molecular-empowered annotation, adapter-based SAM fine-tuning, and Molecular-Oriented Corrective Learning. During inference, segmentation can be accomplished directly from PAS images, without box prompts or molecular images. In the comparative summary provided for that work, All-in-SAM is described as outperforming general SAM, Cellpose-SAM, nnUNet, and other state-of-the-art approaches, especially with weak labels or limited training data, and as going beyond the typical auto-prompting strategy of Cellpose-SAM by incorporating molecular data and corrective learning (Li et al., 21 Aug 2025).

ScSAM advances a different critique. It argues that Cellpose-SAM-like frameworks rely on single-representation mapping and explicit user prompts, which is problematic for subcellular morphology and skewed distributions. Its response is to fuse frozen SAM and MAE encoders through a feature alignment and fusion module, and to replace manual prompting with a cosine similarity matrix-based class prompt encoder. In this formulation, class prototypes activate class-specific features without manual prompts, and the overall loss combines alignment, contrastive, and Dice terms:

$N$6

The reported outcome is state-of-the-art performance on BetaSeg high- and low-glucose subsets, including mIoU values of 0.785 and 0.787 and granule Dice scores of 0.798 and 0.767, together with stronger cross-domain stability than single-encoder alternatives (Fang et al., 23 Jul 2025).

Other SAM-derived systems expand the comparative field around Cellpose-SAM even when they do not name it directly. CellSAM uses CellFinder-generated bounding box prompts and reports that generalist training improves CellSAM while hurting Cellpose; on LIVECell, CellSAM generalist zero-shot segmentation was reported as more than four times better than generalist Cellpose. subCellSAM moves in a zero-shot direction, using a three-step nuclei, cell, and subcellular segmentation pipeline with self-prompting based on morphological and topological priors; on BBBC008 it reported DSC/IoU of 0.901/0.832, compared with 0.887/0.801 for CellPose 3 (Israel et al., 2023, Hanimann et al., 19 Aug 2025).

Taken together, these developments indicate that Cellpose-SAM has become less a fixed endpoint than a reference architecture against which newer systems define improvements in annotation efficiency, prompt quality, semantic specificity, and robustness to morphological variability.

6. Limits, misconceptions, and research directions

A recurrent misconception in the surrounding literature is that out-of-the-box SAM, given enough prompts, is sufficient for biomedical segmentation. Multiple studies dispute this directly. One reports that directly applying SAM to medical segmentation may not produce satisfactory results because of the inherent absence of medical knowledge. Another emphasizes that the generation of high-quality, informative prompts is as crucial as applying state-of-the-art fine-tuning techniques on foundation models. In SAC’s benchmarks, zero-shot SAM with one expert prompt performed much worse than dedicated methods: on MoNuSeg it obtained F1 25.36, IoU 14.24, and Dice 24.03, whereas SAC in the 0-expert setting reached F1 84.11, IoU 72.61, and Dice 84.03; on DSB, zero-shot SAM obtained F1 66.79, IoU 58.37, and Dice 69.79, whereas SAC 0-expert reached F1 93.48, IoU 87.32, and Dice 93.04 (Zhao et al., 2023, Na et al., 2024).

A second misconception is that more supervision, more prompts, or more replay must monotonically improve performance. The evidence is more conditional. Increasing expert prompts for non-fine-tuned SAM does not necessarily improve performance and may degrade it. In transfer, full replay can hinder target adaptation, whereas selective replay of 5–10% source data can substantially restore source performance. The same transfer study also found that training domain sequencing improves generalization and reduces forgetting, with diverse early domains producing more robust representations (Na et al., 2024, Zhao et al., 6 Nov 2025).

This suggests that the future of Cellpose-SAM lies less in scaling a single model component than in coordinating prompt engineering, weak-supervision conversion, data selection, replay scheduling, and domain curricula. The most credible trajectories in the current literature are retention-aware learning, prompt-free or self-prompted class activation, and biologically informed prompting that encodes nucleus–cell relations, neighboring-cell repulsion, or molecular context directly into the segmentation loop.