---
title: 'CytoFM: Dual Cytology & Cytometry Models'
url: https://www.emergentmind.com/topics/cytofm
type: topic
---

# CytoFM: Dual Cytology & Cytometry Models

CytoFM refers to two distinct but foundational methodologies in cytological and cytometric analysis, both of which are central to modern computational pathology and single-cell biology. These include (1) CytoFM—The First Cytology Foundation Model, a self-supervised vision transformer trained on digital cytology images for representation learning and classification [2504.13402]; and (2) CytoFM—A Bayesian Feature Allocation Model for identification of cell subpopulations in cytometry data using a finite Indian buffet process [2002.08609]. Each approach addresses unique methodological and biological challenges: one in image-based diagnostic cytology, the other in mass cytometry-based cell population discovery. Both are architected for robust generalization across heterogeneity in biological specimens, and both introduce rigorous probabilistic or self-supervised learning frameworks for extracting interpretable, transferable features from complex, high-dimensional data.

## 1. Model Architectures and Pre-training/Inferences

### CytoFM: Vision Transformer (ViT) Self-supervised Foundation Model

CytoFM is built on a ViT-Base (B/16) backbone with 12 Transformer encoder layers, an embedding dimension of 768, and 12 attention heads. Input data consists of $256 \times 256$ pixel cytology patches at $40\times$ magnification. The pre-training objective is based on the iBOT self-supervised teacher–student framework, which combines Masked Image Modeling (MIM) and cross-view self-distillation:

- **Masked Image Modeling**: For input patch $x$, a masking operator $M(\cdot)$ occludes part of the input before encoding with the student encoder $R(\cdot)$ and decoder $D(\cdot)$. The MIM loss, a mean squared error, is
  $$
  \mathcal{L}_{\rm MIM} = \mathbb{E}_{x} \left\| D\left(R(M(x))\right) - \text{TeacherEmb}(x) \right\|_2^2,
  $$
  where $\text{TeacherEmb}(x)$ represents patch embeddings from the teacher network given an unmasked view.

- **Self-distillation**: For two augmented views $u$ and $v$ of an image, the distillation loss (cross-entropy on [CLS] tokens) is
  $$
  \mathcal{L}_{\rm distill} = -\sum_i p_{\rm teacher,i}(u) \log p_{\rm student,i}(v) - \sum_i p_{\rm teacher,i}(v) \log p_{\rm student,i}(u).
  $$

- **Total Objective**:
  $$
  \mathcal{L}_{\rm pretrain} = \lambda_{1} \mathcal{L}_{\rm MIM} + \lambda_{2} \mathcal{L}_{\rm distill}.
  $$

The model is initialized from iBOT-ImageNet weights and trained on cytology patches until convergence. After pre-training, the teacher model is frozen for downstream tasks.

### CytoFM: Bayesian Feature Allocation Model (FAM) for Cytometry

This approach models $I$ cytometry samples, each with $N_i$ cells and $J$ markers per cell. The hierarchy is:

- For each cell $n$ in sample $i$, and marker $j$, observe (possibly missing) normalized expression $y_{i,n,j}$ and define a missing-data indicator $m_{i,n,j}$.
- Introduce a latent subpopulation indicator $\lambda_{i,n} \in \{0,1,\dots,K\}$ (0=noise).
- Marker expression conditional on subpopulation is Gaussian:
  $$
  y_{i,n,j} | \lambda_{i,n},\mu_{i,n,j},\sigma_i^2 \sim N(\mu_{i,n,j}, \sigma_i^2)
  $$
- The expression pattern for each subpopulation is encoded in a $J \times K$ binary matrix $Z$, governed by a finite Beta–Bernoulli/IBP prior:
  $$
  v_k \sim \mathrm{Beta}(\alpha/K, 1), \qquad z_{j,k}|v_k \sim \mathrm{Bernoulli}(v_k)
  $$
- Marker means $\mu_{i,n,j}$ are drawn from truncated Gaussian mixtures, with ordering constraints to model expression or silence.
- Missing data are modeled as a function of intensity via
  $$
  \rho_{i,n,j}(y) = \operatorname{logit}^{-1}(\beta_{0,i} + \beta_{1,i} y + \beta_{2,i} y^2)
  $$
- Posterior inference is conducted using Gibbs–Metropolis MCMC, or optionally, mean-field variational inference (ADVI).

## 2. Pre-training and Data Sources

### CytoFM (ViT): Multiorgan, Multicenter Cytology Corpus

- Approximately 1.4 million $256 \times 256$ patches from eight cytology datasets spanning three organs (breast, cervix, thyroid).
- Datasets include: FNAC2019, MLBC, SiPaKMeD, BMT, APACS23, CCEDD, Bialystok cervical cytology, and a private ThyUCLA set filtered using a fine-tuned VGG-16.
- All patches are non-overlapping; no stain normalization applied. Augmentation includes random cropping, color jitter, Gaussian blur, solarization, and blockwise masking (~40% of tokens).

### CytoFM (FAM): CyTOF Natural Killer Cell Data

- Data comprise three umbilical-cord-blood donors, each with $J=20$ measured markers and up to $N_1 \approx 38,636$ cells.
- Model selection using LPML and DIC was performed over $K \in \{3, 6, \dots, 33\}$, yielding $\hat K = 21$.

## 3. Downstream Tasks and Clustering Procedures

### CytoFM (ViT): Attention-based Multiple Instance Learning (ABMIL)

- Frozen ViT embeddings are pooled across slide/image using a learnable attention head; embedding aggregation is weighted by per-patch attention scores:
  $$
  \alpha_i = \frac{\exp(w^\top h_i)}{\sum_j \exp(w^\top h_j)}
  $$
  where $h_i$ are patch embeddings and $w$ is a learnable parameter.
- The aggregate (slide-level) embedding is input to a linear classifier, with:
  - Sigmoid/BCE loss for binary malignancy (FNAC2019, breast)
  - Softmax/categorical cross-entropy for multi-class cell types (MLBC, HiCervix)
- Fine-tuning is performed with identical attention head hyperparameters for all model extractors.

### CytoFM (FAM): Bayesian Cell Subpopulation Discovery

- Posterior samples $\lambda_{i,n}^{(b)}$ provide the full assignment distribution; MAP estimates are derived by maximizing posterior cell-cluster probabilities.
- For feature-summaries, a pairwise allocation matrix of marker co-expression within each subpopulation is constructed and matched across posterior samples via Frobenius norm minimization.

## 4. Performance Evaluation and Benchmarks

### CytoFM (ViT): Classification, Cell Typing

| Model / Task        | FNAC2019 (Acc / AUC)   | MLBC (Acc / AUC)          | HiCervix (Acc / AUC)        |
|---------------------|------------------------|---------------------------|-----------------------------|
| iBOT-ImageNet       | 0.946 ± 0.05 / 0.991 ± 0.01 | 0.879 ± 0.06 / 0.983 ± 0.01 | 0.803 / 0.956               |
| UNI (histopathology)| 0.927 ± 0.06 / 0.983 ± 0.02 | 0.895 ± 0.06 / 0.986 ± 0.01 | 0.800 / 0.952               |
| CytoFM              | 0.908 ± 0.06 / 0.979 ± 0.02 | **0.930 ± 0.05 / 0.993 ± 0.01** | **0.844 / 0.968**           |

- CytoFM demonstrates statistically significant improvements on the MLBC task against both iBOT-ImageNet and UNI (p < 0.001). On HiCervix (unseen organ), CytoFM leads by ∼4 points in accuracy and 0.01 in AUROC.
- FNAC2019 performance is slightly lower than iBOT-ImageNet but still achieves >90% accuracy.
- UMAP visualizations show CytoFM embeddings form distinct, tighter class clusters compared to alternatives.

### CytoFM (FAM): Data-driven Cell Populations

- The model identifies 21 subpopulations in cord blood NK cells, including conserved "mature" and "immature" clusters and memory-like populations, based on combinatorial marker expression.
- Comparative analysis with FlowSOM reveals that CytoFM's latent-feature approach yields more biologically interpretable, reproducible clusters, whereas FlowSOM merges distinct populations and gives no direct marker-pattern inference.
- The model explicitly accounts for non-ignorable missing data arising from CyTOF instrument artifacts, enhancing robustness for real-world cytometry.

## 5. Interpretability and Biological Insights

### CytoFM (ViT)

- Attention-map overlays reveal that ViT attends to nuclei, mitotic figures, nuclear boundaries, and cytoplasmic texture, capturing canonical cytological features.
- UMAP projections evidence segmentation of classes at the bag (WSI) level.

### CytoFM (FAM)

- Feature allocation matrix $Z$ enables interpretable marker–subpopulation signatures.
- The framework facilitates direct biological interpretation of cell types (e.g., EOMES and KIR marker combinations) and supports reproducible subpopulation definitions across independent donors.
- Fine-grained NK-cell annotation afforded by this model suggests potential for targeted ex vivo expansion of immune cell subsets in immuno-oncology.

## 6. Hyperparameters, Sensitivity, and Convergence

- CytoFM (ViT) fine-tuning uses an attention head size of 256, learning rate $= 10^{-4}$, Adam optimizer, batch size 16, and ∼20 epochs until validation loss plateau.
- CytoFM (FAM) employs $\alpha \sim \text{Gamma}(0.1,0.1)$, mixture truncation $L_0=L_1=5$, variance $\sim \text{IG}(3,2)$, and missingness links established by moment-matching at negative quantiles. MCMC convergence is monitored by traceplots of active features; sensitivity to missing-data mechanism was nominal.
- For both models, representation quality and cluster assignments were robust to minor changes in data preprocessing and hyperparameter choices.

## 7. Impact, Applications, and Implications

CytoFM (ViT) demonstrates the feasibility and benefits of cytology-specific self-supervised foundation models, outperforming or matching competing models on downstream WSI-level and cell-type prediction even with a relatively modest pre-training corpus. Its attention-based interpretability aligns with domain experts’ intuition regarding relevant cytopathological features [2504.13402].

CytoFM (FAM) advances the analysis of mass cytometry data by providing an interpretable, probabilistically principled cell-clustering strategy with direct marker-pattern inference and explicit missing-data modeling [2002.08609]. The discovery of biologically meaningful cell subsets, especially within NK cells, informs both fundamental immunobiology and translational protocols for cellular therapies.

A plausible implication is that such foundation models—whether vision- or feature allocation-based—will serve as templates for developing generalizable analytical frameworks across both digital pathology and single-cell -omics, with significant impact on diagnostic precision, annotation efficiency, and discovery of novel cell phenotypes.

Source: https://www.emergentmind.com/topics/cytofm