OCELOT 2023: Joint Cell–Tissue Modeling

• OCELOT 2023 Challenge is a benchmark that combines cell and tissue annotations to model interdependent semantics in histopathology.
• The challenge employs multi-scale fusion by integrating tissue segmentation on large patches with cell detection on small patches to improve diagnostic accuracy.
• Performance metrics such as mean F1 scores and Dice coefficients demonstrated that joint modeling significantly outperforms conventional cell-only approaches.

The OCELOT 2023 Challenge, formally the OCELOT 2023 “Cell Detection from Cell-Tissue Interaction” challenge, is a histopathology benchmark centered on explicit modeling of cell–tissue relationships for cell detection in whole-slide images. It was launched alongside the public release of OCELOT, “Overlapped Cell on Tissue,” a dataset designed to provide overlapping cell-level and tissue-level annotations on hematoxylin and eosin stained material from The Cancer Genome Atlas. The challenge operationalized a specific hypothesis: that models which learn the interdependent semantics between cellular morphology and tissue architecture can outperform conventional cell-only detectors, thereby better approximating the multi-scale diagnostic behavior of pathologists (Ryu et al., 2023, Shin et al., 11 Sep 2025).

1. Scientific motivation and problem formulation

OCELOT emerged from a gap in computational pathology between how pathologists examine slides and how many deep-learning cell detectors are trained. In routine practice, pathologists alternate between different magnifications, using low-magnification tissue architecture to contextualize high-magnification nuclear morphology. Existing deep-learning-based cell detectors, by contrast, typically operate on small, high-resolution crops and therefore lack broader morphological context that can be decisive for distinguishing tumor cells from benign stromal or inflammatory nuclei (Shin et al., 11 Sep 2025).

The benchmark was designed around the claim that this missing context is not merely auxiliary. The OCELOT dataset paper explicitly argues that accurate cell detection benefits from simultaneously understanding tissue-level structures and cell morphology, and it attributes the lack of such models largely to the lack of datasets with both cell and tissue annotations on overlapping regions (Ryu et al., 2023). The challenge therefore framed cell detection not as an isolated local recognition problem, but as a coupled multi-scale inference problem in which tissue semantics are expected to regularize cell classification.

A central implication of this design is that OCELOT is not simply a new annotation resource; it is a benchmark for testing whether explicit cell–tissue interaction modeling improves detection quality. The retrospective challenge analysis reports that the initiative was intended both to validate that hypothesis and to accelerate research on multi-scale semantics in histopathology (Shin et al., 11 Sep 2025).

2. Dataset construction and annotation regime

The underlying OCELOT dataset comprises H&E-stained whole-slide images drawn from six organ sites within The Cancer Genome Atlas. From these slides, roughly 50,000 non-overlapping $512\times512$-pixel patches at $0.2\,\mu\text{m/pixel}$ were extracted, balancing six tumor types and yielding more than twice the number of patches in the earlier TIGER TILs dataset (Ryu et al., 2023). The challenge-specific curation described later consists of 673 spatially paired patches sourced from 306 TCGA whole-slide images, divided into training, validation, and test subsets (Shin et al., 11 Sep 2025). This suggests two related but distinct views of OCELOT: a broad public dataset release and a paired-patch challenge benchmark.

The challenge patches were organized explicitly around scale overlap. For each pair, a large-field-of-view patch of $4096\times4096$ pixels at $0.2\,\mu\text{m/pixel}$, downsampled by 4 to $1024\times1024$ at $0.8\,\mu\text{m/pixel}$, was hand-segmented into Cancer Area (CA), Background stroma (BG), and an “Unknown” class for ambiguous regions. Fully contained within each large patch, a small-field-of-view patch of $1024\times1024$ at $0.2\,\mu\text{m/pixel}$ was independently annotated with point labels for every cell, categorized as Tumor Cell (TC) or Background Cell (BC) (Shin et al., 11 Sep 2025).

The annotation workflow was pathologist-intensive. At tissue level, board-certified pathologists manually traced the boundary of the cancerous area versus non-cancer tissue, and each tissue patch was annotated once. At cell level, three pathologists independently marked the center of every cell and labeled it as TC or BC; two pathologists annotated in parallel and a third reconciled discrepancies to form a consensus (Ryu et al., 2023). The challenge report further notes that each patch consumed on average 15 minutes of multi-rater pathologist time (Ryu et al., 2023).

The spatial overlap between tissue and cell annotations was nontrivial rather than deterministic. Tumor cells sit predominantly on cancer area, at approximately $92.6\%$, but not exclusively; approximately $7.4\%$ lie outside. This exception structure is important because it prevents the benchmark from collapsing into a rule-based relabeling problem and forces models to learn softer contextual associations (Ryu et al., 2023).

To prevent leakage, data splitting was performed at whole-slide-image level. In the original dataset description, patches were split into $0.2\,\mu\text{m/pixel}$0 train, $0.2\,\mu\text{m/pixel}$1 validation, and $0.2\,\mu\text{m/pixel}$2 test with cancer-type ratios held constant (Ryu et al., 2023). In the challenge curation, the organizers performed a WSI-wise random split in a $0.2\,\mu\text{m/pixel}$3 ratio, yielding 400 training pairs, 137 validation pairs, and 130 test pairs (Shin et al., 11 Sep 2025). Standard pre-processing, including color normalization, random flips, rotations, and small color-jitter, was applied on the fly during training in the original study (Ryu et al., 2023).

3. Tasks and evaluation protocol

Participants were asked simultaneously to perform tissue segmentation on the large patches and cell detection with classification on the small patches. The key methodological feature is that the two tasks share spatial overlap, so tissue-level predictions can be propagated back into the cell branch during training or inference (Shin et al., 11 Sep 2025).

In the original OCELOT experiments, cell detection was cast as a two-class pixel-wise segmentation problem from which discrete cell centers are inferred, while tissue segmentation was treated as a region-based task (Ryu et al., 2023). The challenge retrospective formalized evaluation at the level of predicted cell centers. For each predicted cell, if its center fell within a radius of $0.2\,\mu\text{m/pixel}$4 of a ground-truth cell of the same class and that ground-truth cell had not already been matched, it counted as a True Positive; otherwise it was a False Positive. Unmatched ground-truth cells were counted as False Negatives (Shin et al., 11 Sep 2025).

Precision, recall, and the class-wise $0.2\,\mu\text{m/pixel}$5-score were defined as

$0.2\,\mu\text{m/pixel}$6

$0.2\,\mu\text{m/pixel}$7

The overall ranking metric was the mean $0.2\,\mu\text{m/pixel}$8 (mF1) across the two cell classes, TC and BC, and confidence intervals were computed via bootstrapping with 10,000 iterations (Shin et al., 11 Sep 2025). For tissue segmentation, the original study reports the Dice coefficient, equivalent to pixel-wise $0.2\,\mu\text{m/pixel}$9, and the Intersection-over-Union when appropriate (Ryu et al., 2023).

One technically relevant detail is that the two source accounts present slightly different matching descriptions: the dataset paper defines TP, FP, and FN using matched detections within a 3-pixel radius, whereas the later challenge report specifies a 3 $4096\times4096$0 radius and makes mF1 the explicit ranking metric (Ryu et al., 2023, Shin et al., 11 Sep 2025). For the challenge proper, the latter definition is the operational one.

4. Baselines, fusion architectures, and top submissions

The challenge and the accompanying OCELOT studies collectively established a progression from cell-only baselines to joint cell–tissue architectures. As a lower-bound reference in the retrospective challenge report, the organizers trained a “cell-only” baseline consisting of a DeepLabV3+ model with a ResNet-34 encoder pre-trained on ImageNet, trained exclusively on the small patches using a Dice loss, Adam optimizer, dropout $4096\times4096$1, weight decay $4096\times4096$2, and 300 epochs (Shin et al., 11 Sep 2025).

The original multi-task study reported that all top-ranked submissions in that setting shared a DeepLabV3+ backbone with a ResNet-50 encoder but differed in how cell and tissue tasks were combined (Ryu et al., 2023). Its winning method, Pred-to-Inter-2, processed a small field-of-view patch $4096\times4096$3 for cell detection and a larger “context” patch $4096\times4096$4 for tissue segmentation in parallel. After each shallow encoder block, intermediate feature maps from the tissue branch were up-sampled and injected into the cell branch through “inter-task fusion,” rather than relying on simple feature sharing or hard post-processing such as “TC-on-CA” (Ryu et al., 2023).

The loss in Pred-to-Inter-2 was a weighted sum of cell loss and tissue loss. The cell loss was the pixel-wise soft Dice plus cross-entropy loss for the two cell classes, while the tissue loss was cross-entropy for the binary tissue map. Validation-time grid search over $4096\times4096$5 selected $4096\times4096$6 and $4096\times4096$7. Models were trained for 60 epochs using Adam with initial learning rate $4096\times4096$8, polynomial decay, batch size $4096\times4096$9, and standard data augmentation (Ryu et al., 2023).

The later challenge analysis generalized this design space across 23 final Dockerized submissions and grouped the top methods into four broad innovation families (Shin et al., 11 Sep 2025):

• Cell label encoding: teams converted point annotations into fixed-radius disks, Gaussian heatmaps, NuClick-derived soft instance masks, or “repel” coding schemes.
• In-training fusion of tissue and cell branches: three of the top five teams integrated tissue context during training.
• Post-training heuristic refinement: two teams trained cell and tissue models separately and fused predictions by rule-based inference-time schemes.
• Ensembling and strong augmentations: all top five methods used heavy augmentation and test-time augmentation, and some further ensembled multiple models or folds.

The winning entry in the retrospective analysis, li2023enhancing, jointly trained a ViT-based U-Net (CellViT) on both small and large field-of-view inputs, element-wise adding tissue probability maps into intermediate cell feature maps and fine-tuning using LoRA on weights from the Segment-Anything Model. Millward2023dense and Schoenpflug2023softctm concatenated tissue segmentation probabilities directly with the cell-detection inputs. Zheng2023enhanced and lo2023enhancing instead used rule-based fusion at inference time, including adaptive and “extreme” weight-fusion schemes that re-weighted cell class probabilities based on tissue confidence (Shin et al., 11 Sep 2025).

5. Empirical performance and ablation findings

The central empirical result of OCELOT 2023 is that explicit cell–tissue interaction modeling improved cell detection substantially over cell-only baselines. In the retrospective challenge report, the hidden-test baseline achieved

$0.2\,\mu\text{m/pixel}$0

with precision $0.2\,\mu\text{m/pixel}$1 and recall $0.2\,\mu\text{m/pixel}$2 (Shin et al., 11 Sep 2025). The winning entry, li2023enhancing, achieved $0.2\,\mu\text{m/pixel}$3 with $0.2\,\mu\text{m/pixel}$4 confidence interval $0.2\,\mu\text{m/pixel}$5, precision $0.2\,\mu\text{m/pixel}$6, and recall $0.2\,\mu\text{m/pixel}$7. The second-place entry reached $0.2\,\mu\text{m/pixel}$8, and the third-place entry $0.2\,\mu\text{m/pixel}$9 (Shin et al., 11 Sep 2025).

The same retrospective account states that the top five teams outperformed the baseline by large margins, with mF1 improvements ranging from $1024\times1024$0 to $1024\times1024$1 percentage points, and also notes that the highest jump reported was $1024\times1024$2 percentage points (Shin et al., 11 Sep 2025). Regardless of the internal discrepancy in that summary, the broader conclusion is consistent: top challenge methods materially exceeded the cell-only reference.

The original OCELOT paper reported closely aligned gains in its own experimental framework. On the held-out OCELOT test set, Pred-to-Inter-2 achieved a tumor-cell $1024\times1024$3-score of $1024\times1024$4, versus $1024\times1024$5 for the best single-task baseline, an absolute gain of $1024\times1024$6. Even compared with MFoVCE-Net, which used a larger patch but no tissue labels and achieved $1024\times1024$7, Pred-to-Inter-2 improved performance by $1024\times1024$8. The hard TC-on-CA constraint yielded $1024\times1024$9, and paired $0.8\,\mu\text{m/pixel}$0-tests reported $0.8\,\mu\text{m/pixel}$1 for all key comparisons (Ryu et al., 2023).

External generalization was also reported. On the TIGER TILs dataset, performance improved from a prior state of the art $0.8\,\mu\text{m/pixel}$2 $0.8\,\mu\text{m/pixel}$3 to $0.8\,\mu\text{m/pixel}$4 $0.8\,\mu\text{m/pixel}$5. On the internal CARP immunohistochemistry-stained colon dataset, $0.8\,\mu\text{m/pixel}$6 increased from $0.8\,\mu\text{m/pixel}$7 to $0.8\,\mu\text{m/pixel}$8 (Ryu et al., 2023). These results indicate that the benefit of joint cell–tissue modeling was not confined to the challenge’s own test distribution.

Ablation results sharpened the interpretation. The original study found that multi-task fusion outperformed early-sharing and late-sharing strategies by $0.8\,\mu\text{m/pixel}$9–$1024\times1024$0, and that the larger-field-of-view tissue branch was essential for sharpening cell predictions near boundaries (Ryu et al., 2023). The challenge report likewise concluded that models fusing tissue context during training yielded higher $1024\times1024$1-scores and particularly sharper gains in precision than methods relying on post-hoc heuristics (Shin et al., 11 Sep 2025). It further observed that tissue context reduced false positives more substantially than false negatives, suggesting that multi-scale semantics mainly help the model confidently reject implausible cell–region pairings.

6. Interpretation, limitations, and future directions

OCELOT 2023 clarified several points that are easily obscured in simplified summaries of context-aware pathology AI. First, tissue context is informative but not deterministic. A recurrent misconception is that tumor-cell detection can be enforced by a hard TC-on-CA rule. OCELOT explicitly contradicts that view: approximately $1024\times1024$2 of tumor cells lie outside cancer area, and the hard TC-on-CA constraint underperformed learned fusion approaches (Ryu et al., 2023). The challenge analysis further reports weak performance on atypical cell–tissue pairings, including tumor cells in BG and background cells in CA, identifying these as an important clinical blind spot (Shin et al., 11 Sep 2025).

Second, larger field of view alone is not equivalent to explicit interaction modeling. The comparison with MFoVCE-Net showed that simply enlarging the crop without tissue labels remained inferior to joint cell–tissue learning (Ryu et al., 2023). The challenge results reinforce the same point: in-training fusion generally exceeded post-training heuristic refinement, particularly in precision (Shin et al., 11 Sep 2025). A plausible implication is that the key gain comes from learning shared semantics during optimization, not merely from exposing the model to more pixels.

Third, the challenge made annotation economics explicit. High-quality overlapping cell- and tissue-level annotations were costly, with each patch requiring on average 15 minutes of multi-rater pathologist time (Ryu et al., 2023). This cost structure motivates several future directions already identified by the organizers: expanding OCELOT to more cell phenotypes such as immune subtypes and stromal cells; exploring self-supervised pre-training on the large unannotated TCGA corpus; investigating semi-automated or active-learning protocols that focus pathologist effort on uncertain regions, including cells outside predicted CA; and hosting open leaderboards on additional external sets, including digital immunohistochemistry and multiplexed fluorescence (Ryu et al., 2023).

The retrospective challenge report extends that agenda toward more sophisticated multi-scale architectures, including graph-based or attention-guided fusion, as well as evaluation on low-quality or artifact-laden regions more typical of real-world clinical practice (Shin et al., 11 Sep 2025). It also points to downstream prognostic and predictive pipelines, such as analyzing the spatial arrangement of tumor-infiltrating lymphocytes. Taken together, these directions position OCELOT 2023 not only as a benchmark for cell detection, but as an early reference point for a broader research program in joint cell–tissue modeling for digital pathology (Ryu et al., 2023, Shin et al., 11 Sep 2025).