---
title: MIDOG Competition Series
url: https://www.emergentmind.com/topics/midog-competition-series
type: topic
---

# MIDOG Competition Series

The MIDOG Competition Series—**MItosis DOmain Generalization**—is a sequence of benchmark challenges for mitotic figure analysis in histopathology, organized around the central problem of **robust generalization under domain shift**. Across its major editions, the series evolved from scanner-focused breast cancer mitosis detection to broader evaluation across tumor types, laboratories, species, and tissue contexts, and in 2025 added a second track for **atypical mitotic figure** classification. Its clinical motivation is consistent throughout: mitotic activity is a key marker of tumor proliferation, malignancy, grading, and prognosis, yet both manual assessment and algorithmic performance are limited by observer variability and distribution shift [2204.03742][2309.15589][2606.07368].

## 1. Clinical rationale and conceptual scope

MIDOG was created to study a specific failure mode of computational pathology systems: strong performance on training-like material followed by deterioration when the acquisition or biological domain changes. In the 2021 challenge paper, the decisive component of domain shift was identified as **variability caused by using different whole slide scanners**; the 2022 overview broadened this to **tumor types, laboratories, species, staining/processing conditions, and scanners**; the 2025 overview added a further axis, namely **contextual diversity within the slide**, by evaluating not only hotspot regions but also random and challenging regions rich in hard negatives [2204.03742][2309.15589][2606.07368].

The series is anchored in mitotic figure recognition because mitotic count is clinically important but difficult. The 2021 overview states that recognition of mitotic figures by pathologists is subject to a strong inter-rater bias, while later MIDOG reports repeatedly emphasize that automated methods fail under scanner, stain, laboratory, species, and tumor-type shifts. This makes MIDOG less a generic object-detection benchmark than a domain-generalization benchmark for computational pathology [2204.03742][2509.02585].

A recurrent theme across the series is that mitosis analysis is not exhausted by localization. MIDOG 2025 formalized this by introducing **Track 2**, which distinguishes **normal mitotic figures (NMFs)** from **atypical mitotic figures (AMFs)**. That extension reflects the view, stated in multiple 2025 reports, that atypical mitoses are biologically and prognostically relevant, yet are rarer, more morphologically subtle, and more sensitive to domain shift than ordinary mitotic figure detection [2606.07368][2509.02601].

## 2. Evolution of the benchmark design

The series expanded in scope through three major stages.

| Edition | Benchmark design | Key reported outcome |
|---|---|---|
| MIDOG 2021 | Training set of **200 cases** across **four scanning systems**; additional **100 cases** in test, including **two previously unseen scanners** | Winning algorithm yielded **\(F_1 = 0.748\)**; Top5 ensemble reached **0.773** [2204.03742] |
| MIDOG 2022 | Released training data with **405 tumor cases** and **9,501 mitotic figure annotations** across **six domains**; final evaluation on **100 cases** across **10 independent tumor domains** [2309.15589] | Best performance **\(F_1 = 0.764\)** [2309.15589] |
| MIDOG 2025 | Final test set of **365 ROIs from 122 cases/patients**, spanning **12 tumor domains**; Track 1 added **hotspot, random, and challenging ROIs**; Track 2 introduced **AMF classification** [2606.07368] | Top detection **\(F_1 = 0.740\)**; top AMF classification **balanced accuracy = 0.908** [2606.07368] |

MIDOG 2021 was a **MICCAI 2021 challenge** centered on breast cancer ROIs under scanner-induced shift. The dataset contained **300 breast cancer cases** total, retrospectively curated from the University Medical Center Utrecht archive, with **200 cases** for training, **20** for preliminary test, and **80** for final test. The design used six scanners overall, including two scanners present only in testing, and extracted corresponding ROIs across scanners by registration so that the same biological tissue region was evaluated across devices [2204.03742].

MIDOG 2022 deliberately changed the meaning of “domain.” Its training set comprised six domains, including human breast carcinoma, canine lung carcinoma, canine lymphoma, canine cutaneous mast cell tumor, human pancreatic/gastrointestinal neuroendocrine tumor, and unlabeled human melanoma. The final test set contained **10 independent tumor domains**, introduced a **new species: feline**, and exposed models to new scanner/laboratory combinations and spindle-cell morphology. This was the edition in which MIDOG ceased to be primarily a scanner challenge and became a multi-axis domain-generalization challenge [2309.15589].

MIDOG 2025 is presented as the **third edition** of the series and as a substantial redesign rather than a simple continuation. The decisive change was moving beyond expert-selected hotspots to include **random tissue ROIs** and **challenging ROIs** enriched for hard negatives, artifacts, inflammation, and necrosis. The challenge also became explicitly two-track: **Track 1** for mitotic figure detection and **Track 2** for atypical mitotic figure classification [2509.02593][2606.07368].

## 3. Annotation protocols and evaluation criteria

MIDOG’s annotation protocols were designed around the observation that mitoses are sparse and easy to miss, and that expert disagreement is substantial. In MIDOG 2021, the organizers used **three expert pathologists** from different institutions and countries. One expert screened all ROIs and marked mitoses plus hard negatives; a RetinaNet was then trained to propose additional candidates; these candidates were re-reviewed in a **class-blinded** way; and disagreements were resolved by a third expert. The released labels included approximate **50 px** square boxes around mitoses and non-mitotic hard negatives [2204.03742].

The primary detection metric in MIDOG 2021 was micro-averaged \(F_1\), computed over all slides:
$$
F_1 = \frac{2 \sum_k^N \mathrm{TP}_k}{2 \sum_k^N \mathrm{TP}_k + \sum_k^N \mathrm{FN}_k + \sum_k^N \mathrm{FP}_k}.
$$
A predicted detection counted as a true positive if its centroid lay within **\(7.5\,\mu m\)** of a ground-truth mitosis. Multiple detections of the same object were counted as false positives. The 2021 paper also reported **10,000 bootstrap repetitions** with **5th and 95th percentiles** for confidence intervals [2204.03742].

MIDOG 2022 added a second reference standard beyond the official three-expert consensus: a **PHH3-assisted** ground truth. The organizers de-stained H&E slides, re-stained them with **phospho-histone H3 (PHH3)**, digitized the immunohistochemistry slides, aligned H&E and PHH3, and annotated with a superimposed view of both stains. This created an important benchmark-level distinction between **official ranking labels** and a post hoc reference intended to reduce reliance on subjective H&E-only judgments [2309.15589].

MIDOG 2025 continued PHH3-supported annotation for newly added detection ROIs and used the open-source **EXACT** annotation tool for co-registered H&E/PHH3 viewing. For Track 2, atypical versus normal labels were assigned by two pathologists, with disagreements resolved by a third expert. The paper reports **Cohen’s \(\kappa = 0.48\)** for Track 1 and **\(\kappa = 0.68\)** for Track 2. Track 1 evaluation used **\(F_1\)** as the primary metric, with **FROC-AUC** and **AP** as secondary metrics; Track 2 used **balanced accuracy** and **ROC AUC** [2606.07368].

## 4. Detection paradigms in the series

The detection track has never converged to a single dominant architecture. The 2021 and 2022 overviews describe a heterogeneous field including **single-stage detectors**, **multi-stage pipelines**, **semantic or instance segmentation**, **domain-adversarial training**, **stain normalization**, **style-transfer augmentation**, **Fourier-domain adaptation**, and **ensembling/test-time augmentation**. In 2021, the winning method was **AI medical** with **\(F_1 = 0.748\)** and the runner-up was **TIA Centre** with **\(F_1 = 0.747\)**; the organizers specifically noted that all three best-performing methods included an **auxiliary mitosis segmentation task**, and that ensembling or test-time augmentation was common among top submissions [2204.03742]. In 2022, the best overall performance came from **TIA Centre** with **\(F_1 = 0.764\)**, followed closely by **TCS Research** with **\(F_1 = 0.757\)**, despite the use of quite different pipelines [2309.15589].

Representative challenge reports illustrate the range of technical strategies. One MIDOG 2021 submission fused a **DetectoRS** detector with a **deep ensemble classification model**, using **Macenko stain normalization**, color-focused augmentation, and score fusion
$$
S_{final} = \alpha S_{DE} + (1-\alpha) S_{Dect},
$$
with optimal \(\alpha = 0.9\), and achieved **\(F_1 = 0.7550\)** on the preliminary testing set [2108.13983]. A MIDOG 2022 report showed that a single **EfficientNet-B3** classifier could be converted into a detector using **sliding-window inference** and **Grad-CAM++**, with positive windows thresholded at **0.84**, **NMS threshold 0.22**, and CAM hotspot centroids used as mitosis locations; this system reached **\(F_1 = 0.6847\)** on the final test phase of task 1 [2208.12437].

Another 2022 report emphasized **rotation invariance** and **fine-grained hard negative mining**. It used a customized **70-layer ResNet** with **P4-group convolutional layers**, dense whole-image inference, and a hard-negative retention search that ultimately trained the best model with only **19.6% of the overall pixel count of the training partition**, while still obtaining about **0.697** on the final test and **third place overall** [2301.01079]. A separate 2022 entry combined **YOLOv5s** with a **Radial-Prediction Domain Adaption Classifier (RP-DAC)** and **HED-space stain augmentation**, reaching **\(F_1 = 0.6658\)** on the official test set [2208.13902].

The 2025 detection literature broadened the architectural space further. A **YOLOv12-m** detector trained only on organizer-provided data reported **\(F_1 = 0.801\)**, **precision = 0.808**, and **recall = 0.794** on the preliminary test set, without external data or model ensembling [2509.02593]. An **RF-DETR** submission, strengthened by hard negative mining from necrotic tissue, achieved **\(F_1 = 0.789\)**, **recall = 0.839**, and **precision = 0.746** on the preliminary test set [2509.02599]. Team Westwood used **nnUNetV2** for high-sensitivity candidate screening, followed by a **random forest** over three CNNs, and reported **\(F_1 = 0.7450\)** on the preliminary test set [2509.02600]. A separate two-stage study combined improved **YOLO11x** proposals with **ConvNeXt-Tiny** filtering and, on a fused dataset of **MIDOG++**, **MITOS_WSI_CCMCT**, and **MITOS_WSI_CMC**, achieved **\(F_1 = 0.882\)**, which was **0.035 higher than the single-stage YOLO11x baseline** [2509.02627].

## 5. The introduction of atypical mitosis classification

MIDOG 2025 Track 2 formalized a new task: binary classification of mitotic crops into **NMFs** and **AMFs**. The official challenge overview presents this as a biologically motivated extension of the series and reports that a new public dataset of atypical versus normal mitoses was released for the track, built from **11,939 mitotic figures** spanning **7 domains** [2606.07368]. Challenge reports then expanded this corpus with additional public sources such as **AMi-Br**, **LUNG-MITO**, **GBM-TCGA**, **AtNorM-Br**, and **OMG-Octo Atypical**, typically to increase domain diversity and AMF prevalence [2509.02601][2509.02595].

Methodologically, Track 2 has been dominated by **foundation-model adaptation** and strong augmentation. One solution used the pathology foundation model **H-optimus-0** with **LoRA / PEFT**, soft labels from multi-expert consensus, hard negative mining, adaptive focal loss, supervised contrastive regularization, and a gradient-reversal domain head, reporting **mean balanced accuracy \(0.851 \pm 0.037\)** under **10-fold leave-one-domain-out cross-validation** [2509.02601]. Another strong baseline used **DINOv3-H+** pretrained on natural images, with LoRA applied only to query and value projections and about **650k trainable parameters**, reaching **balanced accuracy \(0.8871\)** on the preliminary test set [2508.21041].

Other competitive submissions explored lightweight regularization and augmentation recipes. The **“Mix, Align, Distil”** approach combined **MixStyle**, **CBAM-based feature alignment** using weak domain labels, and **EMA-teacher distillation**, attaining **balanced accuracy \(0.8762\)**, **sensitivity \(0.8873\)**, **specificity \(0.8651\)**, and **ROC AUC \(0.9499\)** on the organizer-run preliminary leaderboard [2508.20745]. A **ConvNeXt Small** model trained with a histopathology-specific augmentation pipeline and weighted sampling achieved **balanced accuracy \(0.8961\)** and **ROC AUC \(0.9561\)** on the preliminary leaderboard [2509.02595]. A **UNI2-h** study found that **Visual Prompt Tuning (VPT)** outperformed a LoRA baseline, and that adding **Vahadane** and **Macenko** stain-normalization TTA yielded **balanced accuracy \(0.8837\)** and **ROC-AUC \(0.9513\)** [2509.02640].

Track 2 did not settle the question of whether pathology foundation models are always superior. One MIDOG 2025 report states that the best result apparently came not from a foundation model, but from a **from-scratch ConvNeXt-Tiny** classifier, which the authors speculate may reflect dataset size limitations [2509.02585]. The official challenge overview reported a top Track 2 performance of **balanced accuracy \(0.908\)**, with the top three teams separated by less than one percentage point [2606.07368]. This suggests a benchmark in which strong solutions exist, but the margin between leading systems is narrow.

## 6. Recurrent findings, limitations, and scientific significance

Across editions, the series has produced several stable findings. First, **domain generalization is possible but incomplete**. MIDOG 2021 showed that scanner-agnostic detection could approach expert level, with the winning algorithm at **\(F_1 = 0.748\)** and the Top5 ensemble at **0.773** [2204.03742]. MIDOG 2022 showed that performance remained strong in a broader setting—top **\(F_1 = 0.764\)**—but also that **new species**, **spindle-cell morphology**, and a **new scanner/lab combination** caused small but significant decreases in performance [2309.15589]. MIDOG 2025 then demonstrated that contextual variation inside the slide is itself a major source of failure: mean \(F_1\) was **0.735** in hotspot ROIs, **0.638** in random ROIs, and **0.479** in challenging ROIs, while precision dropped from **0.805** in hotspots to **0.400** in challenging regions and false detections increased by approximately **208%** [2606.07368].

Second, the series repeatedly shows that annotation uncertainty is structurally important rather than incidental. In MIDOG 2022, re-evaluation against PHH3-assisted ground truth increased the total mitotic figure count by **15.0%**, and **28.78%** of PHH3-identified mitoses were not included in the original three-expert H&E majority vote. All methods then showed lower recall, although the ranking changed only minimally [2309.15589]. A plausible implication is that MIDOG benchmarks measure not only generalization capacity but also robustness to epistemic uncertainty in the reference standard.

Third, ensembling is one of the few interventions that generalized across editions. The 2025 overview reports mean gains of **1.5 percentage points in \(F_1\)** and **1.3 percentage points in balanced accuracy** from ensembling, whereas **TTA showed no relevant improvement** [2606.07368]. That finding is consistent with earlier editions, where many of the strongest methods used either explicit ensembles or multi-stage consensus mechanisms [2204.03742].

Several failure modes recur across challenge reports. One retrospective 2025 report documents a system with **recall 0.9528** but **precision 0.1267** on its best submission, concluding that the main problem was false positives from **morphologically similar imposters** across tissue domains; in the same study, adding a classifier ensemble to the detector reduced the full-pipeline \(F_1\) on PooledDB to **0.0646** because many true positives were also removed [2509.02630]. Another 2025 report isolates a **telophase double-detection problem**, in which one telophase event is predicted as two daughter-nucleus boxes, and resolves it with a distance-based aggregation step merging centroids within **10 \(\mu m\)** [2509.02585]. The CAM-based 2022 detector likewise notes a limitation when multiple mitoses fall inside one positive patch: the CAM may strongly highlight only one and underrepresent the others [2208.12437].

The broader significance of the MIDOG Competition Series is therefore methodological and epistemic as much as leaderboard-oriented. It established scanner shift as a benchmarkable problem, expanded domain generalization to tumor type, species, and laboratory variation, and then exposed a further blind spot by showing that hotspot performance does not reliably predict performance in random or imposter-rich tissue [2204.03742][2309.15589][2606.07368]. This suggests that MIDOG has become a benchmark not merely for mitosis detection, but for **real-world pathology robustness**.

Source: https://www.emergentmind.com/topics/midog-competition-series