Acyclic Oriented Graph Matching (AOGM)

• Acyclic Oriented Graph Matching (AOGM) is a metric that quantifies the similarity between computed and reference cell-lineage graphs using weighted graph-edit operations.
• It measures tracking performance by evaluating both temporal associations and explicit mitosis (cell division) events through a system of penalties on node and edge discrepancies.
• Its application in cell tracking challenges underscores both its formal rigor and limitations, prompting refinements to better align metric scores with biologically meaningful division detection.

Acyclic Oriented Graph Matching (AOGM) is a graph-edit–distance-based evaluation metric originally developed as the de facto standard for benchmarking cell-tracking algorithms, especially in biomedical image analysis, where the task extends beyond standard multi-object tracking (MOT) to include explicit detection of mitosis (cell division) events. AOGM provides a quantitative measure of agreement between a predicted cell-lineage graph and a ground-truth reference, encoding not only the temporal association of cells across frames but also the hierarchical structure induced by cell divisions. Its adoption in cell-tracking challenges, such as the ISBI Cell Tracking Challenge, reflects its formal rigor and its alignment with the lineage-centric requirements unique to this domain (Chen et al., 2020).

1. Formal Definition and Graph Model

AOGM operates on directed, acyclic, time-oriented graphs. Let $G_R=(V_R,E_R)$ denote the reference (ground-truth) cell-lineage graph and $G_C=(V_C,E_C)$ the computed (predicted) cell tracking result. Each vertex $v \in V_t^i$ represents a single tracked cell $i$ at time $t$. Edges $(u \rightarrow v) \in E$ encode two possible semantic relationships: temporal-association edges (linking the same cell across consecutive frames $t \rightarrow t+1$), and mitosis (split) edges (from a mother cell at $t$ to each of its two daughters at $t+1$). By construction, both $G_R$ and $G_C=(V_C,E_C)$0 are forward-oriented acyclic graphs, reflecting the irreversibility of time and cell division events.

The AOGM score is defined as the minimum total weighted cost of editing $G_C=(V_C,E_C)$1 into $G_C=(V_C,E_C)$2 using a prescribed set of graph-edit operations, where each type of edit, including vertex and edge insertion/deletion and the semantic type of each edge, is assigned a distinct penalty weight.

2. Mathematical Formulation and Operation Types

The AOGM cost is formally given as:

$G_C=(V_C,E_C)$3

Here, $G_C=(V_C,E_C)$4 is the count of missed mitosis events (splits in $G_C=(V_C,E_C)$5 lacking two daughter edges in $G_C=(V_C,E_C)$6), $G_C=(V_C,E_C)$7 is the number of false-negative vertices (ground-truth detections unmatched in $G_C=(V_C,E_C)$8), and $G_C=(V_C,E_C)$9 represents false-positive vertices in $v \in V_t^i$0. Edge edits are categorized as $v \in V_t^i$1 (redundant edges in $v \in V_t^i$2 to be deleted), $v \in V_t^i$3 (missing edges in $v \in V_t^i$4 to be inserted), and $v \in V_t^i$5 (edges of incorrect semantic type, e.g., mitosis vs temporal). Each edit type has an associated penalty $v \in V_t^i$6.

AOGM can equivalently be viewed as the solution to an integer-linear program optimizing over these edit penalties, providing a granular quantification of tracking and lineage errors.

3. Canonical Weighting Schemes in Practice

The weights $v \in V_t^i$7 were established in Matula et al. (2015) and adopted by the ISBI Cell Tracking Challenge to reflect the qualitative importance of different error types. The canonical values, also employed in the experiments of Chen & Huo (SPIE 2021), are:

Error Type	Symbol	Weight
Missed splits	$v \in V_t^i$8	5
False negatives	$v \in V_t^i$9	10
False positives	$i$0	1
Edge deletions	$i$1	1
Edge insertions	$i$2	1.5
Edge mislabels	$i$3	1.5

These settings aim to balance biological significance (e.g., missed mitoses vs. spurious cells) and discourage fragmentation of tracks, but their adequacy for mitosis evaluation is not axiomatic.

4. Experimental Evidence and AOGM Pathologies

Chen & Huo (SPIE 2021) report two experiments—one on synthetic data, one on real cell microscopy sequences—demonstrating a key pathology in AOGM's behavior regarding mitosis events (Chen et al., 2020):

• Simulation Study: Starting from a single progenitor cell, a ground-truth tree with seven mitoses (eight terminal lineages) is constructed. Two predicted graphs are evaluated: one that entirely omits mitosis edges and one that perfectly links mother-daughter pairs. Despite identical detected vertices (equivalent $i$4, $i$5, $i$6), the correct-mitosis graph incurs higher $i$7 and $i$8 penalties, yielding an AOGM of 133.5—worse than the mitosis-ignorant version.
• Real Data (Fluo-N2DH-GOWT1-01): Using FairMOT, tracking is performed without mitosis, then post-processed to add mother-daughter edges for true mitoses. In each division event, the AOGM score increases after correct linkage, as the additions trigger higher edge-based penalties that outweigh credit for correct splits.

In both scenarios, adding true mitosis edges—a biologically crucial success—results in a numerically worse (larger) AOGM, indicating a potential contradiction between AOGM optimization and biologically meaningful division detection.

5. Analysis of Limitations and Failure Modes

Two principal limitations of AOGM for mitosis evaluation are identified in Chen & Huo (Chen et al., 2020):

• Under-counting of Mitoses: AOGM's strict demand for topological (structural and temporal) congruence between $i$9 and $t$0 means that even minor timing errors (e.g., a split detected one frame early / late) incur both a missing-split penalty and multiple edge penalties, overwhelming the biological relevance of detecting the division.
• Conflation of Edge-Type Errors: Each semantic edge error (mislabeling, missing, redundant) is independently penalized; thus, introducing a correct mitosis can inadvertently necessitate additional insertions/deletions elsewhere, cascading the cost and paradoxically discouraging proper mitosis linkage.

These effects can cause AOGM to penalize accurate lineage capture more severely than total omission—contravening the intended metric behavior.

6. Recommendations for Metric Refinement and Alternatives

To mitigate these deficiencies, two primary remedies are suggested:

• Recalibration of AOGM Weights: Adjust the penalty weights to better align with biological priorities (e.g., increase $t$1 for missed splits, decrease $t$2/$t$3 for edge errors) so that correct detection of mitosis is rewarded suitably, even when edge errors occur.
• Direct Mitosis-Focused Metrics: Supplement or replace AOGM with measures directly targeting mitosis detection, such as event-level precision/recall or a split-specific F1 score that explicitly assesses the accuracy of mother-daughter linkages. This approach follows recommendations by Powers (2020) and isolates lineage accuracy from generic tracking errors.

A plausible implication is that, until such adjustments are universally adopted, benchmarks should report dedicated mitosis metrics alongside global AOGM scores to avoid under-representing biologically meaningful tracker performance.

7. Context and Application in Cell Tracking Evaluation

AOGM's role as a standard comparative metric in the ISBI Cell Tracking Challenge and related literature highlights both its community acceptance and its influence in shaping algorithm design and evaluation. However, the pathologies demonstrated by Chen & Huo underscore the necessity for metric transparency and adaptation as task complexity or biological requirements evolve. In domains where lineage events are of paramount importance, reliance on unmodified AOGM may obscure true algorithmic advances, especially in mitosis detection and accurate cell-tree reconstruction (Chen et al., 2020). Future benchmarking frameworks may increasingly incorporate hybrid or explicitly lineage-aware metrics, balancing the strengths of AOGM with measures tuned to the biological events underlying cell population dynamics.