Tracking-Based Metrics

• Tracking-based metrics are quantitative methods comparing ground-truth and computed trajectories to assess identity maintenance and object association over time.
• They consolidate errors like false positives, false negatives, and track switches into a unified evaluation scheme, improving performance interpretation.
• Recent advancements offer parameterizable metrics and optimization schemes that dissect localization, association, and classification errors for robust multi-domain tracking evaluation.

Tracking-based metrics provide rigorous, quantitative methods for evaluating the performance of tracking algorithms in multi-object, multi-camera, and multi-target scenarios. Unlike per-frame detection metrics, tracking-based metrics explicitly assess the ability of a system to maintain the correct identities of objects or targets across time, space, and possibly differing sensor modalities. These metrics are essential for applications in video surveillance, autonomous driving, biomedical imaging, radar networks, and human-computer interaction, where robust identity preservation and the correct association of objects over time are critical.

1. Formal Definitions and General Principles

Tracking-based metrics quantify performance by comparing sets of ground-truth trajectories to computed trajectories under some correspondence or matching, often formalized as a multi-dimensional assignment problem. This comparison treats every frame-wise detection as belonging to one of three categories: true positive (TP; correctly assigned identity), false positive (FP; hallucinated instance), or false negative (FN; missed instance). The matching is typically computed globally to optimize a cumulative cost that combines localization error, missed targets, false targets, and track switches (Ristani et al., 2016, García-Fernández et al., 2016).

Mathematically, for ground-truth trajectories $AT$ and computed trajectories $AC$, together with an association threshold $\Delta$, identity-based counts are defined:

• Identity true positives: $$IDTP = \sum_{(\tau,\gamma)\in TPID} cov(\tau,\Delta)$$
• Identity false negatives: $$IDFN = \sum_{\tau\in AT}(\mathrm{len}(\tau) - cov(\tau,\Delta))$$
• Identity false positives: $$IDFP = \sum_{\gamma\in AC}(\mathrm{len}(\gamma) - cov(\gamma,\Delta))$$

With these, precision, recall, and F1 expressions are as follows: $$IDP = \frac{IDTP}{IDTP + IDFP}, \quad IDR = \frac{IDTP}{IDTP + IDFN}, \quad IDF_1 = \frac{2\,IDTP}{2\,IDTP + IDFP + IDFN}$$ (Ristani et al., 2016)

For trajectory-set metrics, additional cost terms capture localization error, missed/false targets, and switches over time steps $k=1,\dots,T$: $$d_{p,c,\gamma}(X, Y) = \min_{\pi^1, \dots, \pi^T} \Biggl( \sum_{k=1}^T d_{X,Y}^k(X,Y,\pi^k)^p + \sum_{k=1}^{T-1} s_{X,Y}(\pi^k,\pi^{k+1})^p \Biggr)^{1/p}$$ (García-Fernández et al., 2016)

2. Error Types and Treatment in Tracking-Based Metrics

Tracking-based metrics characteristically "flatten" all frame-level errors into a unified scheme, eliminating the need for special bookkeeping of identity switches, track fragmentation, or merges. All errors are either false negatives (missed frames/IDs) or false positives (spurious frames/IDs). This approach avoids the ambiguities and inconsistencies of defining, e.g., what constitutes an "identity switch," and focuses evaluation on the total number of frames for which identity is preserved (Ristani et al., 2016).

For metrics that include explicit switch penalties (e.g., TGOSPA),

$$S(\pi^{0:K}) = \sum_{k=0}^{K-1}\sum_{i=1}^{|\mathbf{X}|} s(\pi_i^k, \pi_i^{k+1})$$

where $AC$0 assigns full or half penalties based on assignment changes or losses (Krejčí et al., 2024). This biasing enables detailed analysis of the sources of error: spatial drift, mismatches, track swaps, or fragmentation.

3. Comparison with Classical Detection-Based and MOT Metrics

Traditional metrics such as CLEAR MOT (MOTA, MOTP) (Ristani et al., 2016), which explicitly include counts of identity switches (IDSW), summarize detection and identification performance in a single scalar. However, they can mask poor identity recovery if detection performance is strong, or result in inconsistent error weighting between applications. For example,

$AC$1

In contrast, tracking-based (or identity-based) metrics like $AC$2 penalize switches implicitly (i.e., a single ID switch creates an extra FN and FP at the switch point), leading to measures that are more interpretable and better correlate with subjective identity-purity (Ristani et al., 2016, Iatariene et al., 11 Jun 2025).

The trend in recent tracking evaluation is to further disentangle detection, localization, association, and classification—culminating in multi-factorial metrics such as TETA, which independently measures localization, association, and classification components (Li et al., 2022).

4. Metric Variants and Recent Developments

Numerous variants have been introduced to address domain- or application-specific requirements:

• Trajectory/Set-based Metrics: Generalize assignment-based costs over trajectories, accommodating localization errors, missed/false targets, and switches at each time step (García-Fernández et al., 2016, García-Fernández et al., 2021).
• Parameterizable Metrics (TGOSPA, T-GOSPA): Employ user-tunable penalties for localization, cardinality, and switches, with cut-off, order, and switch-penalty parameters. These are true metrics (or quasi-metrics in the asymmetric cost case), possessing the triangle inequality and decomposability (Krejčí et al., 2024, García-Fernández et al., 18 Jul 2025).
• KL-divergence–Inspired Metrics: Continuous spatio-temporal divergences that remain sensitive to merges, splits, and overlaps, free from arbitrary IoU cutoffs (Adams, 2018).
• Robustness/Experiment-Aware Metrics (EATM, RM): Incorporate experiment parameters (e.g., imaging interval, maximum cell count) to expose how data acquisition design impacts tracker robustness (Seiffarth et al., 2024).
• Probability-Based Metrics: In radar tracking, multi-pulse tracking probability $AC$3 is the joint probability of successive successful target detection, reflecting end-to-end reliability in communication or sensing (Ghatak, 2024).
• Temporally Local Metrics (ALTA): Restrict assignment cost windows to finite temporal horizons, allowing precise tradeoffs between detection and association, and discrimination of short- vs. long-term identity performance (Valmadre et al., 2021).
• Application-Specific Adaptations: In acoustic speaker tracking, modified association metrics (HOTA-style) directly measure identity continuity over intermittent, spatially discontinuous tracks—critical for diarization and downstream audio processing (Iatariene et al., 11 Jun 2025).

5. Computational Schemes and Optimization

Computation of tracking-based metrics generally consists of:

• Constructing pairwise assignment cost matrices (e.g., using IoU, spatial distance, or other domain-specific measures)
• Solving a global minimum-cost matching, often via algorithms such as the Hungarian method or multi-dimensional assignment (MDA) solvers, possibly with linear programming relaxations for efficiency (García-Fernández et al., 2016, García-Fernández et al., 2021)
• Aggregating metric-relevant counts (TP, FP, FN, switches) as specified by the metric's decomposition

For time-weighted or multi-scenario evaluation, metrics may use arbitrary per-frame weights or average over random finite sets of trajectories to yield robust, scenario-spanning comparisons (García-Fernández et al., 2021).

The table below illustrates selected metric categories and their treatment of errors:

Metric Type	Error Decomposition	Customization
ID-based F1	FN/FP only (switches→FN/FP)	Threshold Δ
Trajectory-set (MDA)	Loc, miss, false, switches (via γ, c, p)	(c, γ, p), time-weighting
TGOSPA/T-GOSPA	Loc, miss, false, switches	Cutoff, order, switch penalty
KL-inspired	Merge/split, miss, false, overlaps	Fully parametric
Robustness aware (EATM)	Performance as function of Δt, Nmax	Experimental design
Association metrics	Precision, recall, and accuracy per matched pair, track-level	Angular/IoU threshold θₘₐₓ

6. Impact and Experimental Evidence

Empirical studies demonstrate that ranking trackers by identity-based metrics (e.g., $AC$4) can yield substantial reordering of state-of-the-art methods compared to MOTA, often reflecting subjective perceptions of identity stability more faithfully (Ristani et al., 2016). For instance, on the DukeMTMC multi-camera benchmark, differences of 15–20 percentage points in $AC$5 were observed between trackers with similar MOTA, indicating core weaknesses in identity recovery.

Further, ablation and benchmarking on realistic datasets show that experiment-aware tracking metrics (EATM, RM) reveal rapid degradation in cell tracking performance as imaging intervals increase or as colony size grows, which is invisible to traditional metrics (Seiffarth et al., 2024). Similarly, in automotive radar, the optimal MAC parameter for maximizing multi-pulse tracking probability differs from that maximizing single-shot detection, directly impacting system-level latency/reliability (Ghatak, 2024).

Association metrics tailored to discontinuous tracks, as in acoustic diarization, can expose identity continuity losses that are entirely masked by frame-level MOTA or switch counts (Iatariene et al., 11 Jun 2025).

7. Practical Recommendations and Future Directions

Best practices for tracking-based metric use include:

• Selecting metric parameters (assignment cutoff, switch penalty) appropriate to the application's tolerance for identity errors and the typical spatial/temporal error scale (Krejčí et al., 2024)
• Decomposing overall scores into sub-metrics (localization, association, switch, miss/false) to diagnose algorithmic weaknesses (Krejčí et al., 2024, García-Fernández et al., 18 Jul 2025)
• Visualizing metric variations over key acquisition/experiment parameters to inform system or experiment design (Seiffarth et al., 2024)
• In multi-class and long-tail settings, reporting decomposed localization, association, and classification scores as in TETA to prevent confounded results and enable algorithm-specific comparisons (Li et al., 2022)

Further theoretical developments include more efficient algorithms for high-dimensional or long-horizon assignments, LP relaxations with tightness guarantees, and generalized metrics for complex object properties (e.g., non-symmetric localization costs, time-weighted penalties) (García-Fernández et al., 2021, García-Fernández et al., 18 Jul 2025).

Tracking-based metrics represent the core quantitative foundation for rigorous evaluation of tracking systems in contemporary and future, multidisciplinary settings. This encompasses robust scenario-aware evaluation, dynamically parameterized error weighting, and multi-modal/multi-class identification, ensuring that research and application outcomes remain interpretable, reproducible, and relevant across evolving domains.