Trajectory Metrics in Tracking

• Trajectory metrics are mathematical functions that measure dissimilarity between sets of trajectories by incorporating object localization, identity, and temporal dynamics.
• They benchmark multi-object tracking systems and support robotic evaluations and computer vision analysis by addressing limitations of earlier heuristic methods.
• Recent advances use convex relaxation and time-varying associations to ensure computational efficiency and adherence to true metric properties.

A trajectory metric is a mathematical function designed to quantify the distance (or dissimilarity) between two sets of trajectories, incorporating object localization, identity, and temporal evolution. Trajectory metrics play a central role in benchmarking multi-object tracking algorithms, system evaluation in robotics, and trajectory analysis in computer vision. Recent research identifies major limitations in widely adopted metrics (such as OSPA and the heuristic CLEAR MOT measures) and introduces new, mathematically consistent alternatives that are both practical and computationally efficient. The following sections detail the theory, construction, and implications of these developments.

1. Limitations of Existing Trajectory Metrics

Historically, two main families of trajectory comparison metrics have been used: OSPA (Optimal Subpattern Assignment, and its extension OSPA-ST), and heuristic measures built on frame-by-frame associations such as the CLEAR MOT family. OSPA and OSPA-ST [Schuhmacher et al. 2008] rely on a fixed, time-invariant assignment of trajectories. This approach enforces the same association throughout the entire temporal window, meaning that if trajectories cross or swap, the metric may "lock in" a poor association, resulting in an inflated or non-representative distance even if trackers perform well outside of the intersection region.

Heuristic measures like CLEAR MOT and its variants (MOTA, MOTP) attempt to surmount this by recalculating associations dynamically (anchored to previous frames, using a distance threshold thrₘₒₜ). However, this anchoring strategy introduces sequence-dependent bias: slight changes in when or how identity switches occur can induce large inconsistencies in the reported distance. Crucially, heuristic metrics do not satisfy the triangle inequality or even basic symmetry, so they do not constitute true mathematical metrics, limiting their theoretical and practical reliability.

2. Generalized Metric Construction: Optimal Time-Varying Association

A rigorous resolution to these challenges is the introduction of a metric that allows for time-varying associations—so that identities can "switch"—but incorporates a global penalty for switching, ensuring metric properties. The construction proceeds in two principal stages:

Step 1: Definition of Natural Distance (𝒟ₙₐₜ).

The metric considers all possible sequences of association mappings (Σ ∈ Πᵀ, with Π the set of permutation matrices for m trajectories over T timesteps). The overall cost for a pair of trajectory sets (A, B) is $$\mathcal{D}_\text{nat}(A, B) = \min_{\Sigma \in \Pi^T} \left\{ \mathcal{K}(\Sigma) + \sum_{t=1}^T \sum_{i=1}^m D_{i, \Sigma_i(t)}^{AB}(t) \right\}$$ where 𝒦(Σ) is a switching cost (e.g., proportional to the number of permutation changes over time), and D^{AB}_ij(t) is an extended pairwise state distance.

Step 2: Convex Relaxation for Tractability (𝒟_{comp}).

Solving the discrete optimization is NP-hard due to the combinatorial number of association sequences. The key innovation is to relax the time-varying association Σ(t) to a doubly stochastic matrix W(t) ∈ 𝒫 (the convex hull of permutations by the Birkhoff–von Neumann theorem), leading to a convex optimization problem: $$\mathcal{D}_{comp}(A, B) = \min_{W \in \mathcal{P}^T} \left\{ \sum_{t=1}^{T-1} \| W(t+1) - W(t) \| + \sum_{t=1}^T \text{tr}\left[ W(t)^\top D^{AB}(t) \right] \right\}$$ Choice of the norm (often 1-norm) and scaling in the switching term control the relative penalty for identity changes.

The state-pairwise distance d^+ is defined as:

• d^+(x, y) = min{2M, d(x, y)} for x, y ∈ ℝᵖ,
• d^+(x, ) = d^+(, x) = M (M: fixed missing-detection penalty),
• d^+(*, *) = 0.

3. Mathematical Consistency and Optimization

By enforcing two central mechanisms—the quantification of identity switch costs via a permutation measure 𝒦(Σ), and using a true metric for the state distance d (including the penalization for missed or extra targets)—the construction guarantees that both the discrete (𝒟ₙₐₜ) and the relaxed (𝒟_{comp}) forms satisfy all four metric axioms: non-negativity, zero only for identity, symmetry, and the triangle inequality. The metric structure remains robust as long as the switching cost and underlying distance norms themselves obey metric properties. The convex relaxation allows the metric to be computed by solving a linear program, providing scalability to large sets with polynomial complexity and access to established optimization solvers.

4. Salient Features and Formal Properties

The resulting trajectory metric achieves three defining properties:

Feature	Mechanism	Significance
Efficient computation	Convex/linear programming relaxation of assignment	Solvable in polynomial time for realistic problem sizes
Optimal identity confusion tradeoff	Switching cost 𝒦(Σ) controlled jointly with localization errors	Accurately balances accuracy vs. identity consistency
True metric (triangle inequality)	Global optimization and metric-constrained cost terms	Supports advanced use: clustering, retrieval, benchmarking

The metric avoids counterintuitive evaluations in the presence of identity switches—a crucial issue with fixed-association or strictly local assignment approaches.

5. Implementation, Use Cases, and Deployment Considerations

Concretely, implementation requires:

• Specification of the state distance (often Euclidean) and missing penalty M.
• Choice of switching penalty (e.g., number of switches, or more weighted counts).
• Assembly of distance matrices at each frame, possibly with padded "dummy" (* or unassigned) tracks.
• Solution of the convex optimization via a standard LP solver or efficient ADMM method (for large T and m).

These metrics are suitable for a range of applications:

• Benchmarking and comparing multi-object trackers, with unambiguous assignment of errors to localization vs. identity confusion.
• Enabling geometry- and identity-consistent clustering or nearest neighbor analyses in trajectory space.
• Providing robust means to define distances for evaluation, retrieval, and large-scale database indexing.

The tunable balance between localization accuracy and identity switching cost enables tailoring to specific domains or desired tracker properties.

6. Formula Reference and Theoretical Guarantees

The essential mathematical expressions are:

• Extended per-state distance:

$$d^+(x, y) = \begin{cases} \min\{2M, d(x, y)\} & x, y \in \mathbb{R}^p \ M & x = *,\, y \in \mathbb{R}^p \text{ or } x \in \mathbb{R}^p,\, y = * \ 0 & x = y = * \end{cases}$$

• Natural (combinatorial) trajectory metric:

$$\mathcal{D}_\text{nat}(A, B) = \min_{\Sigma \in \Pi^T} \left\{ \mathcal{K}(\Sigma) + \sum_{t=1}^T \sum_{i=1}^m D_{i, \Sigma_i(t)}^{AB}(t) \right\}$$

• Computable (convex/relaxed) metric:

$$\mathcal{D}_{comp}(A, B) = \min_{W \in \mathcal{P}^T} \left\{ \alpha \cdot \sum_{t=1}^{T-1} \| W(t+1) - W(t) \|_1 + \sum_{t=1}^T \text{tr}( W(t)^\top D^{AB}(t) ) \right\}$$

This approach ensures practical, interpretable, and mathematically sound evaluation of sets of trajectories. It addresses critical weaknesses of previous methods and enables rigorous performance comparison in multi-object tracking, with flexibility for application-specific requirements.