Tracking without bells and whistles (1903.05625v3)

Abstract: The problem of tracking multiple objects in a video sequence poses several challenging tasks. For tracking-by-detection, these include object re-identification, motion prediction and dealing with occlusions. We present a tracker (without bells and whistles) that accomplishes tracking without specifically targeting any of these tasks, in particular, we perform no training or optimization on tracking data. To this end, we exploit the bounding box regression of an object detector to predict the position of an object in the next frame, thereby converting a detector into a Tracktor. We demonstrate the potential of Tracktor and provide a new state-of-the-art on three multi-object tracking benchmarks by extending it with a straightforward re-identification and camera motion compensation. We then perform an analysis on the performance and failure cases of several state-of-the-art tracking methods in comparison to our Tracktor. Surprisingly, none of the dedicated tracking methods are considerably better in dealing with complex tracking scenarios, namely, small and occluded objects or missing detections. However, our approach tackles most of the easy tracking scenarios. Therefore, we motivate our approach as a new tracking paradigm and point out promising future research directions. Overall, Tracktor yields superior tracking performance than any current tracking method and our analysis exposes remaining and unsolved tracking challenges to inspire future research directions.

• The paper demonstrates that Tracktor repurposes an object detector’s bounding box regression to predict object positions without dedicated tracking training.
• It achieves competitive MOTA and IDF1 scores on MOT benchmarks by integrating minimal re-identification and camera motion compensation.
• The simplified, detector-centric tracking pipeline sets a new paradigm and opens avenues for future improvements in handling occlusions and challenging scenarios.

Tracking-by-Detection Using Tracktor Without Bells and Whistles

The paper presents a tracker, coined as Tracktor, tackling the problem of multi-object tracking in video sequences by employing object detection methods. The authors circumvent several traditional tracking-specific tasks such as object re-identification, motion prediction, and dealing with occlusions by leveraging the bounding box regression capabilities of an object detector. Notably, Tracktor operates without training or optimization on tracking datasets, instead repurposing the regressor of an object detector to predict the position of objects in subsequent frames. This novel approach effectively transforms a detector into a "Tracktor".

Methodology

The Tracktor methodology is based on a tracking-by-detection paradigm but simplified by using object detection methods to perform tracking:

1. Bounding Box Regression: Instead of generating new tracking-specific features, Tracktor directly employs the bounding box regressor of a detector for temporal realignment of object bounding boxes. The regressor adjusts the previous frame's bounding box coordinates to predict an object's position in the next frame.
2. Re-identification and Camera Motion Compensation: Tracktor is extended with minimalistic, yet effective re-identification and motion model components that bolster its performance on multi-object tracking benchmarks. The re-identification is achieved using a Siamese network for appearance matching, and the camera motion compensation aligns video frames via ECC maximization.

Experimental Results

Extensive experiments on the MOTChallenge benchmarks (MOT16, MOT17, and 2D MOT 2015) demonstrate Tracktor's efficacy:

1. MOTA and IDF1 Scores: Tracktor achieves state-of-the-art tracking performance across several metrics, particularly excelling in MOTA and identity preservation (IDF1). For instance, Tracktor++ yields a new state-of-the-art MOTA of 53.5% on MOT17, outperforming existing methods.
2. Evaluation on Different Datasets: The paper shows that Tracktor maintains robust performance across various datasets with different sets of public detections (DPM, Faster R-CNN, SDP).
3. Analysis of Tracking Challenges: The authors conduct an intricate analysis of how Tracktor deals with challenging tracking scenarios such as small and occluded objects. Results indicate that despite the seeming simplicity of their approach, Tracktor handles easy tracking scenarios efficiently, highlighting the potential of this method for broader applications.

Implications and Future Directions

The implications of utilizing Tracktor’s regression-based approach are multifaceted:

1. Simplified Tracking Pipeline: By converting the bounding box regression head of an object detector into a tracking mechanism, Tracktor simplifies the tracking pipeline, reducing reliance on complex tracking-specific training and heuristics.
2. Detector-Centric Tracking Paradigm: This work sets a precedent for a detector-centric tracking paradigm, suggesting that advancements in detection algorithms can directly enhance tracking performance.
3. Focus on Challenging Scenarios: Given the superior performance in straightforward tracking scenarios, future research should direct efforts toward improving tracking under challenging conditions like occlusions and crowded environments. Advanced motion models and sophisticated re-identification strategies represent promising avenues for future exploration.

Conclusion

The Tracktor's approach revisits and redefines the boundaries of tracking-by-detection. It verifies that modern object detectors, when smartly utilized, can achieve competitive tracking performance with reduced complexity. By exposing previously unresolved tracking challenges and suggesting future research directions, this paper encourages a shift in focus towards leveraging detectors for tracking, thus offering a streamlined, effective alternative to traditional techniques.