StrongSORT: Make DeepSORT Great Again (2202.13514v2)

Abstract: Recently, Multi-Object Tracking (MOT) has attracted rising attention, and accordingly, remarkable progresses have been achieved. However, the existing methods tend to use various basic models (e.g, detector and embedding model), and different training or inference tricks, etc. As a result, the construction of a good baseline for a fair comparison is essential. In this paper, a classic tracker, i.e., DeepSORT, is first revisited, and then is significantly improved from multiple perspectives such as object detection, feature embedding, and trajectory association. The proposed tracker, named StrongSORT, contributes a strong and fair baseline for the MOT community. Moreover, two lightweight and plug-and-play algorithms are proposed to address two inherent "missing" problems of MOT: missing association and missing detection. Specifically, unlike most methods, which associate short tracklets into complete trajectories at high computation complexity, we propose an appearance-free link model (AFLink) to perform global association without appearance information, and achieve a good balance between speed and accuracy. Furthermore, we propose a Gaussian-smoothed interpolation (GSI) based on Gaussian process regression to relieve the missing detection. AFLink and GSI can be easily plugged into various trackers with a negligible extra computational cost (1.7 ms and 7.1 ms per image, respectively, on MOT17). Finally, by fusing StrongSORT with AFLink and GSI, the final tracker (StrongSORT++) achieves state-of-the-art results on multiple public benchmarks, i.e., MOT17, MOT20, DanceTrack and KITTI. Codes are available at https://github.com/dyhBUPT/StrongSORT and https://github.com/open-mmlab/mmtracking.

• The paper presents significant enhancements to the classical DeepSORT tracker, improving multi-object tracking accuracy and reliability.
• It integrates state-of-the-art detection (YOLOX-X) and feature extraction (BoT) modules along with EMA and adaptive filtering to mitigate detection noise.
• Experimental results on MOT benchmarks show higher IDF1 and HOTA scores, demonstrating superior performance in complex tracking scenarios.

Insights into "StrongSORT: Make DeepSORT Great Again"

The paper "StrongSORT: Make DeepSORT Great Again" presents significant advancements in the domain of Multi-Object Tracking (MOT) by revisiting and enhancing the classical DeepSORT tracker. The researchers have proposed modifications that aim to establish a robust and fair baseline in the MOT community through StrongSORT, including improved components and novel algorithms addressing inherent MOT challenges.

Core Contributions

StrongSORT builds on DeepSORT by integrating state-of-the-art modules and techniques:

1. Advanced Modules: The paper replaces less competitive components of DeepSORT with contemporary, high-performance alternatives such as YOLOX-X for detection and BoT for feature extraction. These improvements directly enhance object detection and re-identification capabilities, allowing for better tracking accuracy and efficiency.
2. Enhanced Mechanisms: The incorporation of exponential moving average (EMA) for feature updating, a camera motion compensation (ECC) model, and an adaptive noise-scaled Kalman filter (NSA Kalman) strategically fortify the tracking model against various environmental and detection noises, leading to a more stable and reliable MOT system.
3. Innovative Algorithms: To address common MOT issues—namely, missing associations and missing detections—two plug-and-play algorithms are introduced:
   • AFLink: A lightweight, appearance-free link model that leverages spatiotemporal information for efficiently associating tracklets without complex appearance models.
   • GSI (Gaussian-smoothed interpolation): This algorithm utilizes Gaussian process regression to seamlessly address missing detections through refined interpolation, enhancing both speed estimation and trajectory stability.

Experimental Outcomes

The series of experiments conducted demonstrate that StrongSORT and its enhanced iteration, StrongSORT++, deliver exceptional performance across several public benchmarks including MOT17, MOT20, DanceTrack, and KITTI. Key results revealed include:

• Superior Performance Metrics: StrongSORT++ notably exceeds previous benchmarks in IDF1 and HOTA scores, indicating both robust identification of objects across frames and high localization accuracy.
• Efficiency in Complex Scenarios: The paper highlights that StrongSORT++ achieves strong performance even amidst challenging conditions such as high occlusion and camera movement, particularly showcased in the MOT20 evaluations.

Implications and Future Directions

From a theoretical standpoint, this research reinforces the prominence and viability of the tracking-by-detection paradigm while emphasizing the need for advanced adaptive features and modular algorithms. Practically, this work holds significant potential implications for real-time surveillance systems, autonomous vehicles, and other domains requiring accurate multi-object tracking.

The introduction of lightweight algorithms such as AFLink and GSI provides insights into creating trackers that are both computationally efficient and less reliant on appearance-based information, making them ideal for crowded or visually ambiguous scenes.

For future explorations, the paper suggests potential enhancements in global link strategies to address lingering challenges such as false associations and trajectory divisions. Additionally, fine-tuning detection thresholds dynamically could lead to further improvements in detection precision.

In conclusion, "StrongSORT: Make DeepSORT Great Again" presents substantial refinements and methodologies that advance the field of MOT, setting a new benchmark for both academic and applied research. The focus on balanced performance between detection accuracy and association correctness illustrates a decisive step forward in the evolution of efficient and effective object tracking systems.