SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos (2204.06918v2)

Abstract: Tracking objects in soccer videos is extremely important to gather both player and team statistics, whether it is to estimate the total distance run, the ball possession or the team formation. Video processing can help automating the extraction of those information, without the need of any invasive sensor, hence applicable to any team on any stadium. Yet, the availability of datasets to train learnable models and benchmarks to evaluate methods on a common testbed is very limited. In this work, we propose a novel dataset for multiple object tracking composed of 200 sequences of 30s each, representative of challenging soccer scenarios, and a complete 45-minutes half-time for long-term tracking. The dataset is fully annotated with bounding boxes and tracklet IDs, enabling the training of MOT baselines in the soccer domain and a full benchmarking of those methods on our segregated challenge sets. Our analysis shows that multiple player, referee and ball tracking in soccer videos is far from being solved, with several improvement required in case of fast motion or in scenarios of severe occlusion.

• The paper introduces SoccerNet-Tracking, the largest publicly available dataset and benchmark specifically designed for Multiple Object Tracking in complex soccer videos, addressing a lack of domain-specific resources.
• The dataset comprises 200 diverse 30-second sequences and a full 45-minute half, sourced from Swiss Super League matches, featuring over 3.6 million annotated bounding boxes for players, balls, and others.
• Benchmarking state-of-the-art MOT methods (DeepSORT, FairMOT, ByteTrack) using metrics like HOTA reveals the challenges of tracking in dense soccer scenarios and highlights the need for algorithms better suited for occlusion and re-identification.

SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos

The paper introduces SoccerNet-Tracking, a comprehensive dataset and benchmark designed specifically for Multiple Object Tracking (MOT) in soccer videos. This work addresses the notable lack of datasets in the specialized domain of soccer video analysis, thus providing a valuable resource for advancing research in this field. The authors have constructed a large-scale dataset composed of 200 sequences, each lasting 30 seconds and encapsulating diverse and challenging soccer scenarios. Additionally, it includes a complete half-time session, spanning 45 minutes, to facilitate long-term tracking evaluation. The dataset is fully annotated with bounding boxes and tracklet IDs, enabling robust training and benchmarking of MOT methods.

Dataset Composition and Annotation

The dataset comprises video sequences sourced from 12 soccer matches recorded during the Swiss Super League 2019 season. These sequences are captured at 1080p resolution and 25 fps from single-camera views, conducive to seamless tracking. The sequences are extracted around key events such as goals, corners, fouls, free-kicks, and other significant actions, chosen to reflect tracking complexity due to factors like player occlusion and fast movements. The annotation process involved meticulous manual tagging, with bounding boxes detailed to track players (with jersey numbers), goalkeepers, referees, balls, and other field actors. This methodology results in over 3.6 million bounding boxes, providing comprehensive ground-truth data for training and evaluation.

Benchmarking and Analysis

The paper conducts a thorough benchmarking of three state-of-the-art MOT algorithms: DeepSORT, FairMOT, and ByteTrack. Each method is evaluated using metrics such as HOTA, which offers a granular view by decoupling detection performance from association accuracy—a critical consideration in complex sports environments. ByteTrack, a leading method in the MOT20 benchmark, exhibits superior association performance when ground-truth detections are provided, although its complete capabilities are highlighted through comprehensive detection and association accuracy when tested on the full dataset without annotations. Fine-tuning the FairMOT model on the SoccerNet-Tracking data notably enhances detection and tracking performance, showcasing the impact of domain-specific training.

Implications for Future Research

SoccerNet-Tracking stands as the largest and most challenging publicly available dataset dedicated to MOT in soccer contexts. Its release sets a robust foundation for the exploration of advanced tracking methods, particularly those that can navigate the difficulties posed by occlusions and long-term re-identification tasks inherent in sporting events. The evident complexities within the dataset underscore the need for improved algorithms capable of high-fidelity tracking and association across diverse match scenarios. The benchmark outcomes and per-class analyses illuminate areas where current methods can improve, specifically in tracking players during dense interactions and fast-paced actions.

Conclusion

This work makes significant strides in the field of sports video analysis by offering a domain-specific dataset for MOT, providing practitioners with the tools necessary to push the boundaries of automatic player tracking and analytics. The dataset and its corresponding benchmark results advocate for a continued focus on enhancing re-identification techniques and refining tracking algorithms, fostering advancements in the field of computer vision as applied to sports. This initiative is likely to spur further research developments and encourage competitive performance enhancement through community-driven challenges utilizing SoccerNet-Tracking.