Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos (2404.13868v1)

Published 22 Apr 2024 in cs.CV

Abstract: Multi-object tracking (MOT) is a critical and challenging task in computer vision, particularly in situations involving objects with similar appearances but diverse movements, as seen in team sports. Current methods, largely reliant on object detection and appearance, often fail to track targets in such complex scenarios accurately. This limitation is further exacerbated by the lack of comprehensive and diverse datasets covering the full view of sports pitches. Addressing these issues, we introduce TeamTrack, a pioneering benchmark dataset specifically designed for MOT in sports. TeamTrack is an extensive collection of full-pitch video data from various sports, including soccer, basketball, and handball. Furthermore, we perform a comprehensive analysis and benchmarking effort to underscore TeamTrack's utility and potential impact. Our work signifies a crucial step forward, promising to elevate the precision and effectiveness of MOT in complex, dynamic settings such as team sports. The dataset, project code and competition is released at: https://atomscott.github.io/TeamTrack/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Social lstm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 961–971, 2016.
  2. Tracking without bells and whistles. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 941–951, 2019.
  3. Learning a neural solver for multiple object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6247–6257, 2020.
  4. End-to-end object detection with transformers. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 213–229. Springer, 2020.
  5. Soccernet-tracking: Multiple object tracking dataset and benchmark in soccer videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3491–3502, 2022.
  6. Sportsmot: A large multi-object tracking dataset in multiple sports scenes. arXiv preprint arXiv:2304.05170, 2023.
  7. Actions speak louder than goals: Valuing player actions in soccer. In KDD, pages 1851–1861, 2019.
  8. Mot20: A benchmark for multi object tracking in crowded scenes. arXiv preprint arXiv:2003.09003, 2020.
  9. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6569–6578, 2019.
  10. Sset: a dataset for shot segmentation, event detection, player tracking in soccer videos. Multimedia Tools and Applications, 79:28971–28992, 2020.
  11. Tracking soccer players aiming their kinematical motion analysis. Computer Vision and Image Understanding, 101(2):122–135, 2006.
  12. The kitti vision benchmark suite. URL http://www. cvlibs. net/datasets/kitti, 2(5), 2015.
  13. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  14. Transreid: Transformer-based object re-identification. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14993–15002, 2021.
  15. The current use of gps, its potential, and limitations in soccer. Strength & Conditioning Journal, 40(3):83–94, 2018.
  16. Self-supervised small soccer player detection and tracking. In Proceedings of the 3rd international workshop on multimedia content analysis in sports, pages 9–18, 2020.
  17. Parallel tracking of all soccer players by integrating detected positions in multiple view images. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., pages 751–754. IEEE, 2004.
  18. YOLO by Ultralytics, 2023.
  19. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
  20. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
  21. Efficient tracking of team sport players with few game-specific annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3461–3471, 2022.
  22. A survey on player tracking in soccer videos. Computer Vision and Image Understanding, 159:19–46, 2017.
  23. Trackformer: Multi-object tracking with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8844–8854, 2022.
  24. Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831, 2016.
  25. Pets 2016: Dataset and challenge. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1–8, 2016.
  26. Soccer video and player position dataset. In Proceedings of the 5th ACM Multimedia Systems Conference, pages 18–23, 2014.
  27. No blind spots: Full-surround multi-object tracking for autonomous vehicles using cameras and lidars. IEEE Transactions on Intelligent Vehicles, 4(4):588–599, 2019.
  28. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.
  29. Soccertrack: A dataset and tracking algorithm for soccer with fish-eye and drone videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3569–3579, 2022.
  30. Aaformer: A multi-modal transformer network for aerial agricultural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1705–1711, 2022.
  31. William Spearman. Beyond expected goals. In Proceedings of the 12th MIT sloan sports analytics conference, pages 1–17, 2018.
  32. Tracking and labelling of interacting multiple targets. In European conference on computer vision, pages 619–632. Springer, 2006.
  33. Multi-target tracking on a large scale: Experiences from football player tracking. In Proceedings of the IEEE ICRA 2009 workshop on people detection and tracking. Citeseer, 2009.
  34. Transtrack: Multiple object tracking with transformer. arXiv preprint arXiv:2012.15460, 2020.
  35. Dancetrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20993–21002, 2022.
  36. Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction. In International Workshop on Machine Learning and Data Mining for Sports Analytics (MLSA). Springer, 2022.
  37. Soccer: Who has the ball? generating visual analytics and player statistics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1749–1757, 2018.
  38. Sportstrack: An innovative method for tracking athletes in sports scenes. arXiv preprint arXiv:2211.07173, 2022.
  39. Ua-detrac: A new benchmark and protocol for multi-object detection and tracking. Computer Vision and Image Understanding, 193:102907, 2020.
  40. Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP), pages 3645–3649. IEEE, 2017.
  41. How to train your deep multi-object tracker. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6787–6796, 2020.
  42. Motr: End-to-end multiple-object tracking with transformer. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVII, pages 659–675. Springer, 2022.
  43. Collaborate ball and player trajectory extraction in broadcast soccer video. In 2008 19th International Conference on Pattern Recognition, pages 1–4. IEEE, 2008.
  44. Motrv2: Bootstrapping end-to-end multi-object tracking by pretrained object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22056–22065, 2023.
  45. Omni-scale feature learning for person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3702–3712, 2019.
Citations (2)
View on Semantic Scholar

Summary

  • The paper introduces the TeamTrack dataset, a large-scale benchmark with over 279,900 frames and 4,374,900 bounding boxes for soccer, basketball, and handball.
  • It details a rigorous methodology using dual camera views and extensive annotation processes, addressing key challenges like occlusion and appearance similarity.
  • Benchmarks with models such as YOLOv8, LSTM, ByteTrack, and BoT-SORT demonstrate the dataset's potential to propel advancements in sports tracking and analytics.

Enhancing MOT in Sports through TeamTrack: A Comprehensive Dataset and Evaluation

Introduction to TeamTrack Dataset

The challenges of Multi-Object Tracking (MOT) in team sports are complex due to rapid player movements, occlusions, and similar player apparences. To address the need for advanced MOT in sports, the paper introduces the TeamTrack dataset, a large-scale benchmark featuring over 279,900 frames and 4,374,900 bounding boxes across soccer, basketball, and handball from multiple perspectives. This dataset not only fills the gap left by incomplete pitch coverage and single-sport focus in existing datasets but provides a unique setup to challenge and enhance current MOT solutions in a well-rounded and extensive way.

Contributions of the Study

The paper outlines three major contributions:

  1. Development of TeamTrack, a large-scale dataset for MOT in sports featuring extensive variety and scope.
  2. In-depth analysis and characterization of this novel dataset highlighting its potential to advance sports analytics and tracking technologies.
  3. Comprehensive benchmarks using TeamTrack for object detection, trajectory forecasting, and multi-object tracking, setting a new standard for future research and applications in sports MOT.

Methodology and Dataset Features

  • Data Collection and Annotation: Utilizing fisheye and drone cameras, videos were captured offering both side and top views. An extensive annotation process was described, applying tools like CVAT and Labelbox, and involved dealing with high volumes of data which required around 600 person-hours.
  • Appearance Similarity and Dynamics: TeamTrack was assessed against other MOT datasets showing higher similarity in player appearances and more complex motion patterns, which are quantified by new metrics for evaluating cosine similarities of re-ID features and average IoU scores between adjacent frames.

Experimental Evaluations

  • Object Detection: This involved fine-tuning a YOLOv8 model on the TeamTrack dataset, with the evaluation highlighting differences in performance based on the view types and the necessity for fine-tuning when dealing with sports footage.
  • Trajectory Forecasting: Employing models like LSTM showed variance in performance across sports within the TeamTrack dataset, indicating complex player movement.
  • Multi-object Tracking: Applying ByteTrack and BoT-SORT, the paper revealed how the diversity and complexity of the dataset provided unique challenges, reflected in the variability of scores across sports and view perspectives.

Dataset Implications and Potential for Future Research

The TeamTrack dataset with its detailed annotations and dual perspectives (top and side) ensures a comprehensive set of data that can help in understanding the nuanced requirements of sports MOT. The dual perspectives particularly allow for the exploration of multi-view tracking systems which could significantly improve precision and robustly handle occlusions and player interactions.

This dataset’s introduction prompts several potential areas for development:

  • Algorithmic Improvements: Given the complexity and scale of the dataset, there is significant room for the development of specialized MOT algorithms that can handle high dynamics, occlusions, and appearance similarities competently.
  • Cross-disciplinary Applications: Insights from TeamTrack could extend into other areas such as crowd analysis and robotics, where similar challenges are prevalent.
  • Enhancement through Augmentation: Including more sports, broader geographic diversity, and environmental conditions in the dataset could further validate and refine the robustness of MOT systems.

In conclusion, the TeamTrack dataset represents a significant step toward addressing the complexities associated with tracking in team sports. By offering a detailed analysis and comprehensive benchmarks, it sets the foundation for future innovations in the field of sports analytics and beyond, promising enhanced capabilities in MOT across various challenging scenarios.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets