Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Self-Supervised Multi-Object Tracking with Path Consistency (2404.05136v1)

Published 8 Apr 2024 in cs.CV and cs.AI

Abstract: In this paper, we propose a novel concept of path consistency to learn robust object matching without using manual object identity supervision. Our key idea is that, to track a object through frames, we can obtain multiple different association results from a model by varying the frames it can observe, i.e., skipping frames in observation. As the differences in observations do not alter the identities of objects, the obtained association results should be consistent. Based on this rationale, we generate multiple observation paths, each specifying a different set of frames to be skipped, and formulate the Path Consistency Loss that enforces the association results are consistent across different observation paths. We use the proposed loss to train our object matching model with only self-supervision. By extensive experiments on three tracking datasets (MOT17, PersonPath22, KITTI), we demonstrate that our method outperforms existing unsupervised methods with consistent margins on various evaluation metrics, and even achieves performance close to supervised methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Self-supervised multi-object tracking with cross-input consistency. In Advances in Neural Information Processing Systems, pages 13695–13706. Curran Associates, Inc., 2021.
  2. Multiple object tracking using k-shortest paths optimization. TPAMI, 33(9):1806–1819, 2011.
  3. Tracking without bells and whistles. In The IEEE International Conference on Computer Vision (ICCV), 2019.
  4. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP), pages 3464–3468, 2016.
  5. Memot: Multi-object tracking with memory. In CVPR 2022, 2022.
  6. Cascade r-cnn: Delving into high quality object detection. In CVPR, 2018.
  7. Observation-centric sort: Rethinking sort for robust multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9686–9696, 2023.
  8. Wongun Choi. Near-online multi-target tracking with aggregated local flow descriptor. In Proceedings of the IEEE international conference on computer vision, pages 3029–3037, 2015.
  9. Famnet: Joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6172–6181, 2019.
  10. Mot20: A benchmark for multi object tracking in crowded scenes, 2020.
  11. Multi-object tracking with multiple cues and switcher-aware classification, 2019.
  12. MeMOTR: Long-term memory-augmented transformer for multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9901–9910, 2023.
  13. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 2013.
  14. Mask r-cnn. In ICCV, 2017.
  15. Improvements to frank-wolfe optimization for multi-detector multi-object tracking. arXiv preprint arXiv:1705.08314, 2017.
  16. Simple unsupervised multi-object tracking. CoRR, abs/2006.02609, 2020.
  17. Multiple hypothesis tracking revisited. In ICCV, 2015.
  18. Multi-object tracking with neural gating using bilinear lstm. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  19. Learning of global objective for network flow in multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8855–8865, 2022.
  20. Ovtrack: Open-vocabulary multiple object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5567–5577, 2023.
  21. Yu-Lei Li. Unsupervised embedding and association network for multi-object tracking. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22. International Joint Conferences on Artificial Intelligence Organization, 2022.
  22. Uncertainty-aware unsupervised multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9996–10005, 2023.
  23. Gsm: Graph similarity model for multi-object tracking. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 530–536. International Joint Conferences on Artificial Intelligence Organization, 2020. Main track.
  24. Online multi-object tracking with unsupervised re-identification learning and occlusion estimation. Neurocomputing, 483:333–347, 2022a.
  25. Multi-object tracking meets moving uav. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8876–8885, 2022b.
  26. Opening up open world tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19045–19055, 2022c.
  27. Unified transformer tracker for object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8781–8790, 2022.
  28. Transforming model prediction for tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8731–8740, 2022.
  29. Trackformer: Multi-object tracking with transformers. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
  30. Tracking without label: Unsupervised multiple object tracking via contrastive similarity learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 16264–16273, 2023.
  31. MOT16: A benchmark for multi-object tracking. CoRR, abs/1603.00831, 2016.
  32. Quasi-dense similarity learning for multiple object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 164–173, 2021.
  33. Motiontrack: Learning robust short-term and long-term motions for multi-object tracking. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  34. Focus on details: Online multi-object tracking with diverse fine-grained representation. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  35. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS, 2015.
  36. Features for multi-target multi-camera tracking and re-identification. In CVPR, 2018.
  37. Heterogeneous association graph fusion for target association in multiple object tracking. TCSVT, 29(11):3269–3280, 2018.
  38. Siammot: Siamese multi-object tracking. In CVPR, 2021.
  39. Large scale real-world multi person tracking. In European Conference on Computer Vision. Springer, 2022a.
  40. Id-free person similarity learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14669–14679, 2022b.
  41. Id-free person similarity learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14689–14699, 2022c.
  42. Transtrack: Multiple-object tracking with transformer. arXiv preprint arXiv: 2012.15460, 2020.
  43. Dancetrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20993–21002, 2022.
  44. Multiple people tracking by lifted multicut and person re-identification. In CVPR, 2017.
  45. Fcos: Fully convolutional one-stage object detection. In ICCV, 2019.
  46. Learning to track with object permanence. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 10840–10849, 2021.
  47. Track without appearance: Learn box and tracklet embedding with local and global motion patterns for vehicle tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9876–9886, 2021.
  48. Towards real-time multi-object tracking. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pages 107–122. Springer, 2020.
  49. Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Conference on Image Processing (ICIP), pages 3645–3649. IEEE, 2017.
  50. Utm: A unified multiple object tracking model with identity-aware feature enhancement. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  51. Towards discriminative representation: Multi-view trajectory contrastive learning for online multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8834–8843, 2022.
  52. Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. In ECCV, 2012.
  53. Motr: End-to-end multiple-object tracking with transformer. In European Conference on Computer Vision (ECCV), 2022.
  54. Global data association for multi-object tracking using network flows. In CVPR, 2008.
  55. Robust multi-modality multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2365–2374, 2019.
  56. Fairmot: On the fairness of detection and re-identification in multiple object tracking. International Journal of Computer Vision, 129:3069–3087, 2021.
  57. Bytetrack: Multi-object tracking by associating every detection box. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
  58. Objects as points. arXiv preprint arXiv:1904.07850, 2019.
  59. Tracking objects as points. ECCV, 2020.
  60. Probabilistic two-stage detection. arXiv preprint arXiv:2103.07461, 2021.
  61. Global tracking transformers. In CVPR, 2022.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now