Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Traffic Scene Parsing through the TSP6K Dataset (2303.02835v2)

Published 6 Mar 2023 in cs.CV

Abstract: Traffic scene perception in computer vision is a critically important task to achieve intelligent cities. To date, most existing datasets focus on autonomous driving scenes. We observe that the models trained on those driving datasets often yield unsatisfactory results on traffic monitoring scenes. However, little effort has been put into improving the traffic monitoring scene understanding, mainly due to the lack of specific datasets. To fill this gap, we introduce a specialized traffic monitoring dataset, termed TSP6K, containing images from the traffic monitoring scenario, with high-quality pixel-level and instance-level annotations. The TSP6K dataset captures more crowded traffic scenes with several times more traffic participants than the existing driving scenes. We perform a detailed analysis of the dataset and comprehensively evaluate previous popular scene parsing methods, instance segmentation methods and unsupervised domain adaption methods. Furthermore, considering the vast difference in instance sizes, we propose a detail refining decoder for scene parsing, which recovers the details of different semantic regions in traffic scenes owing to the proposed TSP6K dataset. Experiments show its effectiveness in parsing the traffic monitoring scenes. Code and dataset are available at https://github.com/PengtaoJiang/TSP6K.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (104)
  1. Aau rainsnow traffic surveillance dataset, 2018.
  2. Self-supervised augmentation consistency for adapting semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 15384–15394, 2021.
  3. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(12):2481–2495, 2017.
  4. Yolact: Real-time instance segmentation. In Int. Conf. Comput. Vis., pages 9157–9166, 2019.
  5. Coco-stuff: Thing and stuff classes in context. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1209–1218, 2018.
  6. End-to-end object detection with transformers. In Eur. Conf. Comput. Vis., pages 213–229. Springer, 2020.
  7. Hybrid task cascade for instance segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4974–4983, 2019.
  8. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019.
  9. Semantic image segmentation with deep convolutional nets and fully connected crfs. In Int. Conf. Learn. Represent., 2015.
  10. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell., 40(4):834–848, 2017.
  11. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.
  12. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Eur. Conf. Comput. Vis., pages 801–818, 2018.
  13. Spgnet: Semantic prediction guidance for scene parsing. In Int. Conf. Comput. Vis., pages 5218–5228, 2019.
  14. Masked-attention mask transformer for universal image segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1290–1299, 2022.
  15. Per-pixel classification is not all you need for semantic segmentation. Adv. Neural Inform. Process. Syst., 34:17864–17875, 2021.
  16. MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark. https://github.com/open-mmlab/mmsegmentation, 2020.
  17. The cityscapes dataset for semantic urban scene understanding. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3213–3223, 2016.
  18. Semantic information for robot navigation: A survey. Applied Sciences, 10(2):497, 2020.
  19. Dark model adaptation: Semantic image segmentation from daytime to nighttime. In Proc. Int. Conf. Intelligent Transportation Systems, pages 3819–3824, 2018.
  20. An image is worth 16x16 words: Transformers for image recognition at scale. In Int. Conf. Learn. Represent., 2021.
  21. Megf-net: multi-exposure generation and fusion network for vehicle detection under dim light conditions. Visual Intelligence 1, 2023.
  22. The pascal visual object classes challenge: A retrospective. Int. J. Comput. Vis., 111(1):98–136, 2015.
  23. Instances as queries. In Int. Conf. Comput. Vis., pages 6910–6919, 2021.
  24. Dual attention network for scene segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3146–3154, 2019.
  25. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1231–1237, 2013.
  26. Is attention better than matrix decomposition? In Int. Conf. Learn. Represent., 2021.
  27. Segnext: Rethinking convolutional attention design for semantic segmentation. In Adv. Neural Inform. Process. Syst., 2022.
  28. Visual attention network. Computational visual media, 2023.
  29. Attention mechanisms in computer vision: A survey. Computational visual media, 8(3):331–368, 2022.
  30. Adaptive pyramid context network for semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7519–7528, 2019.
  31. Mask r-cnn. In Int. Conf. Comput. Vis., pages 2961–2969, 2017.
  32. Deep residual learning for image recognition. In IEEE Conf. Comput. Vis. Pattern Recog., pages 770–778, 2016.
  33. Cycada: Cycle-consistent adversarial domain adaptation. In Int. Conf. Mach. Learn., pages 1989–1998. Pmlr, 2018.
  34. Conditional generative adversarial network for structured domain adaptation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1335–1344, 2018.
  35. Conv2former: A simple transformer-style convnet for visual recognition. arXiv preprint arXiv:2211.11943, 2022.
  36. Strip pooling: Rethinking spatial pooling for scene parsing. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4003–4012, 2020.
  37. Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 9924–9935, 2022.
  38. Hrda: Context-aware high-resolution domain-adaptive semantic segmentation. In Eur. Conf. Comput. Vis., pages 372–391. Springer, 2022.
  39. The apolloscape open dataset for autonomous driving and its application. IEEE Trans. Pattern Anal. Mach. Intell., 42(10):2702–2719, 2019.
  40. Mask scoring r-cnn. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6409–6418, 2019.
  41. Ccnet: Criss-cross attention for semantic segmentation. In Int. Conf. Comput. Vis., pages 603–612, 2019.
  42. Navigation-oriented scene understanding for robotic autonomy: Learning to segment driveability in egocentric images. IEEE Robotics and Automation Letters, 7(2):2913–2920, 2022.
  43. Deep learning-based moving object segmentation: Recent progress and research prospects. Machine Intelligence Research, 20(3):335–369, 2023.
  44. Predicting scene parsing and motion dynamics in the future. In Adv. Neural Inform. Process. Syst., volume 30, 2017.
  45. Urban tracker: Multiple object tracking in urban mixed traffic. In IEEE Winter Conf. Appl. Comput. Vis., pages 885–892. IEEE, 2014.
  46. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
  47. Class-balanced pixel-level self-labeling for domain adaptive semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 11593–11603, 2022.
  48. Expectation-maximization attention networks for semantic segmentation. In Int. Conf. Comput. Vis., pages 9167–9176, 2019.
  49. Ar-cnn: an attention ranking network for learning urban perception. Science China Information Sciences, 65(1):112104, 2022.
  50. Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach. In Int. Conf. Comput. Vis., pages 6758–6767, 2019.
  51. Dynamic-structured semantic propagation network. In IEEE Conf. Comput. Vis. Pattern Recog., pages 752–761, 2018.
  52. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1925–1934, 2017.
  53. Feature pyramid networks for object detection. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2117–2125, 2017.
  54. Microsoft coco: Common objects in context. In Eur. Conf. Comput. Vis., 2014.
  55. Path aggregation network for instance segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8759–8768, 2018.
  56. Swin transformer: Hierarchical vision transformer using shifted windows. In Int. Conf. Comput. Vis., pages 10012–10022, 2021.
  57. Fully convolutional networks for semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3431–3440, 2015.
  58. Traffic flow prediction with big data: a deep learning approach. IEEE Transactions on Intelligent Transportation Systems, 16(2):865–873, 2014.
  59. Pixmatch: Unsupervised domain adaptation via pixelwise consistency training. In IEEE Conf. Comput. Vis. Pattern Recog., pages 12435–12445, 2021.
  60. The mapillary vistas dataset for semantic understanding of street scenes. In Int. Conf. Comput. Vis., pages 4990–4999, 2017.
  61. Playing for data: Ground truth from computer games. In Eur. Conf. Comput. Vis., pages 102–118. Springer, 2016.
  62. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  63. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3234–3243, 2016.
  64. Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation. In Int. Conf. Comput. Vis., pages 7374–7383, 2019.
  65. Acdc: The adverse conditions dataset with correspondences for semantic driving scene understanding. In Int. Conf. Comput. Vis., pages 10765–10775, 2021.
  66. Segmenter: Transformer for semantic segmentation. In Int. Conf. Comput. Vis., pages 7262–7272, 2021.
  67. Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification. In IEEE Conf. Comput. Vis. Pattern Recog., pages 8797–8806, 2019.
  68. Conditional convolutions for instance segmentation. In Eur. Conf. Comput. Vis., pages 282–298. Springer, 2020.
  69. Going deeper with image transformers. In Int. Conf. Comput. Vis., pages 32–42, 2021.
  70. Learning to adapt structured output space for semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7472–7481, 2018.
  71. Domain adaptation for structured output via discriminative patch representations. In Int. Conf. Comput. Vis., pages 1456–1465, 2019.
  72. Idd: A dataset for exploring problems of autonomous navigation in unconstrained environments. In IEEE Winter Conf. Appl. Comput. Vis., pages 1743–1751. IEEE, 2019.
  73. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2517–2526, 2019.
  74. Classes matter: A fine-grained adversarial approach to cross-domain semantic segmentation. In Eur. Conf. Comput. Vis., pages 642–659. Springer, 2020.
  75. Solo: Segmenting objects by locations. In Eur. Conf. Comput. Vis., pages 649–665. Springer, 2020.
  76. Solov2: Dynamic and fast instance segmentation. Adv. Neural Inform. Process. Syst., 33:17721–17732, 2020.
  77. Uncertainty-aware pseudo label refinery for domain adaptive semantic segmentation. In Int. Conf. Comput. Vis., pages 9092–9101, 2021.
  78. Yolop: You only look once for panoptic driving perception. Machine Intelligence Research, 19(6):550–562, 2022.
  79. Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation. In arXiv preprint arXiv:1903.11816, 2019.
  80. Unified perceptual parsing for scene understanding. In Eur. Conf. Comput. Vis., pages 418–434, 2018.
  81. Sepico: Semantic-guided pixel contrast for domain adaptive semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 2023.
  82. Polarmask: Single shot instance segmentation with polar representation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 12193–12202, 2020.
  83. Segformer: Simple and efficient design for semantic segmentation with transformers. Adv. Neural Inform. Process. Syst., 34:12077–12090, 2021.
  84. Fda: Fourier domain adaptation for semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4085–4095, 2020.
  85. Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. Int. J. Comput. Vis., pages 1–18, 2021.
  86. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Eur. Conf. Comput. Vis., pages 325–341, 2018.
  87. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2636–2645, 2020.
  88. Object-contextual representations for semantic segmentation. In Eur. Conf. Comput. Vis., 2020.
  89. Ocnet: Object context for semantic segmentation. Int. J. Comput. Vis., 129(8):2375–2398, 2021.
  90. Wilddash-creating hazard-aware benchmarks. In Eur. Conf. Comput. Vis., pages 402–416, 2018.
  91. Unifying panoptic segmentation for autonomous driving. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21351–21360, 2022.
  92. Context encoding for semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7151–7160, 2018.
  93. Category anchor-guided unsupervised domain adaptation for semantic segmentation. Adv. Neural Inform. Process. Syst., 32, 2019.
  94. Scale-adaptive convolutions for scene parsing. In Int. Conf. Comput. Vis., pages 2031–2039, 2017.
  95. Topformer: Token pyramid transformer for mobile semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 12083–12093, 2022.
  96. K-net: Towards unified image segmentation. In Adv. Neural Inform. Process. Syst., volume 34, pages 10326–10338, 2021.
  97. Icnet for real-time semantic segmentation on high-resolution images. In Eur. Conf. Comput. Vis., pages 405–420, 2018.
  98. Pyramid scene parsing network. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2881–2890, 2017.
  99. Psanet: Point-wise spatial attention network for scene parsing. In Eur. Conf. Comput. Vis., pages 267–283, 2018.
  100. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In IEEE Conf. Comput. Vis. Pattern Recog., pages 6881–6890, 2021.
  101. Scene parsing through ade20k dataset. In IEEE Conf. Comput. Vis. Pattern Recog., pages 633–641, 2017.
  102. Asymmetric non-local neural networks for semantic segmentation. In Int. Conf. Comput. Vis., pages 593–602, 2019.
  103. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Eur. Conf. Comput. Vis., pages 289–305, 2018.
  104. Confidence regularized self-training. In Int. Conf. Comput. Vis., pages 5982–5991, 2019.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - PengtaoJiang/TSP6K: The official PyTorch code for "Traffic Scene Parsing through the TSP6K Dataset". (27 stars)