Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Contextrast: Contextual Contrastive Learning for Semantic Segmentation (2404.10633v2)

Published 16 Apr 2024 in cs.CV

Abstract: Despite great improvements in semantic segmentation, challenges persist because of the lack of local/global contexts and the relationship between them. In this paper, we propose Contextrast, a contrastive learning-based semantic segmentation method that allows to capture local/global contexts and comprehend their relationships. Our proposed method comprises two parts: a) contextual contrastive learning (CCL) and b) boundary-aware negative (BANE) sampling. Contextual contrastive learning obtains local/global context from multi-scale feature aggregation and inter/intra-relationship of features for better discrimination capabilities. Meanwhile, BANE sampling selects embedding features along the boundaries of incorrectly predicted regions to employ them as harder negative samples on our contrastive learning, resolving segmentation issues along the boundary region by exploiting fine-grained details. We demonstrate that our Contextrast substantially enhances the performance of semantic segmentation networks, outperforming state-of-the-art contrastive learning approaches on diverse public datasets, e.g. Cityscapes, CamVid, PASCAL-C, COCO-Stuff, and ADE20K, without an increase in computational cost during inference.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, 30(2):88–97, 2009.
  2. COCO-stuff: Thing and stuff classes in context. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1209–1218, 2018.
  3. Are all negatives created equal in contrastive instance discrimination? arXiv preprint arXiv:2010.06682, 2020.
  4. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint arXiv:1412.7062, 2014.
  5. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4):834–848, 2017a.
  6. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017b.
  7. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision, pages 801–818, 2018.
  8. Cars can’t fly up in the sky: Improving urban-scene segmentation via height-driven attention networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitionn, pages 9373–9383, 2020.
  9. The Cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3213–3223, 2016.
  10. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  11. Semantic correlation promoted shape-variant context for segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8885–8894, 2019.
  12. ColonFormer: An efficient transformer-based method for colon polyp segmentation. IEEE Access, 10:80575–80586, 2022.
  13. Using DUCK-Net for polyp image segmentation. Scientific Reports, 13(1):9803, 2023.
  14. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3146–3154, 2019.
  15. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 297–304. JMLR, 2010.
  16. Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv preprint arXiv:2101.06085, 2021.
  17. Region-aware contrastive learning for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16291–16301, 2021.
  18. Semantic scene segmentation for robotics. In Deep Learning for Robot Perception and Cognition, pages 279–311. Elsevier, 2022.
  19. Progressive semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16755–16764, 2021.
  20. Hard negative mixing for contrastive learning. Advances in Neural Information Processing Systems, 33:21798–21809, 2020.
  21. Adaptive affinity fields for semantic segmentation. In Proceedings of the European Conference on Computer Vision, pages 587–602, 2018.
  22. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661–18673, 2020.
  23. Sub-pixel distance maps and weighted distance transforms. Journal of Mathematical Imaging and Vision, 6:223–233, 1996.
  24. Pyramid attention network for semantic segmentation. arXiv preprint arXiv:1805.10180, 2018.
  25. Deep hierarchical semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1246–1257, 2022a.
  26. Targeted supervised contrastive learning for long-tailed recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6918–6928, 2022b.
  27. Swin Transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021.
  28. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3431–3440, 2015.
  29. The role of context for object detection and semantic segmentation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 891–898, 2014.
  30. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  31. Multi-scale and cross-scale contrastive learning for semantic segmentation. In Proceedings of the European Conference on Computer Vision, pages 413–429. Springer, 2022.
  32. FCN-transformer feature fusion for polyp segmentation. In Proceedings of the Annual Conference on Medical Image Understanding and Analysis, pages 892–907. Springer, 2022.
  33. FaceNet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 815–823, 2015.
  34. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7262–7272, 2021.
  35. High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514, 2019.
  36. Semantic diffusion network for semantic segmentation. Advances in Neural Information Processing Systems, 35:8702–8716, 2022.
  37. Semantic scene segmentation for robotics applications. In Proceedings of the International Conference on Information, Intelligence, Systems & Applications, pages 1–4, 2021.
  38. Active boundary loss for semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2397–2405, 2022.
  39. Deep high-resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3349–3364, 2020.
  40. Stepwise feature fusion: Local guides global. arXiv preprint arXiv:2203.03635, 2023a.
  41. Exploring cross-image pixel contrast for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7303–7313, 2021.
  42. Internimage: Exploring large-scale vision foundation models with deformable convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14408–14419, 2023b.
  43. ConvNext v2: Co-designing and scaling ConvNets with masked autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133–16142, 2023.
  44. Unified perceptual parsing for scene understanding. In Proceedings of the European Conference on Computer Vision, pages 418–434, 2018.
  45. Delving into inter-image invariance for unsupervised visual representations. International Journal of Computer Vision, 130(12):2994–3013, 2022.
  46. PIDNet: A real-time semantic segmentation network inspired by PID controllers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19529–19539, 2023.
  47. Learning a discriminative feature network for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1857–1866, 2018.
  48. Context prior for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12416–12425, 2020.
  49. Object-contextual representations for semantic segmentation. In Proceedings of the European Conference on Computer Vision, pages 173–190. Springer, 2020a.
  50. SegFix: Model-agnostic boundary refinement for segmentation. In Proceedings of the European Conference on Computer Vision, pages 489–506. Springer, 2020b.
  51. Semantic segmentation with extended deeplabv3 architecture. In Proceedings of the Signal Processing and Communications Applications Conference, pages 1–4. IEEE, 2019.
  52. Dual graph convolutional network for semantic segmentation. arXiv preprint arXiv:1909.06121, 2019.
  53. Pyramid scene parsing network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2881–2890, 2017.
  54. Understanding imbalanced semantic segmentation through neural collapse. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19550–19560, 2023.
  55. Scene parsing through ADE20K dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 633–641, 2017.
  56. Fusion PSPNet image segmentation based method for multi-focus image fusion. IEEE Photonics Journal, 11(6):1–12, 2019.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now