Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Semi-Supervised Semantic Segmentation with Dual-Level Siamese Structure Network (2307.13938v2)

Published 26 Jul 2023 in cs.CV and cs.LG

Abstract: Semi-supervised semantic segmentation (SSS) is an important task that utilizes both labeled and unlabeled data to reduce expenses on labeling training examples. However, the effectiveness of SSS algorithms is limited by the difficulty of fully exploiting the potential of unlabeled data. To address this, we propose a dual-level Siamese structure network (DSSN) for pixel-wise contrastive learning. By aligning positive pairs with a pixel-wise contrastive loss using strong augmented views in both low-level image space and high-level feature space, the proposed DSSN is designed to maximize the utilization of available unlabeled data. Additionally, we introduce a novel class-aware pseudo-label selection strategy for weak-to-strong supervision, which addresses the limitations of most existing methods that do not perform selection or apply a predefined threshold for all classes. Specifically, our strategy selects the top high-confidence prediction of the weak view for each class to generate pseudo labels that supervise the strong augmented views. This strategy is capable of taking into account the class imbalance and improving the performance of long-tailed classes. Our proposed method achieves state-of-the-art results on two datasets, PASCAL VOC 2012 and Cityscapes, outperforming other SSS algorithms by a significant margin. The source code is available at https://github.com/kunzhan/DSSN.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. MixMatch: A holistic approach to semi-supervised learning. NeurIPS 32 (2019).
  2. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV. Springer, 801–818.
  3. Semi-supervised semantic segmentation with cross pseudo supervision. In CVPR. IEEE, 2613–2622.
  4. Learning a similarity metric discriminatively, with application to face verification. In CVPR. IEEE, 539–546.
  5. The cityscapes dataset for semantic urban scene understanding. In CVPR. IEEE, 3213–3223.
  6. Generative adversarial networks: An overview. IEEE Signal Processing Magazine 35, 1 (2018), 53–65.
  7. Imagenet: A large-scale hierarchical image database. In CVPR. IEEE, 248–255.
  8. Terrance DeVries and Graham W Taylor. 2017. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017).
  9. The pascal visual object classes challenge: A retrospective. IJCV 111 (2015), 98–136.
  10. DMT: Dynamic mutual training for semi-supervised learning. Pattern Recognition 130 (2022), 108777.
  11. Semi-supervised semantic segmentation needs strong, varied perturbations. In BMVC.
  12. Bootstrap your own latent-a new approach to self-supervised learning. In NeurIPS, Vol. 33. 21271–21284.
  13. Dimensionality reduction by learning an invariant mapping. In CVPR. IEEE, 1735–1742.
  14. Semantic contours from inverse detectors. In ICCV. IEEE, 991–998.
  15. Deep residual learning for image recognition. In CVPR. IEEE, 770–778.
  16. Learning deep representations by mutual information estimation and maximization. In ICLR.
  17. Adversarial learning for semi-supervised semantic segmentation. In BMVC.
  18. Semi-supervised semantic segmentation via gentle teaching assistant. In NeurIPS, Vol. 35. 2803–2816.
  19. Guided collaborative training for pixel-wise semi-supervised learning. In ECCV. Springer, 429–445.
  20. Samuli Laine and Timo Aila. 2017. Temporal ensembling for semi-supervised learning. In ICLR.
  21. Bootstrapping semantic segmentation with regional contrast. In ICLR.
  22. Perturbed and strict mean teachers for semi-supervised semantic segmentation. In CVPR. IEEE, 4258–4267.
  23. Long-tail learning via logit adjustment. In ICLR.
  24. Semi-supervised semantic segmentation with high-and low-level consistency. TPAMI 43, 4 (2019), 1369–1379.
  25. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  26. Semi-supervised semantic segmentation with cross-consistency training. In CVPR. IEEE, 12674–12684.
  27. FixMatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, Vol. 33. 596–608.
  28. Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, Vol. 30.
  29. A. M. Walker. 1969. On the Asymptotic Behaviour of Posterior Distributions. Journal of the Royal Statistical Society: Series B (Methodological) 31, 1 (1969), 80–88.
  30. Semi-supervised semantic segmentation using unreliable pseudo-labels. In CVPR. IEEE, 4248–4257.
  31. Unsupervised data augmentation for consistency training. NeurIPS 33 (2020), 6256–6268.
  32. Semi-supervised semantic segmentation with prototype-based consistency regularization. In NeurIPS, Vol. 35. 26007–26020.
  33. Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. In CVPR.
  34. ST++: Make self-training work better for semi-supervised semantic segmentation. In CVPR. IEEE, 4268–4277.
  35. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV. IEEE, 6023–6032.
  36. Pixel contrastive-consistent semi-supervised semantic segmentation. In ICCV. IEEE, 7273–7282.
  37. PseudoSeg: Designing pseudo labels for semantic segmentation. In ICLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Zhibo Tain (1 paper)
  2. Xiaolin Zhang (29 papers)
  3. Peng Zhang (642 papers)
  4. Kun Zhan (38 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now