Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label (2402.17555v1)

Published 27 Feb 2024 in cs.CV

Abstract: Scribble-based weakly-supervised semantic segmentation using sparse scribble supervision is gaining traction as it reduces annotation costs when compared to fully annotated alternatives. Existing methods primarily generate pseudo-labels by diffusing labeled pixels to unlabeled ones with local cues for supervision. However, this diffusion process fails to exploit global semantics and class-specific cues, which are important for semantic segmentation. In this study, we propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision. Directly adopting pseudo-labels might misguide the segmentation model, thus we design a localization rectification module to correct foreground representations in the feature space. To further combine the advantages of both supervisions, we also introduce a distance entropy loss for uncertainty reduction, which adapts per-pixel confidence weights according to the reliable region determined by the scribble and pseudo-label's boundary. Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method.The code is available at https://github.com/Zxl19990529/Class-driven-Scribble-Promotion-Network.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. What’s the point: Semantic segmentation with point supervision. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14, 549–565. Springer.
  2. Seminar learning for click-level weakly supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6920–6929.
  3. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4): 834–848.
  4. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 801–818.
  5. Class re-activation maps for weakly-supervised semantic segmentation. In CVPR, 969–978.
  6. Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In CVPR, 1635–1643.
  7. The PASCAL visual object classes challenge 2012 (VOC2012) development kit. Pattern Anal. Stat. Model. Comput. Learn., Tech. Rep, 2007(1-45): 5.
  8. Grady, L. 2006. Random walks for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 28(11): 1768–1783.
  9. Semantic contours from inverse detectors. In 2011 international conference on computer vision, 991–998. IEEE.
  10. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  11. Simple does it: Weakly supervised instance and semantic segmentation. In CVPR, 876–885.
  12. Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, 695–711. Springer.
  13. Bbam: Bounding box attribution map for weakly supervised semantic and instance segmentation. In CVPR, 2643–2652.
  14. Threshold matters in wsss: Manipulating the activation for the robust and accurate segmentation model against thresholds. In CVPR, 4330–4339.
  15. Tree energy loss: Towards sparsely annotated semantic segmentation. In CVPR, 16907–16916.
  16. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In CVPR, 3159–3167.
  17. When does label smoothing help? Advances in neural information processing systems, 32.
  18. Scribble-supervised semantic segmentation by uncertainty reduction on neural representation and self-supervision on neural eigenspace. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7416–7425.
  19. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In ICCV, 1742–1750.
  20. Boundary-Enhanced Co-Training for Weakly Supervised Semantic Segmentation. In CVPR, 19574–19584.
  21. ” GrabCut” interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG), 23(3): 309–314.
  22. Learning affinity from attention: end-to-end weakly-supervised semantic segmentation with transformers. In CVPR, 16846–16855.
  23. Token contrast for weakly-supervised semantic segmentation. In CVPR, 3093–3102.
  24. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
  25. Normalized cut loss for weakly-supervised cnn segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1818–1827.
  26. On regularized losses for weakly-supervised cnn segmentation. In ECCV, 507–522.
  27. Learning random-walk label propagation for weakly-supervised semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7158–7166.
  28. Boundary perception guidance: A scribble-supervised semantic segmentation approach. In IJCAI International joint conference on artificial intelligence.
  29. Cycle-consistent learning for weakly supervised semantic segmentation. In Proceedings of the 3rd International Workshop on Human-Centric Multimedia Analysis, 7–13.
  30. Weakly-supervised semantic segmentation by iteratively mining common object features. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1354–1362.
  31. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In CVPR, 12275–12284.
  32. Deep Bilateral Filtering Network for Point-Supervised Semantic Segmentation in Remote Sensing Images. IEEE Transactions on Image Processing, 31: 7419–7434.
  33. Sparsely Annotated Semantic Segmentation With Adaptive Gaussian Mixtures. In CVPR, 15454–15464.
  34. Scribble-supervised semantic segmentation inference. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15354–15363.
  35. Affinity attention graph neural network for weakly supervised semantic segmentation. TPAMI, 44(11): 8082–8096.
  36. Complementary patch for weakly supervised semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, 7242–7251.
  37. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2921–2929.
  38. Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation. arXiv:2308.04949.
  39. Background-aware classification activation map for weakly supervised object localization. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  40. Weakly supervised object localization as domain adaption. In CVPR, 14637–14646.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now