Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation (2403.07630v1)

Published 12 Mar 2024 in cs.CV and cs.AI

Abstract: Recent weakly supervised semantic segmentation (WSSS) methods strive to incorporate contextual knowledge to improve the completeness of class activation maps (CAM). In this work, we argue that the knowledge bias between instances and contexts affects the capability of the prototype to sufficiently understand instance semantics. Inspired by prototype learning theory, we propose leveraging prototype awareness to capture diverse and fine-grained feature attributes of instances. The hypothesis is that contextual prototypes might erroneously activate similar and frequently co-occurring object categories due to this knowledge bias. Therefore, we propose to enhance the prototype representation ability by mitigating the bias to better capture spatial coverage in semantic object regions. With this goal, we present a Context Prototype-Aware Learning (CPAL) strategy, which leverages semantic context to enrich instance comprehension. The core of this method is to accurately capture intra-class variations in object features through context-aware prototypes, facilitating the adaptation to the semantic attributes of various instances. We design feature distribution alignment to optimize prototype awareness, aligning instance feature distributions with dense features. In addition, a unified training framework is proposed to combine label-guided classification supervision and prototypes-guided self-supervision. Experimental results on PASCAL VOC 2012 and MS COCO 2014 show that CPAL significantly improves off-the-shelf methods and achieves state-of-the-art performance. The project is available at https://github.com/Barrett-python/CPAL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. Weakly supervised learning of instance segmentation with inter-pixel relations. In CVPR, 2019.
  2. Single-stage semantic segmentation from image labels. In CVPR, 2020.
  3. Weakly-supervised semantic segmentation via sub-category exploration. In CVPR, 2020.
  4. Weakly supervised semantic segmentation with boundary exploration. In ECCV, 2020.
  5. Fpr: False positive rectification for weakly supervised semantic segmentation. In ICCV, 2023.
  6. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. TPAMI, 2017.
  7. Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation. In CVPR, 2022a.
  8. Extracting class activation maps from non-discriminative features as well. In CVPR, 2023.
  9. Class re-activation maps for weakly-supervised semantic segmentation. In CVPR, 2022b.
  10. Out-of-candidate rectification for weakly supervised semantic segmentation. In CVPR, 2023.
  11. Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In ICCV, 2015.
  12. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
  13. Weakly supervised semantic segmentation by pixel-to-prototype contrast. In CVPR, 2022.
  14. The pascal visual object classes (voc) challenge. IJCV, 2010.
  15. Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation. In CVPR, 2020a.
  16. Cian: Cross-image affinity net for weakly supervised semantic segmentation. In AAAI, 2020b.
  17. Soft neighbors are positive supporters in contrastive visual representation learning. In ICLR, 2023.
  18. Semantic contours from inverse detectors. In ICCV, 2011.
  19. Dynamic multi-scale filters for semantic segmentation. In CVPR, 2019.
  20. Deep residual learning for image recognition. In CVPR, 2016.
  21. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. JMLR, 2021.
  22. Weakly-supervised semantic segmentation network with deep seeded region growing. In CVPR, 2018.
  23. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
  24. Deep incomplete multi-view clustering with cross-view partial sample and prototype alignment. In CVPR, 2023.
  25. Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In ECCV, 2016.
  26. Efficient inference in fully connected crfs with gaussian edge potentials. In NeurIPS, 2011.
  27. Unlocking the potential of ordinary classifier: Class-specific adversarial erasing framework for weakly supervised semantic segmentation. In ICCV, 2021.
  28. Weakly supervised semantic segmentation via adversarial learning of classifier and reconstructor. In CVPR, 2023.
  29. Reducing information bottleneck for weakly supervised semantic segmentation. In NeurIPS, 2021a.
  30. Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation. In CVPR, 2021b.
  31. Bbam: Bounding box attribution map for weakly supervised semantic and instance segmentation. In CVPR, 2021c.
  32. Weakly supervised semantic segmentation using out-of-distribution data. In CVPR, 2022a.
  33. Threshold matters in wsss: manipulating the activation for the robust and accurate segmentation model against thresholds. In CVPR, 2022b.
  34. Railroad is not a train: Saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In CVPR, 2021d.
  35. Expansion and shrinkage of localization for weakly-supervised semantic segmentation. In NeurIPS, 2022.
  36. Group-wise semantic mining for weakly supervised semantic segmentation. In AAAI, 2021.
  37. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In CVPR, 2016.
  38. Microsoft coco: Common objects in context. In ECCV, 2014.
  39. Clip is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. In CVPR, 2023.
  40. Prototype rectification for few-shot learning. In ECCV, 2020a.
  41. Part-aware prototype network for few-shot semantic segmentation. In ECCV, 2020b.
  42. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
  43. Usage: A unified seed area generation paradigm for weakly supervised semantic segmentation. In ICCV, 2023.
  44. Boundary-enhanced co-training for weakly supervised semantic segmentation. In CVPR, 2023.
  45. Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers. In CVPR, 2022.
  46. Token contrast for weakly-supervised semantic segmentation. In CVPR, 2023.
  47. Amp: Adaptive masked proxies for few-shot segmentation. In ICCV, 2019.
  48. Prototypical networks for few-shot learning. NeurIPS, 2017.
  49. Box-driven class-wise region masking and filling rate guided loss for weakly supervised semantic segmentation. In CVPR, 2019.
  50. Context decoupling augmentation for weakly supervised semantic segmentation. In ICCV, 2021.
  51. Mining cross-image semantics for weakly supervised semantic segmentation. In ECCV, 2020.
  52. Ecs-net: Improving weakly supervised semantic segmentation by using connections between class activation maps. In ICCV, 2021.
  53. All-pairs consistency learning forweakly supervised semantic segmentation. In ICCV, 2023.
  54. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016.
  55. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008.
  56. Learning random-walk label propagation for weakly-supervised semantic segmentation. In CVPR, 2017.
  57. Treating pseudo-labels generation as image matting for weakly supervised semantic segmentation. In ICCV, 2023a.
  58. Panet: Few-shot image semantic segmentation with prototype alignment. In ICCV, 2019.
  59. Hunting sparsity: Density-guided contrastive learning for semi-supervised semantic segmentation. In CVPR, 2023b.
  60. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In CVPR, 2020.
  61. Revisiting dilated convolution: A simple approach for weakly-and semi-supervised semantic segmentation. In CVPR, 2018.
  62. Embedded discriminative attention mechanism for weakly supervised semantic segmentation. In CVPR, 2021.
  63. Adaptive spatial-bce loss for weakly supervised semantic segmentation. In ECCV, 2022.
  64. Clims: cross language image matching for weakly supervised semantic segmentation. In CVPR, 2022.
  65. Semi-supervised semantic segmentation with prototype-based consistency regularization. In NeurIPS, 2022a.
  66. Multi-class token transformer for weakly supervised semantic segmentation. In CVPR, 2022b.
  67. Attribute prototype network for zero-shot learning. In NeurIPS, 2020.
  68. Adversarial erasing framework via triplet with gated pyramid pooling layer for weakly supervised semantic segmentation. In ECCV, 2022.
  69. Causal intervention for weakly-supervised semantic segmentation. In NeurIPS, 2020a.
  70. Multi-granular semantic mining for weakly supervised semantic segmentation. In ACM MM, 2022.
  71. Inter-image communication for weakly supervised localization. In ECCV, 2020b.
  72. Sfc: Shared feature calibration in weakly supervised semantic segmentation. In AAAI, 2024.
  73. Dual adaptive representation alignment for cross-domain few-shot learning. TPAMI, 2023.
  74. Learning deep features for discriminative localization. In CVPR, 2016.
  75. Rethinking semantic segmentation: A prototype view. In CVPR, 2022a.
  76. Regional semantic contrast and aggregation for weakly supervised semantic segmentation. In CVPR, 2022b.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now