SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation (2401.11719v1)
Abstract: Image-level weakly supervised semantic segmentation has received increasing attention due to its low annotation cost. Existing methods mainly rely on Class Activation Mapping (CAM) to obtain pseudo-labels for training semantic segmentation models. In this work, we are the first to demonstrate that long-tailed distribution in training data can cause the CAM calculated through classifier weights over-activated for head classes and under-activated for tail classes due to the shared features among head- and tail- classes. This degrades pseudo-label quality and further influences final semantic segmentation performance. To address this issue, we propose a Shared Feature Calibration (SFC) method for CAM generation. Specifically, we leverage the class prototypes that carry positive shared features and propose a Multi-Scaled Distribution-Weighted (MSDW) consistency loss for narrowing the gap between the CAMs generated through classifier weights and class prototypes during training. The MSDW loss counterbalances over-activation and under-activation by calibrating the shared features in head-/tail-class classifier weights. Experimental results show that our SFC significantly improves CAM boundaries and achieves new state-of-the-art performances. The project is available at https://github.com/Barrett-python/SFC.
- Weakly supervised learning of instance segmentation with inter-pixel relations. In CVPR.
- Saving 100x Storage: Prototype Replay for Reconstructing Training Sample Distribution in Class-Incremental Semantic Segmentation. In NeurIPS.
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. PAMI, 40: 834–848.
- Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation. In CVPR.
- Class re-activation maps for weakly-supervised semantic segmentation. In CVPR.
- Imagenet: A large-scale hierarchical image database. In CVPR.
- Weakly supervised semantic segmentation by pixel-to-prototype contrast. In CVPR.
- The pascal visual object classes (voc) challenge. IJCV, 88: 303–338.
- Semantic contours from inverse detectors. In ICCV.
- Deep residual learning for image recognition. In CVPR.
- Batchformer: Learning to explore sample relationships for robust representation learning. In CVPR.
- Efficient inference in fully connected crfs with gaussian edge potentials. In NeurIPS.
- Reducing information bottleneck for weakly supervised semantic segmentation. In NeurIPS.
- Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation. In CVPR.
- Threshold matters in WSSS: manipulating the activation for the robust and accurate segmentation model against thresholds. In CVPR.
- Railroad is not a train: Saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In CVPR.
- Expansion and shrinkage of localization for weakly-supervised semantic segmentation. In NeurIPS.
- Group based deep shared feature learning for fine-grained image classification. arXiv preprint arXiv:2004.01817.
- Group-wise semantic mining for weakly supervised semantic segmentation. In AAAI.
- Microsoft coco: Common objects in context. In ECCV.
- Clip is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. In CVPR.
- Infrared and Visible Cross-Modal Image Retrieval Through Shared Features. TCSVT, 31: 4485–4496.
- Fully convolutional networks for semantic segmentation. In CVPR.
- Image segmentation using deep learning: A survey. PAMI, 44(7): 3523–3542.
- Object-part attention model for fine-grained image classification. TIP, 27(3): 1487–1500.
- Mining cross-image semantics for weakly supervised semantic segmentation. In ECCV.
- Equalization loss v2: A new gradient balance approach for long-tailed object detection. In CVPR.
- Proper reuse of image classification features improves object detection. In CVPR, 13628–13637.
- Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In CVPR.
- Distribution-balanced loss for multi-label classification in long-tailed datasets. In ECCV.
- CLIMS: cross language image matching for weakly supervised semantic segmentation. In CVPR.
- Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation. In CVPR.
- Multi-class token transformer for weakly supervised semantic segmentation. In CVPR.
- Self correspondence distillation for end-to-end weakly-supervised semantic segmentation. In AAAI.
- AutoBD: Automated bi-level description for scalable fine-grained visual categorization. TIP, 27(1): 10–23.
- Non-salient region object mining for weakly supervised semantic segmentation. In CVPR.
- Adversarial erasing framework via triplet with gated pyramid pooling layer for weakly supervised semantic segmentation. In ECCV.
- Affinity attention graph neural network for weakly supervised semantic segmentation. PAMI, 44(11): 8082–8096.
- Credible Dual-Expert Learning for Weakly Supervised Semantic Segmentation. IJCV, 131: 1892 – 1908.
- Causal intervention for weakly-supervised semantic segmentation. In NeurIPS.
- Complementary patch for weakly supervised semantic segmentation. In ICCV.
- SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model. arXiv preprint arXiv:2303.05118.
- Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation. NeurIPS, 35: 24340–24353.
- Learning multi-attention convolutional neural network for fine-grained image recognition. In ICCV.
- Learning deep features for discriminative localization. In CVPR.
- CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservation. In ICCV, 22257–22267.