Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentation (2310.09828v2)

Published 15 Oct 2023 in cs.CV

Abstract: Weakly Supervised Semantic Segmentation (WSSS) using only image-level labels has gained significant attention due to cost-effectiveness. Recently, Vision Transformer (ViT) based methods without class activation map (CAM) have shown greater capability in generating reliable pseudo labels than previous methods using CAM. However, the current ViT-based methods utilize max pooling to select the patch with the highest prediction score to map the patch-level classification to the image-level one, which may affect the quality of pseudo labels due to the inaccurate classification of the patches. In this paper, we introduce a novel ViT-based WSSS method named top-K pooling with patch contrastive learning (TKP-PCL), which employs a top-K pooling layer to alleviate the limitations of previous max pooling selection. A patch contrastive error (PCE) is also proposed to enhance the patch embeddings to further improve the final results. The experimental results show that our approach is very efficient and outperforms other state-of-the-art WSSS methods on the PASCAL VOC 2012 dataset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. “Learning deep features for discriminative localization,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2016, pp. 2921–2929.
  2. “Class re-activation maps for weakly-supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 969–978.
  3. “Single-stage semantic segmentation from image labels,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2020, pp. 4253–4262.
  4. “Weakly supervised learning of instance segmentation with inter-pixel relations,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2019, pp. 2209–2218.
  5. “Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2021, pp. 4071–4080.
  6. “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
  7. “Max pooling with vision transformers reconciles class and shape in weakly supervised semantic segmentation,” in Eur. Conf. Comput. Vis., 2022, pp. 446–463.
  8. “Token contrast for weakly-supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2023, pp. 3093–3102.
  9. “Multi-class token transformer for weakly supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 4310–4319.
  10. “Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 16846–16855.
  11. “The pascal visual object classes (VOC) challenge,” Int. J. Comput. Vis., vol. 88, pp. 303–338, 2010.
  12. “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014, pp. 740–755.
  13. “Efficient inference in fully connected crfs with gaussian edge potentials,” in Int. Conf. Neur. Info. Process. Sys., 2011, vol. 24.
  14. “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, 2017.
  15. “Object region mining with adversarial erasing: A simple classification to semantic segmentation approach,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1568–1576.
  16. “Integral object mining via online attention accumulation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 2070–2079.
  17. “Mining cross-image semantics for weakly supervised semantic segmentation,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer, 2020, pp. 347–365.
  18. “Weakly-supervised semantic segmentation via sub-category exploration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8991–9000.
  19. “Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 4288–4298.
  20. “Weakly supervised semantic segmentation by pixel-to-prototype contrast,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 4320–4329.
  21. “End-to-end object detection with transformers,” in European conference on computer vision. Springer, 2020, pp. 213–229.
  22. “Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2018, pp. 4981–4990.
  23. “Pixel recurrent neural networks,” in Proceedings of The 33rd International Conference on Machine Learning, 2016, pp. 1747–1756.
  24. “Semantic contours from inverse detectors,” in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 991–998.
  25. “Seed, expand and constrain: Three principles for weakly-supervised image segmentation,” in Eur. Conf. Comput. Vis., 2016, pp. 695–711.
  26. “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Eur. Conf. Comput. Vis., 2018, pp. 801–818.
  27. “Self-attention prediction correction with channel suppression for weakly-supervised semantic segmentation,” in 2023 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2023, pp. 846–851.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now