Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

P-NOC: adversarial training of CAM generating networks for robust weakly supervised semantic segmentation priors (2305.12522v3)

Published 21 May 2023 in cs.CV and cs.LG

Abstract: Weakly Supervised Semantic Segmentation (WSSS) techniques explore individual regularization strategies to refine Class Activation Maps (CAMs). In this work, we first analyze complementary WSSS techniques in the literature, their segmentation properties, and the conditions in which they are most effective. Based on these findings, we devise two new techniques: P-NOC and CCAM-H. In the first, we promote the conjoint training of two adversarial CAM generating networks: the generator, which progressively learns to erase regions containing class-specific features, and a discriminator, which is refined to gradually shift its attention to new class discriminant features. In the latter, we employ the high quality pseudo-segmentation priors produced by P-NOC to guide the learning to saliency information in a weakly supervised fashion. Finally, we employ both pseudo-segmentation priors and pseudo-saliency proposals in the random walk procedure, resulting in higher quality pseudo-semantic segmentation masks, and competitive results with the state of the art.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Review the state-of-the-art technologies of semantic segmentation based on deep learning, Neurocomputing 493 (2022) 626–646.
  2. Importance-aware semantic segmentation in self-driving with discrete wasserstein training, in: AAAI Conference on Artificial Intelligence, volume 34, 2020, pp. 11629–11636.
  3. F. Gelana, A. Yadav, Firearm detection from surveillance cameras using image processing and machine learning techniques, in: Smart Innovations in Communication and Computational Sciences, Springer, 2019, pp. 25–34.
  4. Detecting objects using color and depth segmentation with kinect sensor, Procedia Technology 3 (2012) 196–204.
  5. Vegetation land use/land cover extraction from high-resolution satellite images based on adaptive context inference, IEEE Access 8 (2020) 21036–21051.
  6. Capsules for biomedical image segmentation, Medical Image Analysis 68 (2021) 101889.
  7. Skin lesion segmentation using high-resolution convolutional neural network, Computer Methods and Programs in Biomedicine 186 (2020) 105241.
  8. Research of improving semantic image segmentation based on a feature fusion model, Journal of Ambient Intelligence and Humanized Computing (2020) 1–13.
  9. A brief survey on weakly supervised semantic segmentation, International Journal of Online & Biomedical Engineering 18 (2022).
  10. A survey of semi-and weakly supervised semantic segmentation of images, Artificial Intelligence Review 53 (2020) 4259–4288.
  11. Learning deep features for discriminative localization, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2921–2929. doi:10.1109/CVPR.2016.319.
  12. Grad-CAM: Visual explanations from deep networks via gradient-based localization, in: IEEE/CVF International Conference on Computer Vision (ICCV), 2017, pp. 618–626. doi:10.1109/ICCV.2017.74.
  13. G. Vilone, L. Longo, Explainable artificial intelligence: a systematic review, arXiv preprint arXiv:2006.00093 (2020).
  14. S. Jo, I.-J. Yu, Puzzle-cam: Improved localization via matching partial and full features, in: IEEE International Conference on Image Processing (ICIP), 2021, pp. 639–643.
  15. Tell me where to look: Guided attention inference network, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9215–9223.
  16. Object region mining with adversarial erasing: A simple classification to semantic segmentation approach, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1568–1576.
  17. Unlocking the potential of ordinary classifier: Class-specific adversarial erasing framework for weakly supervised semantic segmentation, in: IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 6974–6983.
  18. Reducing information bottleneck for weakly supervised semantic segmentation, Advances in Neural Information Processing Systems 34 (2021) 27408–27421.
  19. C²AM: Contrastive learning of class-agnostic activation map for weakly supervised object localization and semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 989–998.
  20. When does label smoothing help?, Advances in Neural Information Processing Systems (NIPS) 32 (2019).
  21. Cutmix: Regularization strategy to train strong classifiers with localizable features, in: IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 6023–6032.
  22. J. Ahn, S. Kwak, Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4981–4990.
  23. The PASCAL Visual Object Classes Challenge: A retrospective, International Journal of Computer Vision 111 (2015) 98–136.
  24. Microsoft COCO: Common objects in context, in: European Conference on Computer Vision (ECVV), Springer, 2014, pp. 740–755.
  25. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 12275–12284.
  26. Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 4283–4292.
  27. Reducing information bottleneck for weakly supervised semantic segmentation, Advances in Neural Information Processing Systems (NeurIPS) 34 (2021a) 27408–27421.
  28. Railroad is not a train: Saliency as pseudo-pixel supervision for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021b, pp. 5495–5505.
  29. L2g: A simple local-to-global knowledge transfer framework for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 16886–16896.
  30. P. Krähenbühl, V. Koltun, Efficient inference in fully connected CRFs with gaussian edge potentials, Advances in Neural Information Processing Systems (NeurIPS) 24 (2011) 109–117.
  31. Weakly supervised learning of instance segmentation with inter-pixel relations, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2209–2218.
  32. Uncertainty estimation via response scaling for pseudo-mask noise mitigation in weakly-supervised semantic segmentation, in: AAAI Conference on Artificial Intelligence, volume 36, 2022, pp. 1447–1455.
  33. Randaugment: Practical automated data augmentation with a reduced search space, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020, pp. 702–703.
  34. Discovering class-specific pixels for weakly-supervised semantic segmentation, arXiv preprint arXiv:1707.05821 (2017).
  35. Exploiting saliency for object segmentation from image level labels, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5038–5047.
  36. F. Sun, W. Li, Saliency guided deep network for weakly-supervised image segmentation, Pattern Recognition Letters 120 (2019) 62–68.
  37. Semi-supervised semantic segmentation using unreliable pseudo-labels, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 4238–4247.
  38. Rethinking the inception architecture for computer vision, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2818–2826.
  39. A simple pooling-based design for real-time salient object detection, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3917–3926.
  40. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 40 (2017) 834–848.
  41. Encoder-decoder with atrous separable convolution for semantic image segmentation, in: European Conference on Computer Vision (ECCV), 2018, pp. 801–818.
  42. Imagenet-21k pretraining for the masses, arXiv preprint arXiv:2104.10972 (2021).
  43. Causal intervention for weakly-supervised semantic segmentation, Advances in Neural Information Processing Systems 33 (2020) 655–666.
  44. Adaptive early-learning correction for segmentation from noisy annotations, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 2606–2616.
  45. Multi-class token transformer for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 4300–4309.
  46. Token contrast for weakly-supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 3093–3102.
  47. Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 4071–4080.
  48. Activation modulation and recalibration scheme for weakly supervised semantic segmentation, in: AAAI Conference on Artificial Intelligence, volume 36, 2022, pp. 2117–2125.
  49. Threshold matters in wsss: Manipulating the activation for the robust and accurate segmentation model against thresholds, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 4330–4339.
  50. Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 4288–4298.
  51. Weakly-supervised semantic segmentation with superpixel guided local and global consistency, Pattern Recognition 124 (2022) 108504.
  52. Max pooling with vision transformers reconciles class and shape in weakly supervised semantic segmentation, in: European Conference on Computer Vision (ECCV), Springer, 2022, pp. 446–463.
  53. Pseudo-mask matters in weakly-supervised semantic segmentation, in: IEEE/CVF International Conference on Computer Vision (CVPR), 2021, pp. 6964–6973.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Lucas David (2 papers)
  2. Helio Pedrini (30 papers)
  3. Zanoni Dias (5 papers)

Summary

We haven't generated a summary for this paper yet.