Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Knowledge-guided Causal Intervention for Weakly-supervised Object Localization (2301.01060v2)

Published 3 Jan 2023 in cs.CV

Abstract: Previous weakly-supervised object localization (WSOL) methods aim to expand activation map discriminative areas to cover the whole objects, yet neglect two inherent challenges when relying solely on image-level labels. First, the entangled context'' issue arises from object-context co-occurrence (\eg, fish and water), making the model inspection hard to distinguish object boundaries clearly. Second, theC-L dilemma'' issue results from the information decay caused by the pooling layers, which struggle to retain both the semantic information for precise classification and those essential details for accurate localization, leading to a trade-off in performance. In this paper, we propose a knowledge-guided causal intervention method, dubbed KG-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention, which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the disentangled object feature, we introduce a multi-source knowledge guidance framework to strike a balance between absorbing classification knowledge and localization knowledge during model training. Extensive experiments conducted on several benchmark datasets demonstrate the effectiveness of KG-CI-CAM in learning distinct object boundaries amidst confounding contexts and mitigating the dilemma between classification and localization performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Where to look?: Mining complementary image regions for weakly supervised object localization. In WACV, 2021.
  2. Rethinking class activation mapping for weakly supervised object localization. In ECCV, 2020.
  3. Online knowledge distillation with diverse peers. In AAAI, 2020.
  4. Evaluating weakly supervised object localization methods right. In CVPR, 2020.
  5. Attention-based dropout layer for weakly supervised object localization. In CVPR, 2019.
  6. Weakly supervised cascaded convolutional networks. In CVPR, 2017.
  7. Judea pearl: Causality: Models, reasoning, and inference. Politische Vierteljahresschrift, 2001.
  8. Residual error based knowledge distillation. Neurocomputing, 2021.
  9. Knowledge distillation: A survey. IJCV, 2021.
  10. Strengthen learning tolerance for weakly supervised object localization. In CVPR, 2021.
  11. Deep residual learning for image recognition. In CVPR, 2016.
  12. Knowledge distillation with adversarial samples supporting decision boundary. In AAAI, 2019.
  13. Distilling the knowledge in a neural network. arXiv, 2(7), 2015.
  14. Causal inference for leveraging image-text matching bias in multi-modal fake news detection. TKDE, 2022.
  15. Like what you like: Knowledge distill via neuron selectivity transfer. arXiv, 2017.
  16. Two-phase learning for weakly supervised object localization. In ICCV, 2017.
  17. Bridging the gap between classification and localization for weakly supervised object localization. In CVPR, 2022.
  18. Normalization matters in weakly supervised object localization. In ICCV, 2021.
  19. Adam: A method for stochastic optimization. arXiv, 2014.
  20. Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization. In ICCV, 2017.
  21. Self-referenced deep learning. In ACCV, 2018.
  22. Bgae: Auto-encoding multi-view bipartite graph clustering. TKDE, 2024.
  23. Multi-view bipartite graph clustering with coupled noisy feature filter. TKDE, 2023.
  24. Combining graph neural networks with expert knowledge for smart contract vulnerability detection. TKDE, 2021.
  25. Geometry constrained weakly supervised object localization. In ECCV, 2020.
  26. Adversarial style mining for one-shot unsupervised domain adaptation. In NIPS, 2020.
  27. Category-level adversarial adaptation for semantic segmentation using purified features. TPAMI, 2021.
  28. Yawei Luo and Yi Yang. Large language model and domain-specific model collaboration for smart education. FITEE, 2024.
  29. Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In CVPR, 2019.
  30. Macro-micro adversarial network for human parsing. In ECCV, 2018.
  31. Erasing integrated learning: A simple yet effective approach for weakly supervised object localization. In CVPR, 2020.
  32. Foreground activation maps for weakly supervised object localization. In ICCV, 2021.
  33. Improved knowledge distillation via teacher assistant. In AAAI, 2020.
  34. Unveiling the potential of structure preserving for weakly supervised object localization. In CVPR, 2021.
  35. Causal inference in statistics: A primer. 2016.
  36. Imagenet large scale visual recognition challenge. IJCV, 2015.
  37. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.
  38. Deep learning for weakly-supervised object detection and localization: A survey. Neurocomputing, 2022.
  39. Counterfactual co-occurring learning for bias mitigation in weakly-supervised object localization. arXiv, 2024.
  40. Active learning for point cloud semantic segmentation via spatial-structural diversity reasoning. In ACM MM, 2022.
  41. Improving weakly supervised object localization via causal intervention. In ACM MM, 2021.
  42. Very deep convolutional networks for large-scale image recognition. In arXiv, 2014.
  43. Going deeper with convolutions. In CVPR, 2015.
  44. Rethinking the inception architecture for computer vision. In CVPR, 2016.
  45. Long-tailed classification by keeping the good and removing the bad momentum causal effect. NIPS, 2020.
  46. Informed machine learning–a taxonomy and survey of integrating prior knowledge into learning systems. TKDE, 2021.
  47. The caltech-ucsd birds-200-2011 dataset. 2011.
  48. Knowledge graph embedding: A survey of approaches and applications. TKDE, 2017.
  49. Looking beyond single images for weakly supervised semantic segmentation learning. TPAMI, 2022.
  50. Ts2c: Tight box mining with surrounding segmentation context for weakly supervised object detection. In ECCV, 2018.
  51. Caltech-ucsd birds 200. 2010.
  52. Peer collaborative learning for online knowledge distillation. In AAAI, 2021.
  53. Background activation suppression for weakly supervised object localization. In CVPR, 2022.
  54. Online refinement of low-level feature based activation map for weakly supervised object localization. In ICCV, 2021.
  55. Cream: Weakly supervised object localization via class re-activation mapping. In CVPR, 2022.
  56. Show, attend and tell: Neural image caption generation with visual attention. In ICML, 2015.
  57. Combinational class activation maps for weakly supervised object localization. In WACV, 2020.
  58. Interventional few-shot learning. NIPS, 2020.
  59. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, 2019.
  60. Regularizing class-wise predictions via self-knowledge distillation. In CVPR, 2020.
  61. Rethinking the route towards weakly supervised object localization. In CVPR, 2020.
  62. Causal intervention for weakly-supervised semantic segmentation. NIPS, 2020.
  63. Adversarial complementary learning for weakly supervised object localization. In CVPR, 2018.
  64. Self-produced guidance for weakly-supervised object localization. In ECCV, 2018.
  65. Inter-image communication for weakly supervised localization. In ECCV, 2020.
  66. Localization distillation for object detection. arXiv, 2022.
  67. Learning deep features for discriminative localization. In CVPR, 2016.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now