Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Remote Sensing Object Detection with Single Point Supervision (2305.14141v2)

Published 23 May 2023 in cs.CV

Abstract: Pointly Supervised Object Detection (PSOD) has attracted considerable interests due to its lower labeling cost as compared to box-level supervised object detection. However, the complex scenes, densely packed and dynamic-scale objects in Remote Sensing (RS) images hinder the development of PSOD methods in RS field. In this paper, we make the first attempt to achieve RS object detection with single point supervision, and propose a PSOD method tailored for RS images. Specifically, we design a point label upgrader (PLUG) to generate pseudo box labels from single point labels, and then use the pseudo boxes to supervise the optimization of existing detectors. Moreover, to handle the challenge of the densely packed objects in RS images, we propose a sparse feature guided semantic prediction module which can generate high-quality semantic maps by fully exploiting informative cues from sparse objects. Extensive ablation studies on the DOTA dataset have validated the effectiveness of our method. Our method can achieve significantly better performance as compared to state-of-the-art image-level and point-level supervised detection methods, and reduce the performance gap between PSOD and box-level supervised object detection. Code is available at https://github.com/heshitian/PLUG.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. L. Hou, K. Lu, and J. Xue, “Refined one-stage oriented object detection method for remote sensing images,” IEEE TIP, vol. 31, pp. 1545–1558, 2022.
  2. G. Cheng, J. Han, P. Zhou, and D. Xu, “Learning rotation-invariant and fisher discriminative convolutional neural networks for object detection,” IEEE TIP, vol. 28, no. 1, pp. 265–278, 2018.
  3. Z. Huang, W. Li, X.-G. Xia, and R. Tao, “A general gaussian heatmap label assignment for arbitrary-oriented object detection,” IEEE TIP, vol. 31, pp. 1895–1910, 2022.
  4. T. Kong, F. Sun, H. Liu, Y. Jiang, L. Li, and J. Shi, “Foveabox: Beyound anchor-based object detection,” IEEE TIP, vol. 29, pp. 7389–7398, 2020.
  5. B. Liu, C. Xu, Z. Cui, and J. Yang, “Progressive context-dependent inference for object detection in remote sensing imagery,” IEEE TIP, 2022.
  6. W. Li, W. Wei, and L. Zhang, “Gsdet: Object detection in aerial images based on scale reasoning,” IEEE TIP, vol. 30, pp. 4599–4609, 2021.
  7. J. Han, J. Ding, J. Li, and G.-S. Xia, “Align deep features for oriented object detection,” IEEE TGRS, vol. 60, pp. 1–11, 2021.
  8. H. Bilen and A. Vedaldi, “Weakly supervised deep detection networks,” in CVPR, 2016, pp. 2846–2854.
  9. P. Tang, X. Wang, X. Bai, and W. Liu, “Multiple instance detection network with online instance classifier refinement,” in CVPR, 2017, pp. 2843–2851.
  10. P. Tang, X. Wang, S. Bai, W. Shen, X. Bai, W. Liu, and A. Yuille, “Pcl: Proposal cluster learning for weakly supervised object detection,” IEEE TPAMI, vol. 42, no. 1, pp. 176–191, 2018.
  11. Z. Zeng, B. Liu, J. Fu, H. Chao, and L. Zhang, “Wsod2: Learning bottom-up and top-down objectness distillation for weakly-supervised object detection,” in ICCV, 2019, pp. 8292–8300.
  12. X. Zhang, Y. Wei, J. Feng, Y. Yang, and T. S. Huang, “Adversarial complementary learning for weakly supervised object localization,” in CVPR, 2018, pp. 1325–1334.
  13. J. Xie, C. Luo, X. Zhu, Z. Jin, W. Lu, and L. Shen, “Online refinement of low-level feature based activation map for weakly supervised object localization,” in ICCV, 2021, pp. 132–141.
  14. D. P. Papadopoulos, J. R. Uijlings, F. Keller, and V. Ferrari, “Training object class detectors with click supervision,” in CVPR, 2017, pp. 6374–6383.
  15. Z. Ren, Z. Yu, X. Yang, M.-Y. Liu, A. G. Schwing, and J. Kautz, “Ufo 2: A unified framework towards omni-supervised object detection,” in ECCV.   Springer, 2020, pp. 288–313.
  16. P. Chen, X. Yu, X. Han, N. Hassan, K. Wang, J. Li, J. Zhao, H. Shi, Z. Han, and Q. Ye, “Point-to-box network for accurate object detection via single point supervision,” in ECCV.   Springer, 2022, pp. 51–67.
  17. X. Yu, P. Chen, D. Wu, N. Hassan, G. Li, J. Yan, H. Shi, Q. Ye, and Z. Han, “Object localization under single coarse point supervision,” in CVPR, 2022, pp. 4868–4877.
  18. K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,” ISPRS, vol. 159, pp. 296–307, 2020.
  19. X. Yang, J. Yang, J. Yan, Y. Zhang, T. Zhang, Z. Guo, X. Sun, and K. Fu, “Scrdet: Towards more robust detection for small, cluttered and rotated objects,” in ICCV, 2019, pp. 8232–8241.
  20. J. Ma, W. Shao, H. Ye, L. Wang, H. Wang, Y. Zheng, and X. Xue, “Arbitrary-oriented scene text detection via rotation proposals,” IEEE TMM, vol. 20, no. 11, pp. 3111–3122, 2018.
  21. Y. Jiang, X. Zhu, X. Wang, S. Yang, W. Li, H. Wang, P. Fu, and Z. Luo, “R 2 cnn: Rotational region cnn for arbitrarily-oriented scene text detection,” in ICPR.   IEEE, 2018, pp. 3610–3615.
  22. J. Ding, N. Xue, Y. Long, G.-S. Xia, and Q. Lu, “Learning roi transformer for oriented object detection in aerial images,” in CVPR, 2019, pp. 2849–2858.
  23. X. Xie, G. Cheng, J. Wang, X. Yao, and J. Han, “Oriented r-cnn for object detection,” in ICCV, 2021, pp. 3520–3529.
  24. H. Wei, Y. Zhang, Z. Chang, H. Li, H. Wang, and X. Sun, “Oriented objects as pairs of middle lines,” ISPRS, vol. 169, pp. 268–279, 2020.
  25. H. Wei, Y. Zhang, B. Wang, Y. Yang, H. Li, and H. Wang, “X-linenet: Detecting aircraft in remote sensing images by a pair of intersecting line segments,” IEEE TGRS, vol. 59, no. 2, pp. 1645–1659, 2020.
  26. J. Yi, P. Wu, B. Liu, Q. Huang, H. Qu, and D. Metaxas, “Oriented object detection in aerial images with box boundary-aware vectors,” in WACV, 2021, pp. 2150–2159.
  27. P. Zhao, Z. Qu, Y. Bu, W. Tan, and Q. Guan, “Polardet: A fast, more precise detector for rotated target in aerial images,” IJRS, vol. 42, no. 15, pp. 5831–5861, 2021.
  28. P. Dai, S. Yao, Z. Li, S. Zhang, and X. Cao, “Ace: Anchor-free corner evolution for real-time arbitrarily-oriented object detection,” IEEE TIP, vol. 31, pp. 4076–4089, 2022.
  29. X. Yang, J. Yan, Q. Ming, W. Wang, X. Zhang, and Q. Tian, “Rethinking rotated object detection with gaussian wasserstein distance loss,” in ICML.   PMLR, 2021, pp. 11 830–11 841.
  30. X. Yang, Y. Zhou, G. Zhang, J. Yang, W. Wang, J. Yan, X. Zhang, and Q. Tian, “The kfiou loss for rotated object detection,” arXiv preprint arXiv:2201.12558, 2022.
  31. J. Han, J. Ding, N. Xue, and G.-S. Xia, “Redet: A rotation-equivariant detector for aerial object detection,” in CVPR, 2021, pp. 2786–2795.
  32. X. Yang, J. Yan, Z. Feng, and T. He, “R3det: Refined single-stage detector with feature refinement for rotating object,” in AAAI, vol. 35, no. 4, 2021, pp. 3163–3171.
  33. W. Qian, X. Yang, S. Peng, J. Yan, and Y. Guo, “Learning modulated loss for rotated object detection,” in AAAI, vol. 35, no. 3, 2021, pp. 2458–2466.
  34. X. Yang and J. Yan, “Arbitrary-oriented object detection with circular smooth label,” in ECCV.   Springer, 2020, pp. 677–694.
  35. X. Yang, L. Hou, Y. Zhou, W. Wang, and J. Yan, “Dense label encoding for boundary discontinuity free rotation detection,” in CVPR, 2021, pp. 15 819–15 829.
  36. H. Hu, J. Gu, Z. Zhang, J. Dai, and Y. Wei, “Relation networks for object detection,” in CVPR, 2018, pp. 3588–3597.
  37. C. Deng, M. Wang, L. Liu, Y. Liu, and Y. Jiang, “Extended feature pyramid network for small object detection,” IEEE TMM, vol. 24, pp. 1968–1979, 2021.
  38. F. Yang, H. Fan, P. Chu, E. Blasch, and H. Ling, “Clustered object detection in aerial images,” in ICCV, 2019, pp. 8311–8320.
  39. C. Li, T. Yang, S. Zhu, C. Chen, and S. Guan, “Density map guided object detection in aerial images,” in CVPRW, 2020, pp. 190–191.
  40. K. Fu, Z. Chang, Y. Zhang, and X. Sun, “Point-based estimator for arbitrary-oriented object detection in aerial images,” IEEE TGRS, vol. 59, no. 5, pp. 4370–4387, 2020.
  41. E. Liu, Y. Zheng, B. Pan, X. Xu, and Z. Shi, “Dcl-net: Augmenting the capability of classification and localization for remote sensing object detection,” IEEE TGRS, vol. 59, no. 9, pp. 7933–7944, 2021.
  42. C. Xu, J. Wang, W. Yang, H. Yu, L. Yu, and G.-S. Xia, “Rfla: Gaussian receptive field based label assignment for tiny object detection,” in ECCV.   Springer, 2022, pp. 526–543.
  43. Y. Zhu, J. Du, and X. Wu, “Adaptive period embedding for representing oriented objects in aerial images,” IEEE TGRS, vol. 58, no. 10, pp. 7247–7257, 2020.
  44. S. Liu, L. Zhang, H. Lu, and Y. He, “Center-boundary dual attention for oriented object detection in remote sensing images,” IEEE TGRS, vol. 60, pp. 1–14, 2021.
  45. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV.   Springer, 2020, pp. 213–229.
  46. X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” arXiv preprint arXiv:2010.04159, 2020.
  47. F. Li, H. Zhang, S. Liu, J. Guo, L. M. Ni, and L. Zhang, “Dn-detr: Accelerate detr training by introducing query denoising,” in CVPR, 2022, pp. 13 619–13 627.
  48. L. Dai, H. Liu, H. Tang, Z. Wu, and P. Song, “Ao2-detr: Arbitrary-oriented object detection transformer,” IEEE TCSVT, 2022.
  49. D. Wang, Q. Zhang, Y. Xu, J. Zhang, B. Du, D. Tao, and L. Zhang, “Advancing plain vision transformer towards remote sensing foundation model,” IEEE TGRS, 2022.
  50. C. Fasana, S. Pasini, F. Milani, and P. Fraternali, “Weakly supervised object detection for remote sensing images: A survey,” Remote Sensing, vol. 14, no. 21, p. 5362, 2022.
  51. G. Cheng, J. Yang, D. Gao, L. Guo, and J. Han, “High-quality proposals for weakly supervised object detection,” IEEE TIP, vol. 29, pp. 5794–5804, 2020.
  52. X. Feng, J. Han, X. Yao, and G. Cheng, “Progressive contextual instance refinement for weakly supervised object detection in remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 11, pp. 8002–8012, 2020.
  53. Y. Li, Y. Zhang, X. Huang, and A. L. Yuille, “Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images,” ISPRS journal of photogrammetry and remote sensing, vol. 146, pp. 182–196, 2018.
  54. F. Shao, L. Chen, J. Shao, W. Ji, S. Xiao, L. Ye, Y. Zhuang, and J. Xiao, “Deep learning for weakly-supervised object detection and localization: A survey,” Neurocomputing, vol. 496, pp. 192–207, 2022.
  55. J. R. Uijlings, K. E. Van De Sande, T. Gevers, and A. W. Smeulders, “Selective search for object recognition,” IJCV, vol. 104, pp. 154–171, 2013.
  56. C. L. Zitnick and P. Dollár, “Edge boxes: Locating object proposals from edges,” in ECCV.   Springer, 2014, pp. 391–405.
  57. D. Zhang, J. Han, G. Cheng, Z. Liu, S. Bu, and L. Guo, “Weakly supervised learning for target detection in remote sensing images,” IEEE Geoscience and Remote Sensing Letters, vol. 12, no. 4, pp. 701–705, 2014.
  58. J. Han, D. Zhang, G. Cheng, L. Guo, and J. Ren, “Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 6, pp. 3325–3337, 2014.
  59. X. Feng, J. Han, X. Yao, and G. Cheng, “Tcanet: Triple context-aware network for weakly supervised object detection in remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 8, pp. 6946–6955, 2020.
  60. X. Qian, C. Li, W. Wang, X. Yao, and G. Cheng, “Semantic segmentation guided pseudo label mining and instance re-detection for weakly supervised object detection in remote sensing images,” International Journal of Applied Earth Observation and Geoinformation, vol. 119, p. 103301, 2023.
  61. B. Wang, Y. Zhao, and X. Li, “Multiple instance graph learning for weakly supervised remote sensing object detection,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–12, 2021.
  62. X. Feng, X. Yao, G. Cheng, and J. Han, “Weakly supervised rotation-invariant aerial object detection network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 14 146–14 155.
  63. G. Wang, X. Zhang, Z. Peng, X. Jia, X. Tang, and L. Jiao, “Mol: Towards accurate weakly supervised remote sensing object detection via multi-view noisy learning,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 196, pp. 457–470, 2023.
  64. X. Yao, X. Feng, J. Han, G. Cheng, and L. Guo, “Automatic weakly supervised object detection from high spatial resolution remote sensing images via dynamic curriculum learning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, pp. 675–685, 2020.
  65. X. Qian, Y. Huo, G. Cheng, X. Yao, K. Li, H. Ren, and W. Wang, “Incorporating the completeness and difficulty of proposals into weakly supervised object detection in remote sensing images,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 1902–1911, 2022.
  66. A. Bearman, O. Russakovsky, V. Ferrari, and L. Fei-Fei, “What’s the point: Semantic segmentation with point supervision,” in ECCV.   Springer, 2016, pp. 549–565.
  67. R. Qian, Y. Wei, H. Shi, J. Li, J. Liu, and T. Huang, “Weakly supervised scene parsing with point-based distance metric learning,” in AAAI, vol. 33, no. 01, 2019, pp. 8843–8850.
  68. L. Wu, L. Fang, J. Yue, B. Zhang, P. Ghamisi, and M. He, “Deep bilateral filtering network for point-supervised semantic segmentation in remote sensing images,” IEEE TIP, vol. 31, pp. 7419–7434, 2022.
  69. I. H. Laradji, N. Rostamzadeh, P. O. Pinheiro, D. Vazquez, and M. Schmidt, “Proposal-based instance segmentation with point supervision,” in ICIP.   IEEE, 2020, pp. 2126–2130.
  70. B. Cheng, O. Parkhi, and A. Kirillov, “Pointly-supervised instance segmentation,” in CVPR, 2022, pp. 2617–2626.
  71. M. Liao, Z. Guo, Y. Wang, P. Yuan, B. Feng, and F. Wan, “Attentionshift: Iteratively estimated part-based attention map for pointly supervised instance segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19 519–19 528.
  72. J. Fan, Z. Zhang, and T. Tan, “Pointly-supervised panoptic segmentation,” in ECCV.   Springer, 2022, pp. 319–336.
  73. J. Ribera, D. Guera, Y. Chen, and E. J. Delp, “Locating objects without bounding boxes,” in CVPR, 2019, pp. 6479–6489.
  74. Q. Song, C. Wang, Z. Jiang, Y. Wang, Y. Tai, C. Wang, J. Li, F. Huang, and Y. Wu, “Rethinking counting and localization in crowds: A purely point-based framework,” in ICCV, 2021, pp. 3365–3374.
  75. X. Ying, L. Liu, Y. Wang, R. Li, N. Chen, Z. Lin, W. Sheng, and S. Zhou, “Mapping degeneration meets label evolution: Learning infrared small target detection with single point supervision,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 15 528–15 538.
  76. B. Li, Y. Wang, L. Wang, F. Zhang, T. Liu, Z. Lin, W. An, and Y. Guo, “Monte carlo linear clustering with single-point supervision is enough for infrared small target detection,” arXiv preprint arXiv:2304.04442, 2023.
  77. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in CVPR, 2017, pp. 2117–2125.
  78. T. Lin, P. Goyal, R. B. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” CoRR, vol. abs/1708.02002, 2017. [Online]. Available: http://arxiv.org/abs/1708.02002
  79. S. Waqas Zamir, A. Arora, A. Gupta, S. Khan, G. Sun, F. Shahbaz Khan, F. Zhu, L. Shao, G.-S. Xia, and X. Bai, “isaid: A large-scale dataset for instance segmentation in aerial images,” in CVPRW, 2019, pp. 28–37.
  80. K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open mmlab detection toolbox and benchmark,” arXiv preprint arXiv:1906.07155, 2019.
  81. H. Robbins and S. Monro, “A stochastic approximation method,” AMS, pp. 400–407, 1951.
  82. Z. Tian, C. Shen, X. Wang, and H. Chen, “Boxinst: High-performance instance segmentation with box annotations,” in CVPR, 2021, pp. 5443–5452.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now