Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Task-wise Sampling Convolutions for Arbitrary-Oriented Object Detection in Aerial Images (2209.02200v3)

Published 6 Sep 2022 in cs.CV

Abstract: Arbitrary-oriented object detection (AOOD) has been widely applied to locate and classify objects with diverse orientations in remote sensing images. However, the inconsistent features for the localization and classification tasks in AOOD models may lead to ambiguity and low-quality object predictions, which constrains the detection performance. In this article, an AOOD method called task-wise sampling convolutions (TS-Conv) is proposed. TS-Conv adaptively samples task-wise features from respective sensitive regions and maps these features together in alignment to guide a dynamic label assignment for better predictions. Specifically, sampling positions of the localization convolution in TS-Conv are supervised by the oriented bounding box (OBB) prediction associated with spatial coordinates, while sampling positions and convolutional kernel of the classification convolution are designed to be adaptively adjusted according to different orientations for improving the orientation robustness of features. Furthermore, a dynamic task-consistent-aware label assignment (DTLA) strategy is developed to select optimal candidate positions and assign labels dynamically according to ranked task-aware scores obtained from TS-Conv. Extensive experiments on several public datasets covering multiple scenes, multimodal images, and multiple categories of objects demonstrate the effectiveness, scalability, and superior performance of the proposed TS-Conv.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. G.-S. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, “DOTA: A large-scale dataset for object detection in aerial images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake City, Utah, USA, June 2018, pp. 3974–3983.
  2. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, 2017.
  3. J. Bai, J. Ren, Z. Xiao, Z. Chen, C. Gao, T. A. A. Ali, and L. Jiao, “Localizing from classification: self-directed weakly supervised object localization for remote sensing images,” IEEE Trans. Neural Netw. Learn. Syst., pp. 1–15, 2023.
  4. J. Ding, N. Xue, Y. Long, G.-S. Xia, and Q. Lu, “Learning RoI transformer for oriented object detection in aerial images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Long Beach, CA, USA, June 2019, pp. 2849–2858.
  5. Y. Xu, M. Fu, Q. Wang, Y. Wang, K. Chen, G.-S. Xia, and X. Bai, “Gliding vertex on the horizontal bounding box for multi-oriented object detection,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, Feb. 2020.
  6. X. Yang, Q. Liu, J. Yan, and A. Li, “R3Det: Refined single-stage detector with feature refinement for rotating object,” arXiv preprint arXiv:1908.05612, 2019.
  7. Z. Huang, W. Li, X.-G. Xia, and R. Tao, “A general gaussian heatmap label assignment for arbitrary-oriented object detection,” IEEE Trans. Image Process., vol. 31, pp. 1895–1910, 2022.
  8. B. Jiang, R. Luo, J. Mao, T. Xiao, and Y. Jiang, “Acquisition of localization confidence for accurate object detection,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 784–799.
  9. G. Song, Y. Liu, and X. Wang, “Revisiting the sibling head in object detector,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 11 563–11 572.
  10. Z. Piao, L. Tang, and B. Zhao, “Unsupervised domain-adaptive object detection via localization regression alignment,” IEEE Trans. Neural Netw. Learn. Syst., pp. 1–15, 2023.
  11. J. Han, J. Ding, N. Xue, and G.-S. Xia, “ReDet: A rotation-equivariant detector for aerial object detection,” arXiv preprint arXiv:2103.07733, 2021.
  12. X. Yang, J. Yang, J. Yan, Y. Zhang, T. Zhang, Z. Guo, X. Sun, and K. Fu, “SCRDet: Towards more robust detection for small, cluttered and rotated objects,” in Proc. IEEE Int. Conf. Comput. Vis., Seoul, South Korea, Oct. 2019, pp. 8232–8241.
  13. Z. Yang, S. Liu, H. Hu, L. Wang, and S. Lin, “Reppoints: Point set representation for object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 9657–9666.
  14. W. Li, Y. Chen, K. Hu, and J. Zhu, “Oriented reppoints for aerial object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 1829–1838.
  15. B. Zhu, J. Wang, Z. Jiang, F. Zong, S. Liu, Z. Li, and J. Sun, “Autoassign: Differentiable label assignment for dense object detection,” arXiv preprint arXiv:2007.03496, 2020.
  16. S. Zhang, C. Chi, Y. Yao, Z. Lei, and S. Z. Li, “Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 9759–9768.
  17. J. Han, J. Ding, J. Li, and G.-S. Xia, “Align deep features for oriented object detection,” IEEE Trans. Geosci. Remote Sens., pp. 1–11, 2021.
  18. X. Yang and J. Yan, “Arbitrary-oriented object detection with circular smooth label,” in Proc. Eur. Conf. Comput. Vis., Online, Aug. 2020, pp. 677–694.
  19. J. Yi, P. Wu, B. Liu, Q. Huang, H. Qu, and D. N. Metaxas, “Oriented object detection in aerial images with box boundary-aware vectors,” in Proc. IEEE Winter Conf. Appl. Comput. Vis., Dec. 2020, pp. 2150–2159.
  20. H. Wei, Y. Zhang, Z. Chang, H. Li, H. Wang, and X. Sun, “Oriented objects as pairs of middle lines,” ISPRS J. Photogramm. Remote Sens., vol. 169, pp. 268–279, 2020.
  21. L. Dai, H. Liu, H. Tang, Z. Wu, and P. Song, “AO2-DETR: Arbitrary-oriented object detection transformer,” arXiv preprint arXiv:2205.12785, 2022.
  22. X. Yang, J. Yan, Q. Ming, W. Wang, X. Zhang, and Q. Tian, “Rethinking rotated object detection with gaussian wasserstein distance loss,” arXiv preprint arXiv:2101.11952, 2021.
  23. X. Yang, X. Yang, J. Yang, Q. Ming, W. Wang, Q. Tian, and J. Yan, “Learning high-precision bounding box for rotated object detection via kullback-leibler divergence,” arXiv preprint arXiv:2106.01883, 2021.
  24. Z. Guo, C. Liu, X. Zhang, J. Jiao, X. Ji, and Q. Ye, “Beyond bounding-box: Convex-hull feature adaptation for oriented and densely packed object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 8788–8797.
  25. C. Xu, J. Ding, J. Wang, W. Yang, H. Yu, L. Yu, and G.-S. Xia, “Dynamic coarse-to-fine learning for oriented tiny object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., June 2023, pp. 7318–7328.
  26. Q. Ming, L. Miao, Z. Zhou, and Y. Dong, “CFC-Net: A critical feature capturing network for arbitrary-oriented object detection in remote sensing images,” arXiv preprint arXiv:2101.06849, 2021.
  27. H. Zhang, Y. Wang, F. Dayoub, and N. Sunderhauf, “Varifocalnet: An iou-aware dense object detector,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 8514–8523.
  28. Y. Wu, Y. Chen, L. Yuan, Z. Liu, L. Wang, H. Li, and Y. Fu, “Rethinking classification and localization for object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 10 186–10 195.
  29. Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “Yolox: Exceeding yolo series in 2021,” arXiv preprint arXiv:2107.08430, 2021.
  30. J. Cao, H. Cholakkal, R. M. Anwer, F. S. Khan, Y. Pang, and L. Shao, “D2det: Towards high quality object detection and instance segmentation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 11 485–11 494.
  31. X. Zhu, H. Hu, S. Lin, and J. Dai, “Deformable convnets v2: More deformable, better results,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 9308–9316.
  32. J. Wang, K. Chen, S. Yang, C. C. Loy, and D. Lin, “Region proposal by guided anchoring,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 2965–2974.
  33. J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, “Deformable convolutional networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 764–773.
  34. X. Pan, Y. Ren, K. Sheng, W. Dong, H. Yuan, X. Guo, C. Ma, and C. Xu, “Dynamic refinement network for oriented and densely packed object detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Online, June 2020, pp. 11 207–11 216.
  35. R. Liu, J. Lehman, P. Molino, F. Petroski Such, E. Frank, A. Sergeev, and J. Yosinski, “An intriguing failing of convolutional neural networks and the coordconv solution,” Adv. Neur. In., vol. 31, 2018.
  36. S.-C. Huang, Q.-V. Hoang, and T.-H. Le, “SFA-Net: A selective features absorption network for object detection in rainy weather conditions,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 8, pp. 5122–5132, 2023.
  37. Z. Shao, J. Han, and D. Marnerides, and K. Debattista, “Region-object relation-aware dense captioning via transformer,” IEEE Trans. Neural Netw. Learn. Syst., pp. 1–12, 2022.
  38. B. Yang, G. Bender, Q. V. Le, and J. Ngiam, “Condconv: Conditionally parameterized convolutions for efficient inference,” Adv. Neur. In., vol. 32, 2019.
  39. H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Long Beach, CA, USA, June 2019, pp. 658–666.
  40. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” in Proc. IEEE Int. Conf. Comput. Vis., Venice, Italy, Oct. 2017, pp. 2999–3007.
  41. Z. Huang, W. Li, X.-G. Xia, H. Wang, F. Jie, and R. Tao, “LO-Det: Lightweight oriented object detection in remote sensing images,” IEEE Trans. Geosci. Remote Sens., pp. 1–15, 2021.
  42. K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,” ISPRS J. Photogramm. Remote Sens., vol. 159, pp. 296–307, 2020.
  43. Z. Liu, L. Yuan, L. Weng, and Y. Yang, “A high resolution optical satellite image dataset for ship recognition and some new baselines,” in Proc. Int. Conf. Pattern Recognit. Appl. Methods, vol. 2.   SciTePress, 2017, pp. 324–331.
  44. J. Ding, N. Xue, G.-S. Xia, X. Bai, W. Yang, M. Y. Yang, S. Belongie, J. Luo, M. Datcu, M. Pelillo, et al., “Object detection in aerial images: A large-scale benchmark and challenges,” arXiv preprint arXiv:2102.12219, 2021.
  45. G. Cheng, J. Wang, K. Li, X. Xie, C. Lang, Y. Yao, and J. Han, “Anchor-free oriented proposal generator for object detection,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–11, 2022.
  46. Y. Sun, B. Cao, P. Zhu, and Q. Hu, “Drone-based RGB-Infrared cross-modality vehicle detection via uncertainty-aware learning,” IEEE Trans. Circuits Syst. Video Technol., 2022.
  47. J. Li, C. Qu, and J. Shao, “Ship detection in sar images based on an improved faster r-cnn,” in Proc. SAR Big Data Era: Model., Methods Appl., Beijing, China, Nov. 2017, pp. 1–6.
  48. C. Feng, Y. Zhong, Y. Gao, M. R. Scott, and W. Huang, “Tood: Task-aligned one-stage object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2021, pp. 3490–3499.
  49. J. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.
  50. Z. Tian, C. Shen, H. Chen, and T. He, “FCOS: Fully convolutional one-stage object detection,” in Proc. IEEE Int. Conf. Comput. Vis., Seoul, South Korea, Oct. 2019, pp. 9627–9636.
  51. Z. Huang, W. Li, and R. Tao, “Extracting and distilling direction-adaptive knowledge for lightweight object detection in remote sensing images,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process.   IEEE, 2022, pp. 2360–2364.
  52. W. Qian, X. Yang, S. Peng, Y. Guo, and C. Yan, “Learning modulated loss for rotated object detection,” arXiv preprint arXiv:1911.08299, 2019.
  53. X. Xie, G. Cheng, J. Wang, X. Yao, and J. Han, “Oriented R-CNN for object detection,” in Proc. IEEE Int. Conf. Comput. Vis., 2021, pp. 3520–3529.
  54. Y. Wang, and Z. Zhang, W. Xu, L. Chen, G. Wang, L. Yan, S. Zhong, and X. Zou, “Learning oriented object detection via naive geometric computing,” IEEE Trans. Neural Netw. Learn. Syst., pp. 1–13, 2023.
  55. G. Cheng, Y. Yao, S. Li, K. Li, X. Xie, J. Wang, X. Yao, and J. Han, “Dual-aligned oriented detector,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–11, 2022.
  56. Q. Ming, Z. Zhou, L. Miao, X. Yang, and Y. Dong, “Optimization for oriented object detection via representation invariance loss,” arXiv preprint arXiv:2103.11636, 2021.
  57. P. Zhao, Z. Qu, Y. Bu, W. Tan, and Q. Guan, “Polardet: A fast, more precise detector for rotated target in aerial images,” Int. J. Remote Sens., vol. 42, no. 15, pp. 5831–5861, 2021.
  58. X. Yang, Y. Zhou, G. Zhang, J. Yang, W. Wang, J. Yan, X. Zhang, and Q. Tian, “The KFIoU loss for rotated object detection,” arXiv preprint arXiv:2201.12558, 2022.
  59. L. Hou, K. Lu, X. Yang, Y. Li, and J. Xue, “G-rep: Gaussian representation for arbitrary-oriented object detection,” arXiv preprint arXiv:2205.11796, 2022.
  60. Y. Jiang, X. Zhu, X. Wang, S. Yang, W. Li, H. Wang, P. Fu, and Z. Luo, “R2CNN: Rotational region CNN for orientation robust dcene text detection,” arXiv preprint arXiv:1706.09579, 2017.
  61. J. Wang, W. Yang, H.-C. Li, H. Zhang, and G.-S. Xia, “Learning center probability map for detecting objects in aerial images,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 5, pp. 4307–4323, 2020.
  62. X. Yang, L. Hou, Y. Zhou, W. Wang, and J. Yan, “Dense label encoding for boundary discontinuity free rotation detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 15 819–15 829.
  63. J. Wang, C. Lu, and W. Jiang, “Simultaneous ship detection and orientation estimation in SAR images based on attention module and angle regression,” Sensors-basel., vol. 18, no. 9-2851, pp. 1–17, 2018.
  64. Q. An, Z. Pan, L. Liu, and H. You, “DRBox-v2: An improved detector with rotatable boxes for target detection in SAR images,” IEEE Trans. Geosci. Remote Sens., vol. 57, no. 11, pp. 8333–8349, 2019.
  65. C. Yao, X. Bai, W. Liu, Y. Ma, and Z. Tu, “Detecting texts of arbitrary orientations in natural images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.   IEEE, 2012, pp. 1083–1090.
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now