Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SpecDETR: A Transformer-based Hyperspectral Point Object Detection Network (2405.10148v3)

Published 16 May 2024 in cs.CV

Abstract: Hyperspectral target detection (HTD) aims to identify specific materials based on spectral information in hyperspectral imagery and can detect extremely small-sized objects, some of which occupy a smaller than one-pixel area. However, existing HTD methods are developed based on per-pixel binary classification, neglecting the three-dimensional cube structure of hyperspectral images (HSIs) that integrates both spatial and spectral dimensions. The synergistic existence of spatial and spectral features in HSIs enable objects to simultaneously exhibit both, yet the per-pixel HTD framework limits the joint expression of these features. In this paper, we rethink HTD from the perspective of spatial-spectral synergistic representation and propose hyperspectral point object detection as an innovative task framework. We introduce SpecDETR, the first specialized network for hyperspectral multi-class point object detection, which eliminates dependence on pre-trained backbone networks commonly required by vision-based object detectors. SpecDETR uses a multi-layer Transformer encoder with self-excited subpixel-scale attention modules to directly extract deep spatial-spectral joint features from hyperspectral cubes. We develop a simulated hyperspectral point object detection benchmark termed SPOD, and for the first time, evaluate and compare the performance of visual object detection networks and HTD methods on hyperspectral point object detection. Extensive experiments demonstrate that our proposed SpecDETR outperforms SOTA visual object detection networks and HTD methods. Our code and dataset are available at https://github.com/ZhaoxuLi123/SpecDETR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. C. Zhao, B. Qin, S. Feng, W. Zhu, W. Sun, W. Li, and X. Jia, “Hyperspectral image classification with multi-attention transformer and adaptive superpixel segmentation-based active learning,” IEEE Trans. Image Process., 2023.
  2. L. Drumetz, J. Chanussot, C. Jutten, W.-K. Ma, and A. Iwasaki, “Spectral variability aware blind hyperspectral image unmixing based on convex geometry,” IEEE Trans. Image Process., vol. 29, pp. 4568–4582, 2020.
  3. J. E. Fowler and Q. Du, “Anomaly detection and reconstruction from random projections,” IEEE Trans. Image Process., vol. 21, no. 1, pp. 184–195, 2011.
  4. Z. Li, Y. Wang, C. Xiao, Q. Ling, Z. Lin, and W. An, “You only train once: Learning a general anomaly enhancement network with random masks for hyperspectral anomaly detection,” IEEE Trans. Geosci. Remote Sens., vol. 61, pp. 1–18, 2023.
  5. B. Du, Y. Zhang, L. Zhang, and D. Tao, “Beyond the sparsity-based target detector: A hybrid sparsity and statistics-based detector for hyperspectral images,” IEEE Trans. Image Process., vol. 25, no. 11, pp. 5345–5357, 2016.
  6. Y. Li, Y. Shi, K. Wang, B. Xi, J. Li, and P. Gamba, “Target detection with unconstrained linear mixture model and hierarchical denoising autoencoder in hyperspectral imagery,” IEEE Trans. Image Process., vol. 31, pp. 1418–1432, 2022.
  7. G. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, “DOTA: A large-scale dataset for object detection in aerial images,” in IEEE Conf. Comput. Vis. Pattern Recog., 2018, pp. 3974–3983.
  8. G. Vane, R. O. Green, T. G. Chrien, H. T. Enmark, E. G. Hansen, and W. M. Porter, “The airborne visible/infrared imaging spectrometer (AVIRIS),” Remote sensing of environment, vol. 44, no. 2-3, pp. 127–143, 1993.
  9. C. E. Caefer, S. R. Rotman, J. Silverman, and P. W. Yip, “Algorithms for point target detection in hyperspectral imagery,” in Imaging Spectrometry VIII, vol. 4816.   SPIE, 2002, pp. 242–257.
  10. J. C. Harsanyi, “Detection and classification of subpixel spectral signatures in hyperspectral image sequences,” Ph.D. dissertation, University of Maryland Baltimore County, Baltimore, MD, USA, 1993.
  11. H. Ren and C.-I. Chang, “Target-constrained interference-minimized approach to subpixel target detection for hyperspectral images,” Opt. Eng., vol. 39, pp. 3138–3145, 2000.
  12. T. Wang, B. Du, and L. Zhang, “A kernel-based target-constrained interference-minimized filter for hyperspectral sub-pixel target detection,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 6, no. 2, pp. 626–637, Apr. 2013.
  13. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Adv. Neural Inform. Process. Syst., vol. 28, 2015.
  14. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2016, pp. 779–788.
  15. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Eur. Conf. Comput. Vis., 2020, pp. 213–229.
  16. S. Kumar, I. Arevalo, A. Iftekhar, and B. Manjunath, “Methanemapper: Spectral absorption aware hyperspectral transformer for methane detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2023, pp. 17 609–17 618.
  17. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Adv. Neural Inform. Process. Syst., vol. 30, 2017.
  18. X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: Deformable transformers for end-to-end object detection,” arXiv, 2020.
  19. H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H.-Y. Shum, “DINO: DETR with improved denoising anchor boxes for end-to-end object detection,” arXiv, 2022.
  20. N. M. Nasrabadi, “Hyperspectral target detection : An overview of current and future challenges,” IEEE Signal Process. Mag., vol. 31, no. 1, pp. 34–44, Dec. 2013.
  21. F. C. Robey, D. R. Fuhrmann, E. J. Kelly, and R. Nitzberg, “A CFAR adaptive matched filter detector,” IEEE Trans. Aerosp. Electron. Syst., vol. 28, no. 1, pp. 208–216, Jan. 1992.
  22. S. Kraut, L. L. Scharf, and R. W. Butler, “The adaptive coherence estimator: A uniformly most-powerful-invariant adaptive detection statistic,” IEEE Trans. Signal Process., vol. 53, no. 2, pp. 427–438, Feb 2005.
  23. B. Du, Y. Zhang, L. Zhang, and D. Tao, “Beyond the sparsity-based target detector: A hybrid sparsity and statistics based detector for hyperspectral images,” IEEE Trans. Image Process., vol. 25, no. 11, pp. 5345–5357, Nov. 2016.
  24. S. Kraut and L. L. Scharf, “The CFAR adaptive subspace detector is a scale-invariant GLRT,” IEEE Trans. Signal Process., vol. 47, no. 9, pp. 2538–2541, Sep. 1999.
  25. J. C. Harsanyi and C.-I. Chang, “Hyperspectral image classification and dimensionality reduction: An orthogonal subspace projection approach,” IEEE Trans. Geosci. Remote Sens., vol. 32, no. 4, pp. 779–785, Jul. 1994.
  26. Y. Chen, N. M. Nasrabadi, and T. D. Tran, “Simultaneous joint sparsity model for target detection in hyperspectral imagery,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 4, pp. 676–680, Jul. 2011.
  27. Y. Chen, N. M. Nasrabadi, and T. D. Tran, “Sparse representation for target detection in hyperspectral imagery,” IEEE J. Sel. Topics Signal Process., vol. 5, no. 3, pp. 629–640, Jun. 2011.
  28. Y. Chen, N. M. Nasrabadi, and T. D. Tran, “Hyperspectral image classification via kernel sparse representation,” IEEE Trans. Geosci. Remote Sens., vol. 51, no. 1, pp. 217–231, Jan. 2013.
  29. Y. Zhang, B. Du, and L. Zhang, “A sparse representation-based binary hypothesis model for target detection in hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 3, pp. 1346–1354, Mar. 2015.
  30. Q. Ling, Y. Guo, Z. Lin, L. Liu, and W. An, “A constrained sparse-representation-based binary hypothesis model for target detection in hyperspectral imagery,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 12, no. 6, pp. 1933–1947, 2019.
  31. Y. Zhang, L. Zhang, B. Du, and S. Wang, “A nonlinear sparse representation-based binary hypothesis model for hyperspectral target detection,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 6, pp. 2513–2522, Jun. 2015.
  32. X. Jiao and C.-I. Chang, “Kernel-based constrained energy minimization (K-CEM),” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XIV.   SPIE, 2008, pp. 523–533.
  33. H. Kwon and N. M. Nasrabadi, “Kernel orthogonal subspace projection for hyperspectral signal classification,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 12, pp. 2952–2962, 2005.
  34. H. Kwon and N. M. Nasrabadi, “A comparative analysis of kernel subspace target detectors for hyperspectral imagery,” EURASIP Journal on Advances in Signal Processing, vol. 2007, pp. 1–13, 2006.
  35. H. Kwon and N. M. Nasrabadi, “Kernel matched subspace detectors for hyperspectral target detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 2, pp. 178–194, Feb 2006.
  36. G. Camps-Valls, L. Gomez-Chova, J. Munoz-Marí, J. Vila-Francés, and J. Calpe-Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Geosci. Remote Sens. Lett., vol. 3, no. 1, pp. 93–97, Jan. 2006.
  37. W. Rao, L. Gao, Y. Qu, X. Sun, B. Zhang, and J. Chanussot, “Siamese transformer network for hyperspectral image target detection,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–19, 2022.
  38. D. Zhu, B. Du, and L. Zhang, “Two-stream convolutional networks for hyperspectral target detection,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 8, pp. 6907–6921, 2020.
  39. L. Yan, M. Zhao, X. Wang, Y. Zhang, and J. Chen, “Object detection in hyperspectral images,” IEEE Signal Process. Lett., vol. 28, pp. 508–512, 2021.
  40. G. Lee, J. Lee, J. Baek, H. Kim, and D. Cho, “Channel sampler in hyperspectral images for vehicle detection,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2021.
  41. A. Rangnekar, Z. Mulhollan, A. Vodacek et al., “Semi-supervised hyperspectral object detection challenge results-PBVS 2022,” in IEEE Conf. Comput. Vis. Pattern Recog. Worksh., 2022, pp. 390–398.
  42. X. He, C. Tang, X. Liu, W. Zhang, K. Sun, and J. Xu, “Object detection in hyperspectral image via unified spectral–spatial feature aggregation,” IEEE Trans. Geosci. Remote Sens., vol. 61, pp. 1–13, 2023.
  43. S. Liu, F. Li, H. Zhang, X. Yang, X. Qi, H. Su, J. Zhu, and L. Zhang, “DAD-DETR: Dynamic anchor boxes are better queries for DETR,” arXiv, 2022.
  44. F. Li, H. Zhang, S. Liu, J. Guo, L. M. Ni, and L. Zhang, “DN-DETR: Accelerate DETR training by introducing query denoising,” in IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 13 619–13 627.
  45. Q. Chen, X. Chen, J. Wang, S. Zhang, K. Yao, H. Feng, J. Han, E. Ding, G. Zeng, and J. Wang, “Group DETR: Fast DETR training with group-wise one-to-many assignment,” in Int. Conf. Comput. Vis., 2023, pp. 6633–6642.
  46. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Eur. Conf. Comput. Vis., 2014, pp. 740–755.
  47. G. Cheng, X. Yuan, X. Yao, K. Yan, Q. Zeng, X. Xie, and J. Han, “Towards large-scale small object detection: Survey and benchmarks,” IEEE Trans. Pattern Anal. Mach. Intell., 2023.
  48. M. Zhang, R. Zhang, Y. Yang, H. Bai, J. Zhang, and J. Guo, “Isnet: Shape matters for infrared small target detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 877–886.
  49. B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, and Y. Guo, “Dense nested attention network for infrared small target detection,” IEEE Trans. Image Process., vol. 32, pp. 1745–1758, 2022.
  50. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv, 2016.
  51. A. Giannandrea, N. Raqueno, D. W. Messinger, J. Faulring, J. P. Kerekes et al., “The share 2012 data campaign,” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX.   SPIE, 2013, pp. 94–108.
  52. J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei, “Deformable convolutional networks,” in Int. Conf. Comput. Vis., 2017, pp. 764–773.
  53. J. Liu and J. Zhang, “Spectral unmixing via compressive sensing,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 11, pp. 7099–7110, Nov. 2014.
  54. R. Kokaly, R. Clark, G. Swayze, K. Livo, T. Hoefen, N. Pearson, R. Wise, W. Benzel, H. Lowers, R. Driscoll et al., “USGS spectral library version 7 data: US geological survey data release,” United States Geological Survey (USGS): Reston, VA, USA, vol. 61, 2017.
  55. K. Chen, J. Wang, J. Pang, Y. Cao et al., “MMDetection: Open mmlab detection toolbox and benchmark,” arXiv, 2019.
  56. L. L. Scharf and B. Friedlander, “Matched subspace detectors,” IEEE Trans. Signal Process., vol. 42, no. 8, pp. 2146–2157, Aug. 1994.
  57. C.-I. Chang, “An effective evaluation tool for hyperspectral target detection: 3D receiver operating characteristic curve analysis,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 6, pp. 5131–5153, 2020.
  58. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conf. Comput. Vis. Pattern Recog., 2016, pp. 770–778.
  59. I. Radosavovic, R. P. Kosaraju, R. Girshick, K. He, and P. Dollár, “Designing network design spaces,” in IEEE Conf. Comput. Vis. Pattern Recog., 2020, pp. 10 428–10 436.
  60. H. Zhang, C. Wu, Z. Zhang, Y. Zhu, H. Lin, Z. Zhang, Y. Sun, T. He, J. Mueller, R. Manmatha et al., “ResNeSt: Split-attention networks,” in IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 2736–2746.
  61. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in IEEE Conf. Comput. Vis. Pattern Recog., 2017, pp. 1492–1500.
  62. K. Sun, B. Xiao, D. Liu, and J. Wang, “Deep high-resolution representation learning for human pose estimation,” in IEEE Conf. Comput. Vis. Pattern Recog., 2019, pp. 5693–5703.
  63. C. Feng, Y. Zhong, Y. Gao, M. R. Scott, and W. Huang, “TOOD: Task-aligned one-stage object detection,” in Int. Conf. Comput. Vis., 2021, pp. 3490–3499.
  64. Z. Dong, G. Li, Y. Liao, F. Wang, P. Ren, and C. Qian, “Centripetalnet: Pursuing high-quality keypoint pairs for object detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2020, pp. 10 519–10 528.
  65. A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in Eur. Conf. Comput. Vis.   Springer, 2016, pp. 483–499.
  66. H. Law and J. Deng, “Cornernet: Detecting objects as paired keypoints,” in Eur. Conf. Comput. Vis., 2018, pp. 734–750.
  67. Z. Yang, S. Liu, H. Hu, L. Wang, and S. Lin, “Reppoints: Point set representation for object detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2019, pp. 9657–9666.
  68. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in IEEE Conf. Comput. Vis. Pattern Recog., 2017, pp. 2980–2988.
  69. M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in ICML, 2019, pp. 6105–6114.
  70. W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “PVTv2: Improved baselines with pyramid vision transformer,” Computational Visual Media, vol. 8, no. 3, pp. 415–424, 2022.
  71. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Int. Conf. Comput. Vis., 2021, pp. 10 012–10 022.
  72. Z. Zong, G. Song, and Y. Liu, “Detrs with collaborative hybrid assignments training,” in Int. Conf. Comput. Vis., 2023, pp. 6748–6758.
  73. Y. Dai, Y. Wu, F. Zhou, and K. Barnard, “Asymmetric contextual modulation for infrared small target detection,” in IEEE Winter Conference on Applications of Computer Vision, 2021, pp. 950–959.
  74. Y. Dai, Y. Wu, F. Zhou, and K. Barnard, “Attentional local contrast networks for infrared small target detection,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 11, pp. 9813–9824, 2021.
  75. Q. Hou, L. Zhang, F. Tan, Y. Xi, H. Zheng, and N. Li, “ISTDU-Net: Infrared small-target detection U-Net,” IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022.
  76. H. Sun, J. Bai, F. Yang, and X. Bai, “Receptive-field and direction induced attention network for infrared dim small target detection with a large-scale dataset irdst,” IEEE Trans. Geosci. Remote Sens., vol. 61, pp. 1–13, 2023.
  77. X. Wu, D. Hong, and J. Chanussot, “UIU-Net: U-Net in U-Net for infrared small object detection,” IEEE Trans. Image Process., vol. 32, pp. 364–376, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets