Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Robust Target Detection of Intelligent Integrated Optical Camera and mmWave Radar System (2312.06983v1)

Published 12 Dec 2023 in eess.IV

Abstract: Target detection is pivotal for modern urban computing applications. While image-based techniques are widely adopted, they falter under challenging environmental conditions such as adverse weather, poor lighting, and occlusion. To improve the target detection performance under complex real-world scenarios, this paper proposes an intelligent integrated optical camera and millimeter-wave (mmWave) radar system. Utilizing both physical knowledge and data-driven methods, a long-term robust radar-camera fusion algorithm is proposed to solve the heterogeneous data fusion problem for detection improvement. For the occlusion scenarios, the proposed algorithm can effectively detect occluded targets with the help of memory through performing long-term detection. For dark scenarios with low-light conditions, the proposed algorithm can effectively mark the target in the dark picture as well as provide rough stickman imaging. The above two innovative functions of the hybrid optical camera and mmWave radar system are tested in real-world scenarios. The results demonstrate the robustness and significant enhancement in the target detection performance of our integrated system.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. I. Bekkerman and J. Tabrikian. Target detection and localization using mimo radars and sonars. IEEE Transactions on Signal Processing, 54(10):3873–3883, 2006.
  2. Nasser M. Nasrabadi. Hyperspectral target detection : An overview of current and future challenges. IEEE Signal Processing Magazine, 31(1):34–44, 2014.
  3. Attentional local contrast networks for infrared small target detection. IEEE Transactions on Geoscience and Remote Sensing, 59(11):9813–9824, 2021.
  4. Marine targets detection for scanning radar images based on radar- yolonet. In 2021 CIE International Conference on Radar (Radar), pages 1256–1260, 2021.
  5. Radar moving target detection in clutter background via adaptive dual-threshold sparse fourier transform. IEEE Access, 7:58200–58211, 2019.
  6. Space-range-doppler focus-based low-observable moving target detection using frequency diverse array mimo radar. IEEE Access, 6:43892–43904, 2018.
  7. Target detection and tracking algorithm based on improved mask rcnn and lmb. In 2021 International Conference on Control, Automation and Information Sciences (ICCAIS), pages 1037–1041, 2021.
  8. Edge learning for b5g networks with distributed signal processing: Semantic communication, edge computing, and wireless sensing. IEEE journal of selected topics in signal processing, 17(1):9–39, 2023.
  9. Energy efficient semantic communication over wireless networks with rate splitting. IEEE Journal on Selected Areas in Communications, 41(5):1484–1495, 2023.
  10. Semantic communication with probability graph: A joint communication and computation design. arXiv preprint arXiv:2310.00015, 2023.
  11. Big ai models for 6g wireless networks: Opportunities, challenges, and research directions. arXiv preprint arXiv:2308.06250, 2023.
  12. Wireless communications and applications above 100 ghz: Opportunities and challenges for 6g and beyond. IEEE Access, 7:78729–78757, 2019.
  13. Joint channel estimation and radar sensing for uav networks with mmwave massive mimo. In 2022 International Wireless Communications and Mobile Computing (IWCMC), pages 44–49, 2022.
  14. A review of the application of optical and radar remote sensing data fusion to land use mapping and monitoring. Remote Sensing, 8(1):70, 2016.
  15. Optical-and-radar image fusion for dynamic estimation of spin satellites. IEEE Transactions on Image Processing, 29:2963–2976, 2019.
  16. A novel tracking algorithm using thermal and optical cameras fused with mmwave radar sensor data. IEEE Transactions on Consumer Electronics, 67(4):372–382, 2021.
  17. Double kill: Compressive-sensing-based narrow-band interference and impulsive noise mitigation for vehicular communications. IEEE Transactions on Vehicular Technology, 65(7):5099–5109, 2016.
  18. A priori aided compressive sensing approach for impulsive noise reconstruction. In 2015 International Wireless Communications and Mobile Computing Conference (IWCMC), pages 205–209, 2015.
  19. A multi-sensor fusion system for moving object detection and tracking in urban driving environments. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 1836–1843. IEEE, 2014.
  20. Fusing mmwave radar with camera for 3-d detection in autonomous driving. IEEE Internet of Things Journal, 9(20):20408–20421, 2022.
  21. 3d multi-object tracking based on radar-camera fusion. In 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pages 2502–2507. IEEE, 2022.
  22. Global-local feature enhancement network for robust object detection using mmwave radar and camera. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4708–4712. IEEE, 2022.
  23. Pedestrian liveness detection based on mmwave radar and camera fusion. In 2022 19th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), pages 262–270. IEEE, 2022.
  24. Extending reliability of mmwave radar tracking and detection via fusion with camera. IEEE Access, 7:137065–137079, 2019.
  25. Fusion of optical and millimeter wave sar sensing for object recognition in indoor environment. In 2022 Fifth International Workshop on Mobile Terahertz Systems (IWMTS), pages 1–5. IEEE, 2022.
  26. millieye: A lightweight mmwave radar and camera fusion system for robust object detection. In Proceedings of the International Conference on Internet-of-Things Design and Implementation, pages 145–157, 2021.
  27. Lane-cross detection using optical camera-based road-to-vehicle communications. In 2021 17th International Symposium on Wireless Communication Systems (ISWCS), pages 1–5, 2021.
  28. Statistical study of hardware impairments effect on mmwave 77 ghz fmcw automotive radar. In 2020 IEEE Radar Conference (RadarConf20), pages 1–6, 2020.
  29. Multi-view sensing for wireless communications: Architectures, designs, and opportunities. IEEE Communications Magazine, 2023.
  30. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.
  31. Ssd: Single shot multibox detector. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pages 21–37. Springer, 2016.
  32. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226–231, 1996.
  33. Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97, 1955.
  34. Light-head r-cnn: In defense of two-stage object detector. arXiv preprint arXiv:1711.07264, 2017.
  35. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  36. Zhengyou Zhang. Flexible camera calibration by viewing a plane from unknown orientations. In Proceedings of the Seventh IEEE International Conference on Computer Vision, volume 1, pages 666–673 vol.1, 1999.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now