Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Event-based Object Detection in Autonomous Driving: A Survey (2405.03995v1)

Published 7 May 2024 in cs.CV

Abstract: Object detection plays a critical role in autonomous driving, where accurately and efficiently detecting objects in fast-moving scenes is crucial. Traditional frame-based cameras face challenges in balancing latency and bandwidth, necessitating the need for innovative solutions. Event cameras have emerged as promising sensors for autonomous driving due to their low latency, high dynamic range, and low power consumption. However, effectively utilizing the asynchronous and sparse event data presents challenges, particularly in maintaining low latency and lightweight architectures for object detection. This paper provides an overview of object detection using event data in autonomous driving, showcasing the competitive benefits of event cameras.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. Davison, J. Conradt, K. Daniilidis, and D. Scaramuzza, “Event-based vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
  2. M. Iacono, S. Weber, A. Glover, and C. Bartolozzi, “Towards event-driven object detection with off-the-shelf deep learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2018.
  3. “Pseudo-labels for supervised learning on dynamic vision sensor data, applied to object detection under ego-motion,” 2018.
  4. M. Cannici, M. Ciccone, A. Romanoni, and M. Matteucci, “Asynchronous convolutional networks for object detection in neuromorphic cameras,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2019.
  5. Cham: Springer International Publishing, 2020.
  6. P. de Tournemire, D. Nitti, E. Perot, D. Migliore, and A. Sironi, “A large scale event-based detection dataset for automotive,” arXiv.org, 2020.
  7. H. Xu, D. Shi, L. Jing, and C. Liu, “Polar loss for event-based object detection,” in 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), IEEE, 2021.
  8. J. Li, J. Li, L. Zhu, X. Xiang, T. Huang, and Y. Tian, “Asynchronous spatio-temporal memory network for continuous event-based object detection,” IEEE Transactions on Image Processing, vol. 31, p. 2975–2987, 2022.
  9. B. Liu, C. Xu, W. Yang, H. Yu, and L. Yu, “Motion robust high-speed light-weighted object detection with event camera,” IEEE Transactions on Instrumentation and Measurement, vol. 72, p. 1–13, 2023.
  10. D. Wang, X. Jia, Y. Zhang, X. Zhang, Y. Wang, Z. Zhang, D. Wang, and H. Lu, “Dual memory aggregation network for event-based object detection with learnable representation,” AAAI Conference on Artificial Intelligence, 2023.
  11. M. Gehrig and D. Scaramuzza, “Recurrent vision transformers for object detection with event cameras,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2023.
  12. Cham: Springer International Publishing, 2021.
  13. L. Cordone, B. Miramond, and P. Thierion, “Object detection with spiking neural networks on automotive event data,” in 2022 International Joint Conference on Neural Networks (IJCNN), IEEE, 2022.
  14. H. Zhang, L. Leng, K. Che, Q. Liu, J. Cheng, Q. Guo, J. Liao, and R. Cheng, “Automotive object detection via learning sparse events by temporal dynamics of spiking neurons,” arXiv.org, 2023.
  15. M. Nagaraj, C. M. Liyanagedera, and K. Roy, “Dotie - detecting objects through temporal isolation of events using a spiking architecture,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023.
  16. S. Schaefer, D. Gehrig, and D. Scaramuzza, “Aegnn: Asynchronous event-based graph neural networks,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2022.
  17. D. Gehrig and D. Scaramuzza, “Pushing the limits of asynchronous graph-based object detection with event cameras,” arXiv.org, 2022.
  18. Z. Jiang, P. Xia, K. Huang, W. Stechele, G. Chen, Z. Bing, and A. Knoll, “Mixed frame-/event-driven fast pedestrian detection,” in 2019 International Conference on Robotics and Automation (ICRA), IEEE, 2019.
  19. J. Li, S. Dong, Z. Yu, Y. Tian, and T. Huang, “Event-based vision enhanced: A joint detection framework in autonomous driving,” in 2019 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2019.
  20. M. Liu, N. Qi, Y. Shi, and B. Yin, “An attention fusion network for event-based vehicle object detection,” in 2021 IEEE International Conference on Image Processing (ICIP), IEEE, 2021.
  21. A. Tomy, A. Paigwar, K. S. Mann, A. Renzaglia, and C. Laugier, “Fusing event-based and rgb camera for robust object detection in adverse conditions,” in 2022 International Conference on Robotics and Automation (ICRA), IEEE, 2022.
  22. Z. Zhou, Z. Wu, R. Boutteau, F. Yang, C. Demonceaux, and D. Ginhac, “Rgb-event fusion for moving object detection in autonomous driving,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023.
  23. D. Li, J. Li, and Y. Tian, “Sodformer: Streaming object detection with transformer using events and frames,” IEEE Transactions on Pattern Analysis and Machine Intelligence, p. 1–18, 2023.
  24. E. Perot, P. de Tournemire, D. Nitti, J. Masci, and A. Sironi, “Learning to detect objects with a 1 megapixel event camera,” Neural Information Processing Systems, 2020.
  25. S.-C. Liu, T. Delbruck, G. Indiveri, A. Whatley, and R. Douglas, “Event-based neuromorphic systems,” 2015.
  26. P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128 128 120 db 15 s latency asynchronous temporal contrast vision sensor,” 2006.
  27. C. Posch, D. Matolin, and R. Wohlgenannt, “A qvga 143db dynamic range asynchronous address-event pwm dynamic image sensor with lossless pixel-level video compression,” 2010.
  28. R. Berner, C. Brandli, M. Yang, S.-C. Liu, and T. Delbruck, “A 240x180 120db 10mw 12us-latency sparse output vision sensor for mobile applications,” 2013.
  29. C. Posch, T. Serrano-Gotarredona, B. Linares-Barranco, and T. Delbruck, “Retinomorphic event-based vision sensors: Bioinspired cameras with spiking output,” Proceedings of the IEEE, vol. 102, no. 10, pp. 1470–1484, 2014.
  30. A. Z. Zhu, L. Yuan, K. Chaney, and K. Daniilidis, “Unsupervised event-based learning of optical flow, depth, and egomotion,” 2019.
  31. A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12689–12697, 2019.
  32. A. Bochkovskiy, C.-Y. Wang, and H. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv.org, 2020.
  33. X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” 2021.
  34. J. Ren, X. Chen, J. Liu, W. Sun, J. Pang, Q. Yan, Y.-W. Tai, and L. Xu, “Accurate single stage detector using recurrent rolling convolution,” 2017.
  35. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  36. K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu, Z. Yang, Y. Zhang, and D. Tao, “A survey on vision transformer,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 87–110, 2023.
  37. N. Zubi’c, M. Gehrig, and D. Scaramuzza, “State space models for event cameras,” arXiv.org, 2024.
  38. A. Gu, K. Goel, and C. Ré, “Efficiently modeling long sequences with structured state spaces,” 2022.
  39. Y. Sekikawa, K. Hara, and H. Saito, “Eventnet: Asynchronous recursive event processing,” 2019.
  40. C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” 2017.
  41. B. Graham and L. van der Maaten, “Submanifold sparse convolutional networks,” 2017.
  42. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2021.
  43. K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine intelligence with neuromorphic computing,” Nature, vol. 575, pp. 607–617, 11 2019.
  44. S. Singh, A. Sarma, S. Lu, A. Sengupta, V. Narayanan, and C. R. Das, “Gesture-snn: Co-optimizing accuracy, latency and energy of snns for neuromorphic vision sensors,” in 2021 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pp. 1–6, 2021.
  45. L. Cordone, B. Miramond, and P. Thierion, “Object detection with spiking neural networks on automotive event data,” in Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), July 2022.
  46. F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and ¡0.5mb model size,” 2016.
  47. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015.
  48. A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” 2017.
  49. G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” 2018.
  50. W. Fang, Y. Chen, J. Ding, Z. Yu, T. Masquelier, D. Chen, L. Huang, H. Zhou, G. Li, and Y. Tian, “Spikingjelly: An open-source machine learning infrastructure platform for spike-based intelligence,” Science Advances, vol. 9, no. 40, p. eadi1480, 2023.
  51. Q. Su, Y. Chou, Y. Hu, J. Li, S. Mei, Z. Zhang, and G. Li, “Deep directly-trained spiking neural networks for object detection,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6532–6542, 2023.
  52. Y.-K. Wang, S.-E. Wang, and P.-H. Wu, “Spike-event object detection for neuromorphic vision,” IEEE Access, vol. 11, p. 5215–5230, 2023.
  53. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” 2017.
  54. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2016.
  55. Heidelberg: Physica-Verlag HD, 2001.
  56. Q. Zhou, X. Li, L. He, Y. Yang, G. Cheng, Y. Tong, L. Ma, and D. Tao, “Transvod: End-to-end video object detection with spatial-temporal transformers,” 2022.
  57. M. Gehrig, W. Aarents, D. Gehrig, and D. Scaramuzza, “DSEC: A stereo event camera dataset for driving scenarios,” 2021.
  58. J. Pang, L. Qiu, X. Li, H. Chen, Q. Li, T. Darrell, and F. Yu, “Quasi-dense similarity learning for multiple object tracking,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 2021.
  59. J. Binas, D. Neil, S.-C. Liu, and T. Delbruck, “Ddd17: End-to-end davis driving dataset,” 2017.
  60. A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361, 2012.
  61. A. Z. Zhu, D. Thakur, T. Özaslan, B. Pfrommer, V. Kumar, and K. Daniilidis, “The multivehicle stereo event camera dataset: An event camera dataset for 3d perception,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2032–2039, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Bingquan Zhou (1 paper)
  2. Jie Jiang (246 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets