Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fast Window-Based Event Denoising with Spatiotemporal Correlation Enhancement (2402.09270v1)

Published 14 Feb 2024 in cs.CV

Abstract: Previous deep learning-based event denoising methods mostly suffer from poor interpretability and difficulty in real-time processing due to their complex architecture designs. In this paper, we propose window-based event denoising, which simultaneously deals with a stack of events while existing element-based denoising focuses on one event each time. Besides, we give the theoretical analysis based on probability distributions in both temporal and spatial domains to improve interpretability. In temporal domain, we use timestamp deviations between processing events and central event to judge the temporal correlation and filter out temporal-irrelevant events. In spatial domain, we choose maximum a posteriori (MAP) to discriminate real-world event and noise, and use the learned convolutional sparse coding to optimize the objective function. Based on the theoretical analysis, we build Temporal Window (TW) module and Soft Spatial Feature Embedding (SSFE) module to process temporal and spatial information separately, and construct a novel multi-scale window-based event denoising network, named MSDNet. The high denoising accuracy and fast running speed of our MSDNet enables us to achieve real-time denoising in complex scenes. Extensive experimental results verify the effectiveness and robustness of our MSDNet. Our algorithm can remove event noise effectively and efficiently and improve the performance of downstream tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Event-based feature extraction using adaptive selection thresholds. Sensors, 20(6):1600, 2020.
  2. Neuromorphic camera denoising using graph neural network-driven transformers. IEEE Transactions on Neural Networks and Learning Systems, 2022.
  3. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7243–7252, 2017.
  4. Event probability mask (epm) and event denoising convolutional neural network (edncnn) for neuromorphic cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1701–1710, 2020.
  5. Inceptive event time-surfaces for object classification using neuromorphic cameras. In Image Analysis and Recognition: 16th International Conference, ICIAR 2019, Waterloo, ON, Canada, August 27–29, 2019, Proceedings, Part II 16, pages 395–403. Springer, 2019.
  6. A 240x180 130 db 3 s latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State, 2013.
  7. A 240×\times× 180 10mw 12us latency sparse-output vision sensor for mobile applications. In 2013 Symposium on VLSI Circuits, pages C186–C187. IEEE, 2013.
  8. The discrete gaussian for differential privacy. Advances in Neural Information Processing Systems, 33:15676–15688, 2020.
  9. Live demonstration: Celex-v: A 1m pixel multi-mode event-based sensor. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1682–1683. IEEE, 2019.
  10. Learning to super resolve intensity images from events. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2768–2776, 2020.
  11. An embedded aer dynamic vision sensor for low-latency pole balancing. In 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pages 780–785. IEEE, 2009.
  12. Interacting maps for fast visual interpretation. In The 2011 International Joint Conference on Neural Networks, pages 770–776. IEEE, 2011.
  13. Evaluating noise filtering for event-based asynchronous change detection image sensors. In 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), pages 19–24. IEEE, 2016.
  14. Tobi Delbruck et al. Frame-free dynamic digital vision. In Proceedings of Intl. Symp. on Secure-Life Electronics, Advanced Electronics for Quality Life and Society, volume 1, pages 21–26. Citeseer, 2008.
  15. Guided event filtering: Synergy between intensity images and neuromorphic events for high performance imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):8261–8275, 2021.
  16. Eventzoom: Learning to denoise and super resolve neuromorphic events. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12824–12833, 2021.
  17. Aednet: Asynchronous event denoising with spatial-temporal correlation among irregular data. In Proceedings of the 30th ACM International Conference on Multimedia, pages 1427–1435, 2022.
  18. A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3867–3876, 2018.
  19. Video to events: Recycling video datasets for event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3586–3595, 2020.
  20. Asynchronous, photometric feature tracking using events and frames. In Proceedings of the European Conference on Computer Vision (ECCV), pages 750–765, 2018.
  21. Low cost and latency event camera background activity denoising. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):785–795, 2022.
  22. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  23. 1000×\times× faster camera and machine vision with ordinary devices. Engineering, 25:110–119, 2023.
  24. o⁢(n)𝑜𝑛o(n)italic_o ( italic_n ) o (n)-space spatiotemporal filter for reducing noise in neuromorphic vision sensors. IEEE Transactions on Emerging Topics in Computing, 9(1):15–23, 2018.
  25. Real-time 3d reconstruction and 6-dof tracking with an event camera. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI 14, pages 349–364. Springer, 2016.
  26. Learning multiple layers of features from tiny images. 2009.
  27. Low-latency visual odometry using event-based feature tracks. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 16–23. IEEE, 2016.
  28. Handwritten digit recognition with a back-propagation network. Advances in neural information processing systems, 2, 1989.
  29. Real-time gesture interface based on event-driven processing from stereo silicon retinas. IEEE transactions on neural networks and learning systems, 25(12):2250–2263, 2014.
  30. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:309, 2017.
  31. A 128× 128 120 db 15 us latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 43(2):566–576, 2008.
  32. A 128 x 128 120db 30mw asynchronous vision sensor that responds to relative intensity change. In 2006 IEEE International Solid State Circuits Conference-Digest of Technical Papers, pages 2060–2069. IEEE, 2006.
  33. Biologically inspired composite vision system for multiple depth-of-field vehicle tracking and speed detection. In Computer Vision-ACCV 2014 Workshops: Singapore, Singapore, November 1-2, 2014, Revised Selected Papers, Part I 12, pages 473–486. Springer, 2015.
  34. Design of a spatiotemporal correlation filter for event-based sensors. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS), pages 722–725. IEEE, 2015.
  35. An asynchronous time-based image sensor. In IEEE International Symposium on Circuits & Systems, 2008.
  36. Retinomorphic event-based vision sensors: bioinspired cameras with spiking output. Proceedings of the IEEE, 102(10):1470–1484, 2014.
  37. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
  38. Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization. 2017.
  39. Real-time panoramic tracking for event cameras. In 2017 IEEE International Conference on Computational Photography (ICCP), pages 1–9. IEEE, 2017.
  40. Evaluation of event-based algorithms for optical flow with ground-truth from inertial measurement sensor. Frontiers in neuroscience, 10:176, 2016.
  41. Fast image reconstruction with an event camera. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 156–163, 2020.
  42. A 128 x 128 1.5% contrast sensitivity 0.9% fpn 3 us latency 4 mw asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE Journal of Solid-State Circuits, 48(3):827–838, 2013.
  43. Linda G Shapiro. Connected component labeling and adjacency graph construction. In Machine intelligence and pattern recognition, volume 19, pages 1–30. Elsevier, 1996.
  44. Hats: Histograms of averaged time surfaces for robust event-based object classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1731–1740, 2018.
  45. Learned convolutional sparse coding. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2191–2195. IEEE, 2018.
  46. Fast event-based harris corner detection exploiting the advantages of event-driven cameras. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 4144–4149. IEEE, 2016.
  47. Eventsr: From asynchronous events to image reconstruction, restoration, and super-resolution via end-to-end adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8315–8325, 2020.
  48. Ev-gait: Event-based robust gait recognition using dynamic vision sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6358–6367, 2019.
  49. Probabilistic undirected graph based denoising method for dynamic vision sensor. IEEE Transactions on Multimedia, 23:1148–1159, 2020.
  50. Adaptive event address map denoising for event cameras. IEEE Sensors Journal, 22(4):3417–3429, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now