Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Spike Stream Denoising via Spike Camera Simulation (2304.03129v2)

Published 6 Apr 2023 in cs.CV

Abstract: As a neuromorphic sensor with high temporal resolution, the spike camera shows enormous potential in high-speed visual tasks. However, the high-speed sampling of light propagation processes by existing cameras brings unavoidable noise phenomena. Eliminating the unique noise in spike stream is always a key point for spike-based methods. No previous work has addressed the detailed noise mechanism of the spike camera. To this end, we propose a systematic noise model for spike camera based on its unique circuit. In addition, we carefully constructed the noise evaluation equation and experimental scenarios to measure noise variables. Based on our noise model, the first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream. Further, we design a tailored spike stream denoising framework (DnSS) where denoised spike stream is obtained by decoding inferred inter-spike intervals. Experiments show that DnSS has promising performance on the proposed benchmark. Eventually, DnSS can be generalized well on real spike stream.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Noise2void - learning denoising from single noisy images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 2124–2132, 2019.
  2. A non-local algorithm for image denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 60–65, 2005.
  3. Event probability mask (epm) and event denoising convolutional neural network (edncnn) for neuromorphic cameras. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1698–1707, 2020.
  4. A 240 × 180 130 db 3 μ⁢s𝜇s\mu\rm sitalic_μ roman_s latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State Circuits (JSSC), 49(10):2333–2341, 2014.
  5. Self-supervised mutual learning for dynamic scene reconstruction of spiking camera. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI, pages 2859–2866, 2022.
  6. Activity-driven, event-based vision sensors. IEEE International Symposium on Circuits and Systems (ISCAS), pages 2426–2429, 2010.
  7. An efficient coding method for spike camera using inter-spike intervals. In Data Compression Conference (DCC), pages 437–437, 2017.
  8. Eventzoom: Learning to denoise and super resolve neuromorphic events. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 12824–12833, 2021.
  9. Event density based denoising method for dynamic vision sensor. Applied Sciences, 10(6):2024, 2020.
  10. Live demonstration: A 768× 640 pixels 200meps dynamic vision sensor. IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–1, 2017.
  11. Optical flow estimation for spiking camera. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 17844–17853, 2022.
  12. Neighbor2neighbor: Self-supervised denoising from single noisy images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14781–14790, 2021.
  13. Noisier2noise: Learning to denoise from unpaired noisy data. In International Conference on Machine Learning (ICML), page 2965–2974, 2018.
  14. Noise2self: Blind denoising by self-supervision. In International Conference on Machine Learning (ICML), page 524–533, 2019.
  15. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. 27(9):4608–4622, 2017.
  16. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. page 3142–3155, 2017.
  17. Retinomorphic sensing: A novel paradigm for future multimedia computing. In Proceedings of the ACM International Conference on Multimedia (ACMMM), page 144–152, 2021.
  18. o⁢(n)𝑜𝑛o(n)italic_o ( italic_n )-space spatiotemporal filter for reducing noise in neuromorphic vision sensors. IEEE Transactions on Emerging Topics in Computing, 9(1):15–23, 2021.
  19. Image denoising by sparse 3-d transform-domain collaborative filtering. page 2080–2095, 2007.
  20. A 128 × 128 120 db 15 μ⁢s𝜇s\mu\rm sitalic_μ roman_s latency asynchronous temporal contrast vision sensor. IEEE Journal of Solid-State Circuits (JSSC), 43(2):566–576, 2008.
  21. Embedded vision system for real-time object tracking using an asynchronous transient vision sensor. In IEEE Digital Signal Processing Workshop, pages 173–178. IEEE, 2006.
  22. Design of a spatiotemporal correlation filter for event-based sensors. In IEEE International Symposium on Circuits and Systems (ISCAS), pages 722–725, 2015.
  23. Noisier2noise: Learning to denoise from unpaired noisy data. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 12061–12069, 2020.
  24. An asynchronous time-based image sensor. IEEE International Symposium on Circuits and Systems (ISCAS), 49(10):2130–2133, 2008.
  25. High-quality self-supervised deep image denoising. In Advances in Neural Information Processing Systems (NIPS), page 6970–6980, 2019.
  26. Toward convolutional blind denoising of real photographs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 1712–1722, 2019.
  27. Self-guided network for fast image denoising. In IEEE International Conference on Computer Vision (ICCV), page 2511–2520, 2019.
  28. Lefkimmiatis Stamatios. Universal denoising networks : A novel cnn architecture for image denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 3204–3213, 2018.
  29. Unpaired learning of deep image denoising. In European Conference on Computer Vision (ECCV), 2020.
  30. Spike transformer: Monocular depth estimation for spiking camera. In European Conference on Computer Vision (ECCV), 2022.
  31. Super resolve dynamic scene from continuous spike streams. In IEEE International Conference on Computer Vision (ICCV), pages 2533–2542, 2021.
  32. High-speed motion scene reconstruction for spike camera via motion aligned filtering. In International Symposium on Circuits and Systems (ISCAS), pages 1–5, 2020.
  33. Spk2imgnet: Learning to reconstruct dynamic scene from continuous spike stream. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11996–12005, 2021.
  34. High-speed image reconstruction through short-term plasticity for spiking cameras. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6358–6367, 2021.
  35. A retina-inspired sampling method for visual texture reconstruction. In IEEE International Conference on Multimedia and Expo (ICME), pages 1432–1437, 2019.
  36. Retina-like visual image reconstruction via spiking neural model. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1435–1443, 2020.
  37. Neuspike-net: High speed video reconstruction via bio-inspired neuromorphic cameras. In IEEE International Conference on Computer Vision (ICCV), pages 2400–2409, 2021.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now