Papers
Topics
Authors
Recent
2000 character limit reached

Restoring Real-World Degraded Events Improves Deblurring Quality (2407.20502v1)

Published 30 Jul 2024 in cs.CV

Abstract: Due to its high speed and low latency, DVS is frequently employed in motion deblurring. Ideally, high-quality events would adeptly capture intricate motion information. However, real-world events are generally degraded, thereby introducing significant artifacts into the deblurred results. In response to this challenge, we model the degradation of events and propose RDNet to improve the quality of image deblurring. Specifically, we first analyze the mechanisms underlying degradation and simulate paired events based on that. These paired events are then fed into the first stage of the RDNet for training the restoration model. The events restored in this stage serve as a guide for the second-stage deblurring process. To better assess the deblurring performance of different methods on real-world degraded events, we present a new real-world dataset named DavisMCR. This dataset incorporates events with diverse degradation levels, collected by manipulating environmental brightness and target object contrast. Our experiments are conducted on synthetic datasets (GOPRO), real-world datasets (REBlur), and the proposed dataset (DavisMCR). The results demonstrate that RDNet outperforms classical event denoising methods in event restoration. Furthermore, RDNet exhibits better performance in deblurring tasks compared to state-of-the-art methods. DavisMCR are available at https://github.com/Yeeesir/DVS_RDNet.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. A residual learning approach to deblur and generate high frame rate video with an event camera. IEEE Transactions on Multimedia (2022).
  2. Simple baselines for image restoration. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VII. Springer, 17–33.
  3. Hinet: Half instance normalization network for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 182–192.
  4. Non-Coaxial Event-guided Motion Deblurring with Spatial Alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12492–12503.
  5. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE/CVF international conference on computer vision. 4641–4650.
  6. Multi-scale Residual Low-Pass Filter Network for Image Deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 12345–12354.
  7. Video to events: Recycling video datasets for event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3586–3595.
  8. Evintsr-net: Event guided multiple latent frames reconstruction and super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4882–4891.
  9. Pyramid architecture search for real-time image deblurring. In Proceedings of the IEEE/CVF international conference on computer vision. 4298–4307.
  10. v2e: From video frames to realistic DVS events. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1312–1321.
  11. Learning event-based motion deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3320–3329.
  12. Event-guided Deblurring of Unknown Exposure Time Videos. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVIII. Springer, 519–538.
  13. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8183–8192.
  14. DVS-Voltmeter: Stochastic process-based event simulator for dynamic vision sensors. In European Conference on Computer Vision. Springer, 578–593.
  15. Learning event-driven video deblurring and interpolation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16. Springer, 695–710.
  16. Design of a spatiotemporal correlation filter for event-based sensors. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 722–725.
  17. Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3883–3891.
  18. Event-based Blur Kernel Estimation For Blind Motion Deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4119–4127.
  19. Variational-EM-based deep learning for noise-blind image deblurring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3626–3635.
  20. Efi-net: Video frame interpolation from fusion of events and frames. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1291–1301.
  21. Bringing a blurry frame alive at high frame-rate with an event camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6820–6829.
  22. Spatially-adaptive image restoration using distortion-guided networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2309–2319.
  23. ESIM: an open event camera simulator. In Conference on robot learning. PMLR, 969–982.
  24. Multiscale structure guided diffusion for image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10721–10733.
  25. Realistic blur synthesis for learning image deblurring. In European conference on computer vision. Springer, 487–503.
  26. Bringing events into video deblurring with non-consecutively blurry frames. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4531–4540.
  27. Reducing the sim-to-real gap for event cameras. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVII 16. Springer, 534–549.
  28. Event-based fusion for motion deblurring with cross-modal attention. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVIII. Springer, 412–428.
  29. Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8174–8182.
  30. Event-Based Image Deblurring with Dynamic Motion Awareness. In Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part V. Springer, 95–112.
  31. Event enhanced high-quality image recovery. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16. Springer, 155–171.
  32. Joint filtering of intensity images and neuromorphic events for high-resolution noise-robust imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1609–1619.
  33. EVA 2: Event-Assisted Video Frame Interpolation via Cross-Modal Alignment and Aggregation. IEEE Transactions on Computational Imaging 8 (2022), 1145–1158.
  34. Motion deblurring with real events. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2583–2592.
  35. Event-based Motion Deblurring with Modality-Aware Decomposition and Recomposition. In Proceedings of the 31st ACM International Conference on Multimedia. 8327–8335.
  36. Learning for Motion Deblurring with Hybrid Frames and Events. In Proceedings of the 30th ACM International Conference on Multimedia. 1396–1404.
  37. Training weakly supervised video frame interpolation with events. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14589–14598.
  38. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5728–5739.
  39. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14821–14831.
  40. Deep stacked hierarchical multi-patch network for image deblurring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5978–5986.
  41. Deblurring by realistic blurring. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2737–2746.
  42. Hybrid deblur net: Deep non-uniform deblurring with event camera. IEEE Access 8 (2020), 148075–148083.
  43. Fine-grained video deblurring with event camera. In MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22–24, 2021, Proceedings, Part I 27. Springer, 352–364.
  44. Xiang Zhang and Lei Yu. 2022. Unifying motion deblurring and frame interpolation with events. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17765–17774.
  45. Generalizing Event-Based Motion Deblurring in Real-World Scenarios. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10734–10744.
  46. Sdwnet: A straight dilated network with wavelet transformation for image deblurring. In Proceedings of the IEEE/CVF international conference on computer vision. 1895–1904.

Summary

We haven't generated a summary for this paper yet.

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

GitHub

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.