Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Image Deraining with Frequency-Enhanced State Space Model (2405.16470v4)

Published 26 May 2024 in cs.CV and eess.IV

Abstract: Removing rain degradations in images is recognized as a significant issue. In this field, deep learning-based approaches, such as Convolutional Neural Networks (CNNs) and Transformers, have succeeded. Recently, State Space Models (SSMs) have exhibited superior performance across various tasks in both natural language processing and image processing due to their ability to model long-range dependencies. This study introduces SSM to image deraining with deraining-specific enhancements and proposes a Deraining Frequency-Enhanced State Space Model (DFSSM). To effectively remove rain streaks, which produce high-intensity frequency components in specific directions, we employ frequency domain processing concurrently with SSM. Additionally, we develop a novel mixed-scale gated-convolutional block, which uses convolutions with multiple kernel sizes to capture various scale degradations effectively and integrates a gating mechanism to manage the flow of information. Finally, experiments on synthetic and real-world rainy image datasets show that our method surpasses state-of-the-art methods. Code is available at https://github.com/ShugoYamashita/DFSSM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Removing rain from single images via a deep detail network. In: CVPR. (2017) 3855–3863
  2. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: ECCV. (2018) 254–269
  3. Progressive image deraining networks: A better and simpler baseline. In: CVPR. (2019) 3937–3946
  4. Multi-scale progressive fusion network for single image deraining. In: CVPR. (2020) 8346–8355
  5. Dcsfn: Deep cross-scale fusion network for single image rain removal. In: ACM MM. (2020) 1643–1651
  6. Multi-stage progressive image restoration. In: CVPR. (2021) 14821–14831
  7. A model-driven deep neural network for single image rain removal. In: CVPR. (2020) 3103–3112
  8. Removing raindrops and rain streaks in one go. In: CVPR. (2021) 9147–9156
  9. Rain streak removal via dual graph convolutional network. In: AAAI. (2021) 1352–1360
  10. Structure-preserving deraining with residue channel prior guidance. In: ICCV. (2021) 4238–4247
  11. Attention is all you need. NeurIPS 30 (2017)
  12. Restormer: Efficient transformer for high-resolution image restoration. In: CVPR. (2022) 5728–5739
  13. Image de-raining transformer. IEEE TPAMI 45 (2023) 12978–12995
  14. Learning a sparse transformer network for effective image deraining. In: CVPR. (2023) 5896–5905
  15. Sparse sampling transformer with uncertainty-driven ranking for unified removal of raindrops and rain streaks. In: ICCV. (2023) 13106–13117
  16. Efficiently modeling long sequences with structured state spaces. In: ICLR. (2022)
  17. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 (2023)
  18. Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv:2401.09417 (2024)
  19. Vmamba: Visual state space model. arXiv preprint arXiv:2401.10166 (2024)
  20. Localmamba: Visual state space model with windowed selective scan. arXiv preprint arXiv:2403.09338 (2024)
  21. Efficientvmamba: Atrous selective scan for light weight visual mamba. arXiv preprint arXiv:2403.09977 (2024)
  22. Mambair: A simple baseline for image restoration with state-space model. arXiv preprint arXiv:2402.15648 (2024)
  23. Vmambair: Visual state space model for image restoration. arXiv preprint arXiv:2403.11423 (2024)
  24. U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv:2401.04722 (2024)
  25. Swin-umamba: Mamba-based unet with imagenet-based pretraining. arXiv preprint arXiv:2402.03302 (2024)
  26. Maxim: Multi-axis mlp for image processing. In: CVPR. (2022) 5769–5780
  27. Intriguing findings of frequency selection for image deblurring. In: AAAI. (2023) 1905–1913
  28. Deep joint rain detection and removal from a single image. In: CVPR. (2017) 1357–1366
  29. Spatial attentive single-image deraining with a high quality real rain dataset. In: CVPR. (2019) 12270–12279
  30. Automatic single-image-based rain streaks removal via image decomposition. IEEE TIP 21 (2011) 1742–1755
  31. Removing rain from a single image via discriminative sparse coding. In: ICCV. (2015) 3397–3405
  32. Rain streak removal using layer priors. In: CVPR. (2016) 2736–2744
  33. Convolutional sparse and low-rank coding-based rain streak removal. In: WACV. (2017) 1259–1267
  34. Joint convolutional analysis and synthesis sparse representation for single image layer separation. In: ICCV. (2017) 1708–1716
  35. DARTS: Differentiable architecture search. In: ICLR. (2019)
  36. U-net: Convolutional networks for biomedical image segmentation. In: MICCAI. (2015) 234–241
  37. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR. (2016) 1874–1883
  38. Fast fourier convolution. NeurIPS 33 (2020) 4479–4488
  39. Swinfir: Revisiting the swinir with fast fourier convolution and improved training for image super-resolution. arXiv preprint arXiv:2208.11247 (2022)
  40. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks 107 (2018) 3–11
  41. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016)
  42. Squeeze-and-excitation networks. In: CVPR. (2018) 7132–7141
  43. Pytorch: An imperative style, high-performance deep learning library. NeurIPS 32 (2019)
  44. SGDR: Stochastic gradient descent with warm restarts. In: ICLR. (2017)
  45. Decoupled weight decay regularization. In: ICLR. (2019)
  46. Scope of validity of psnr in image/video quality assessment. Electronics letters 44 (2008) 800–801
  47. Image quality assessment: from error visibility to structural similarity. IEEE TIP 13 (2004) 600–612
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now