Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploiting Regional Information Transformer for Single Image Deraining (2402.16033v2)

Published 25 Feb 2024 in cs.CV

Abstract: Transformer-based Single Image Deraining (SID) methods have achieved remarkable success, primarily attributed to their robust capability in capturing long-range interactions. However, we've noticed that current methods handle rain-affected and unaffected regions concurrently, overlooking the disparities between these areas, resulting in confusion between rain streaks and background parts, and inabilities to obtain effective interactions, ultimately resulting in suboptimal deraining outcomes. To address the above issue, we introduce the Region Transformer (Regformer), a novel SID method that underlines the importance of independently processing rain-affected and unaffected regions while considering their combined impact for high-quality image reconstruction. The crux of our method is the innovative Region Transformer Block (RTB), which integrates a Region Masked Attention (RMA) mechanism and a Mixed Gate Forward Block (MGFB). Our RTB is used for attention selection of rain-affected and unaffected regions and local modeling of mixed scales. The RMA generates attention maps tailored to these two regions and their interactions, enabling our model to capture comprehensive features essential for rain removal. To better recover high-frequency textures and capture more local details, we develop the MGFB as a compensation module to complete local mixed scale modeling. Extensive experiments demonstrate that our model reaches state-of-the-art performance, significantly improving the image deraining quality. Our code and trained models are publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Remove and recover: Deep end-to-end two-stage attention network for single-shot heavy rain removal. Neurocomputing, 481:216–227, 2022.
  2. Ovarnet: Towards open-vocabulary object attribute recognition. In CVPR, pp.  23518–23527, 2023a.
  3. Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: Toward a unified model. In CVPR, pp.  17632–17641, 2022.
  4. Learning a sparse transformer network for effective image deraining. In CVPR, pp.  1–10, 2023b.
  5. Single image de-raining using gan for accurate video surveillance. Intelligence Enabled Research: DoSIER, pp.  7–11, 2020.
  6. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, pp.  1–21, 2021.
  7. Restoring an image taken through a window covered with dirt or rain. In ICCV, pp.  633–640, 2013.
  8. Removing rain from single images via a deep detail network. In CVPR, pp.  3855–3863, 2017.
  9. Rain streak removal via dual graph convolutional network. In AAAI, pp.  1352–1360, 2021.
  10. Depth-attentional features for single-image rain removal. In CVPR, pp.  8022–8031, 2019.
  11. Memory oriented transfer learning for semi-supervised image deraining. In CVPR, pp.  7732–7741, 2021.
  12. Rain removal of single image based on directional gradient priors. Applied Sciences, 12(22):11628, 2022.
  13. Image-to-image translation with conditional adversarial networks. In CVPR, pp.  1125–1134, 2017.
  14. Multi-scale progressive fusion network for single image deraining. In CVPR, pp.  8346–8355, 2020.
  15. Magic ELF: image deraining meets association learning and transformer. In ACM MM, pp.  827–836, 2022.
  16. A review of remote sensing for environmental monitoring in china. Remote Sensing, 12(7):1130, 2020.
  17. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In ECCV, pp.  254–269, 2018.
  18. Rain streak removal using layer priors. In CVPR, pp.  2736–2744, 2016.
  19. Sequential dual attention network for rain streak removal in a single image. IEEE TIP, 29:9250–9265, 2020.
  20. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pp.  10012–10022, 2021.
  21. Decoupled weight decay regularization. In ICLR, pp.  1–18, 2019.
  22. Removing rain from a single image via discriminative sparse coding. In ICCV, pp.  3397–3405, 2015.
  23. Scope of validity of psnr in image/video quality assessment. Electronics Letters, 44(13):800–801, 2008.
  24. Attentive generative adversarial network for raindrop removal from a single image. In CVPR, pp.  2482–2491, 2018.
  25. Deep learning for seeing through window with raindrops. In ICCV, pp.  2463–2471, 2019.
  26. Progressive image deraining networks: A better and simpler baseline. In CVPR, pp.  3937–3946, 2019.
  27. A convolutional network for joint deraining and dehazing from a single image for autonomous driving in rain. In IROS, pp.  962–969, 2019.
  28. Attention is all you need. In NeurIPS, pp.  1–11, 2017.
  29. A model-driven deep neural network for single image rain removal. In CVPR, pp.  3103–3112, 2020.
  30. Spatial attentive single-image deraining with a high quality real rain dataset. In CVPR, pp.  12270–12279, 2019.
  31. Uformer: A general u-shaped transformer for image restoration. In CVPR, pp.  17683–17693, 2022.
  32. Image quality assessment: from error visibility to structural similarity. TIP, 13(4):600–612, 2004.
  33. Semi-supervised transfer learning for image rain removal. In CVPR, pp.  3877–3886, 2019.
  34. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking. IEEE TIP, pp.  4788–4801, 2021.
  35. Image de-raining transformer. IEEE TPAMI, pp.  1–18, 2022.
  36. Feature-aligned video raindrop removal with temporal constraints. IEEE TIP, 31:3440–3448, 2022.
  37. Deep joint rain detection and removal from a single image. In CVPR, pp.  1357–1366, 2017.
  38. Single image deraining: From model-based to data-driven and beyond. IEEE TPAMI, 43(11):4059–4077, 2020.
  39. Structure-preserving deraining with residue channel prior guidance. In ICCV, pp.  4238–4247, 2021.
  40. Multi-stage progressive image restoration. In CVPR, pp.  14821–14831, 2021.
  41. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pp.  5728–5739, 2022.
  42. He Zhang and Vishal M Patel. Density-aware single image de-raining using a multi-stream dense network. In CVPR, pp.  695–704, 2018.
  43. Image de-raining using a conditional generative adversarial network. IEEE TCSVT, 30(11):3943–3956, 2019.
  44. How can objects help action recognition? In CVPR, pp.  2353–2362, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now