Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining (2404.01547v1)

Published 2 Apr 2024 in cs.CV

Abstract: How to effectively explore multi-scale representations of rain streaks is important for image deraining. In contrast to existing Transformer-based methods that depend mostly on single-scale rain appearance, we develop an end-to-end multi-scale Transformer that leverages the potentially useful features in various scales to facilitate high-quality image reconstruction. To better explore the common degradation representations from spatially-varying rain streaks, we incorporate intra-scale implicit neural representations based on pixel coordinates with the degraded inputs in a closed-loop design, enabling the learned features to facilitate rain removal and improve the robustness of the model in complex scenarios. To ensure richer collaborative representation from different scales, we embed a simple yet effective inter-scale bidirectional feedback operation into our multi-scale Transformer by performing coarse-to-fine and fine-to-coarse information communication. Extensive experiments demonstrate that our approach, named as NeRD-Rain, performs favorably against the state-of-the-art ones on both synthetic and real-world benchmark datasets. The source code and trained models are available at https://github.com/cschenxiang/NeRD-Rain.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Not just streaks: Towards ground truth for single image deraining. In ECCV, pages 723–740, 2022.
  2. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In ICCV, pages 5855–5864, 2021.
  3. Crossvit: Cross-attention multi-scale vision transformer for image classification. In ICCV, pages 357–366, 2021a.
  4. Nerv: Neural representations for videos. NeurIPS, 34:21557–21568, 2021b.
  5. Pre-trained image processing transformer. In CVPR, pages 12299–12310, 2021c.
  6. Cascaded local implicit transformer for arbitrary-scale super-resolution. In CVPR, pages 18257–18267, 2023a.
  7. Unpaired deep image deraining using dual contrastive learning. In CVPR, pages 2017–2026, 2022.
  8. Learning a sparse transformer network for effective image deraining. In CVPR, pages 5896–5905, 2023b.
  9. Towards unified deep image deraining: A survey and a new benchmark. arXiv preprint arXiv:2310.03535, 2023c.
  10. Learning continuous image representation with local implicit image function. In CVPR, pages 8628–8638, 2021d.
  11. Learning implicit fields for generative shape modeling. In CVPR, pages 5939–5948, 2019.
  12. Rethinking coarse-to-fine approach in single image deblurring. In ICCV, pages 4641–4650, 2021.
  13. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2020.
  14. Multiscale vision transformers. In ICCV, pages 6824–6835, 2021.
  15. Removing rain from single images via a deep detail network. In CVPR, pages 3855–3863, 2017.
  16. Lightweight pyramid networks for image deraining. IEEE TNNLS, 31(6):1794–1807, 2019.
  17. Rain streak removal via dual graph convolutional network. In AAAI, pages 1352–1360, 2021.
  18. Shacira: Scalable hash-grid compression for implicit neural representations. In CVPR, pages 17513–17524, 2023.
  19. Joint convolutional analysis and synthesis sparse representation for single image layer separation. In ICCV, pages 1708–1716, 2017.
  20. From sky to the ground: A large-scale benchmark and simple baseline towards real rain removal. In CVPR, pages 12097–12107, 2023.
  21. Local implicit grid representations for 3d scenes. In CVPR, pages 6001–6010, 2020a.
  22. Multi-scale progressive fusion network for single image deraining. In CVPR, pages 8346–8355, 2020b.
  23. Focal frequency loss for image reconstruction and synthesis. In ICCV, pages 13919–13929, 2021.
  24. Automatic single-image-based rain streaks removal via image decomposition. IEEE TIP, 21(4):1742–1755, 2011.
  25. Zero-shot blind image denoising via implicit neural representations. arXiv preprint arXiv:2204.02405, 2022a.
  26. Mssnet: Multi-scale-stage network for single image deblurring. In ECCV, pages 524–539, 2022b.
  27. Adam: A method for stochastic optimization. In ICLR, 2014.
  28. Ffcv: Accelerating training by removing data bottlenecks. In CVPR, pages 12011–12020, 2023.
  29. Local texture estimator for implicit representation function. In CVPR, pages 1929–1938, 2022.
  30. Deep scale-space mining network for single image deraining. In CVPRW, pages 4276–4285, 2022.
  31. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In ECCV, pages 254–269, 2018.
  32. Rain streak removal using layer priors. In CVPR, pages 2736–2744, 2016.
  33. Scale-aware modulation meet transformer. In ICCV, 2023.
  34. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016.
  35. Removing rain from a single image via discriminative sparse coding. In ICCV, pages 3397–3405, 2015.
  36. Intriguing findings of frequency selection for image deblurring. In AAAI, pages 1905–1913, 2023.
  37. Pyramid attention network for image restoration. IJCV, pages 1–19, 2023.
  38. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  39. Neural image representations for multi-image fusion and layer separation. In ECCV, pages 216–232, 2022.
  40. Scope of validity of psnr in image/video quality assessment. Electronics Letters, 44(13):800–801, 2008.
  41. Single image defocus deblurring via implicit neural inverse kernels. In ICCV, pages 12600–12610, 2023.
  42. Progressive image deraining networks: A better and simpler baseline. In CVPR, pages 3937–3946, 2019.
  43. Ice: Implicit coordinate encoder for multiple image neural representation. IEEE TIP, 2023.
  44. Miner: Multiscale implicit neural representation. In ECCV, pages 318–333, 2022.
  45. Implicit neural representations for image compression. In ECCV, pages 74–91, 2022.
  46. Joint implicit image function for guided depth super-resolution. In ACM MM, pages 4390–4399, 2021.
  47. Scale-recurrent network for deep image deblurring. In CVPR, pages 8174–8182, 2018.
  48. Maxim: Multi-axis mlp for image processing. In CVPR, pages 5769–5780, 2022.
  49. Attention is all you need. NeurIPS, 30, 2017.
  50. A model-driven deep neural network for single image rain removal. In CVPR, pages 3103–3112, 2020a.
  51. Spatial attentive single-image deraining with a high quality real rain dataset. In CVPR, pages 12270–12279, 2019.
  52. A hierarchical approach for rain or snow removing in a single color image. IEEE TIP, 26(8):3936–3950, 2017.
  53. Rethinking image deraining via rain streaks and vapors. In ECCV, pages 367–382, 2020b.
  54. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600–612, 2004.
  55. Uformer: A general u-shaped transformer for image restoration. In CVPR, pages 17683–17693, 2022.
  56. Image de-raining transformer. IEEE TPAMI, 2022.
  57. Signal processing for implicit neural representations. NeurIPS, 35:13404–13418, 2022.
  58. Implicit neural representation for cooperative low-light image enhancement. In ICCV, 2023.
  59. Deep joint rain detection and removal from a single image. In CVPR, pages 1357–1366, 2017.
  60. Structure-preserving deraining with residue channel prior guidance. In ICCV, pages 4238–4247, 2021.
  61. Multi-stage progressive image restoration. In CVPR, pages 14821–14831, 2021.
  62. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pages 5728–5739, 2022.
  63. Density-aware single image de-raining using a multi-stream dense network. In CVPR, pages 695–704, 2018.
  64. Joint bi-layer optimization for single-image rain streak removal. In ICCV, pages 2526–2534, 2017.
Citations (9)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now