Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Plug-and-Play image restoration with Stochastic deNOising REgularization (2402.01779v2)

Published 1 Feb 2024 in eess.IV, cs.CV, cs.LG, and stat.ML

Abstract: Plug-and-Play (PnP) algorithms are a class of iterative algorithms that address image inverse problems by combining a physical model and a deep neural network for regularization. Even if they produce impressive image restoration results, these algorithms rely on a non-standard use of a denoiser on images that are less and less noisy along the iterations, which contrasts with recent algorithms based on Diffusion Models (DM), where the denoiser is applied only on re-noised images. We propose a new PnP framework, called Stochastic deNOising REgularization (SNORE), which applies the denoiser only on images with noise of the adequate level. It is based on an explicit stochastic regularization, which leads to a stochastic gradient descent algorithm to solve ill-posed inverse problems. A convergence analysis of this algorithm and its annealing extension is provided. Experimentally, we prove that SNORE is competitive with respect to state-of-the-art methods on deblurring and inpainting tasks, both quantitatively and qualitatively.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Ntire 2017 challenge on single image super-resolution: Dataset and study. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.  1122–1131, 2017. doi: 10.1109/CVPRW.2017.150.
  2. Benaïm, M. Dynamics of stochastic approximation algorithms. In Azéma, J., Émery, M., Ledoux, M., and Yor, M. (eds.), Séminaire de Probabilités XXXIII, pp.  1–68, Berlin, Heidelberg, 1999. Springer Berlin Heidelberg. ISBN 978-3-540-48407-3.
  3. Benaïm, M. Dynamics of stochastic approximation algorithms. In Seminaire de probabilites XXXIII, pp.  1–68. Springer, 2006.
  4. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Now Foundations and Trends, 2011.
  5. Verifying the union of manifolds hypothesis for image data, 2023.
  6. An inertial newton algorithm for deep learning, 2021.
  7. Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Transactions on image processing, 14(10):1479–1485, 2005.
  8. Plug-and-play admm for image restoration: Fixed point convergence and applications, 2016.
  9. Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5):1632–1648, 2006.
  10. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683–25696, 2022.
  11. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023.
  12. Plug-and-play split gibbs sampler: embedding deep generative priors in bayesian inference, 2023.
  13. Regularization by denoising via fixed-point projection (red-pro). SIAM Journal on Imaging Sciences, 14(3):1374–1406, 2021.
  14. Stochastic subgradient method converges at the rate o⁢(k−1/4)𝑜superscript𝑘14o(k^{-1/4})italic_o ( italic_k start_POSTSUPERSCRIPT - 1 / 4 end_POSTSUPERSCRIPT ) on weakly convex functions, 2018.
  15. De Bortoli, V. Convergence of denoising diffusion models under the manifold hypothesis, 2023.
  16. Diffusion schrödinger bridge with applications to score-based generative modeling, 2023.
  17. Mulog, or how to apply gaussian denoisers to multi-channel sar speckle reduction? IEEE Transactions on Image Processing, 26(9):4389–4403, 2017.
  18. Efron, B. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011.
  19. Testing the manifold hypothesis, 2013.
  20. Efficient bayesian computational imaging with a surrogate score-based prior. In NeurIPS 2023 Workshop on Deep Learning and Inverse Problems, 2023.
  21. Pnp-reg: Learned regularizing gradient for plug-and-play gradient descent, 2023.
  22. On the proof of fixed-point convergence for plug-and-play admm. IEEE Signal Processing Letters, 26(12):1817–1821, 2019.
  23. Stochastic first- and zeroth-order methods for nonconvex stochastic programming, 2013.
  24. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  25. Mixed noise removal via laplacian scale mixture modeling and nonlocal low-rank approximation. IEEE Transactions on Image Processing, 26(7):3171–3186, 2017.
  26. Gradient step denoiser for convergent plug-and-play, 2022a.
  27. Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization. In International Conference on Machine Learning, pp.  9483–9505. PMLR, 2022b.
  28. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems, 34:14938–14954, 2021.
  29. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593–23606, 2022.
  30. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  31. Bayesian imaging using plug and play priors: When langevin meets tweedie. SIAM Journal on Imaging Sciences, 15(2):701–737, 2022a.
  32. On maximum-a-posteriori estimation with plug & play priors and stochastic gradient descent, 2022b.
  33. Noise2Noise: Learning image restoration without clean data. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.  2965–2974. PMLR, 2018.
  34. Understanding and evaluating blind deconvolution algorithms. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.  1964–1971, 2009.
  35. Enhanced deep residual networks for single image super-resolution, 2017.
  36. Adversarial regularizers in inverse problems. Advances in neural information processing systems, 31, 2018.
  37. Ddgm: Solving inverse problems by diffusive denoising of gradient-based minimization, 2023.
  38. Waterloo Exploration Database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2):1004–1016, Feb. 2017.
  39. Mallat, S. A Wavelet Tour of Signal Processing. ScienceDirect, 1999.
  40. A variational perspective on solving inverse problems with diffusion models, 2023.
  41. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE International Conference on Computer Vision, volume 2, pp.  416–423 vol.2, 2001.
  42. Applications of a kushner and clark lemma to general classes of stochastic algorithms. IEEE Transactions on Information Theory, 30(2):140–151, 1984.
  43. prDeep: Robust phase retrieval with a flexible deep network. In Proc. 36th Int. Conf. Mach. Learn., pp.  3501–3510, 2018.
  44. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 21(12):4695–4708, 2012.
  45. Neal, R. M. Annealed importance sampling, 1998.
  46. Nikolova, M. A variational approach to remove outliers and impulse noise. Journal of Mathematical Imaging and Vision, 20, 2004.
  47. Nurminski, E. The quasigradient method for the solving of the nonlinear programming problems. Cybernetics, 9, 1973.
  48. Learning maximally monotone operators for image recovery, 2021.
  49. Dreamfusion: Text-to-3d using 2d diffusion, 2022.
  50. Multiscale structure guided diffusion for image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  10721–10733, 2023.
  51. Plug-and-play posterior sampling under mismatched measurement and prior models, 2023.
  52. The little engine that could: Regularization by denoising (red), 2017.
  53. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1):259–268, 1992. ISSN 0167-2789.
  54. Plug-and-Play Methods Provably Converge with Properly Trained Denoisers. In International Conference on Machine Learning, 2019.
  55. Closed-form diffusion models. arXiv preprint arXiv:2310.12395, 2023.
  56. Prior mismatch and adaptation in pnp-admm with a nonconvex convergence analysis, 2023.
  57. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
  58. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:1415–1428, 2021a.
  59. Score-based generative modeling through stochastic differential equations, 2021b.
  60. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, 2022.
  61. Plug-and-play priors for bright field electron tomography and sparse interpolation. IEEE Transactions on Computational Imaging, 2(4):408–423, 2016.
  62. Block coordinate regularization by denoising, 2019a.
  63. An online plug-and-play algorithm for regularized image reconstruction. IEEE Transactions on Computational Imaging, 5(3):395–408, 2019b.
  64. Provable probabilistic imaging using score-based generative priors, 2023.
  65. Asymptotic bias of stochastic gradient search, 2017.
  66. A Fast Stochastic Plug-and-Play ADMM for Imaging Inverse Problems, 2020.
  67. Robustness to spatially-correlated speckle in Plug-and-Play PolSAR despeckling. working paper or preprint, 2023.
  68. Plug-and-play priors for model based reconstruction. In IEEE Global Conference on Signal and Information Processing, pp.  945–948, 2013.
  69. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  12619–12629, 2023.
  70. Learning deep cnn denoiser prior for image restoration, 2017.
  71. Plug-and-play image restoration with deep denoiser prior, 2021.
  72. Fast single image super-resolution, 2016.
  73. Denoising diffusion models for plug-and-play image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.  1219–1229, 2023.
  74. From learning models of natural image patches to whole image restoration. In 2011 International Conference on Computer Vision, pp.  479–486, 2011.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now